2024-03-29T04:03:05.267 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-29T04:03:05.268 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_mchangir@teuthology --description fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/uninlining} --name mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi --archive /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629086 --verbose -- /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629086/orig.config.yaml 2024-03-29T04:03:05.314 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-29T04:03:05.406 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629086 branch: wip-mchangir-testing1-main-20240328-072412 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/uninlining} email: mchangir@redhat.com first_in_suite: false job_id: '7629086' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-mchangir-testing1-main-20240328-072412 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing1-main-20240328-072412 sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 owner: scheduled_mchangir@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8207 sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-mchangir-testing1-main-20240328-072412 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi022.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKCXo6Fu6f+uCk70spbH9SnoOQpq10bOkMvJsyVbHpskB9nuO/VnU7ECSDvwyeyjtUXCiDdkBiSlBW101K0GTAo= smithi151.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHDWNwZHqvmHe//FSX8MiigxaJK++ZfMmopww64yeuTn8TzDtScnW51x2HSUHK62BijHfDGyhbfDRhgiE8FiUoo= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_uninlining teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-29_02:01:30 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-29T04:03:05.406 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa; will attempt to use it 2024-03-29T04:03:05.407 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks 2024-03-29T04:03:05.407 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-29T04:03:05.408 INFO:teuthology.task.internal:Checking packages... 2024-03-29T04:03:05.432 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '69e2cd73c52284dfd5096c66b60c6e6b26a3c447' 2024-03-29T04:03:05.432 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-29T04:03:05.432 INFO:teuthology.packaging:ref: None 2024-03-29T04:03:05.432 INFO:teuthology.packaging:tag: None 2024-03-29T04:03:05.432 INFO:teuthology.packaging:branch: wip-mchangir-testing1-main-20240328-072412 2024-03-29T04:03:05.432 INFO:teuthology.packaging:sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 2024-03-29T04:03:05.432 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-mchangir-testing1-main-20240328-072412 2024-03-29T04:03:05.648 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2520-g69e2cd73-1jammy 2024-03-29T04:03:05.650 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-29T04:03:05.656 INFO:teuthology.task.internal:no buildpackages task found 2024-03-29T04:03:05.657 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-29T04:03:05.679 INFO:teuthology.task.internal:Saving configuration 2024-03-29T04:03:05.696 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-29T04:03:05.702 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T04:03:05.728 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629086', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 03:57:02.836702', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGR+uhU0DVVqdcvSAwmn7hDK4YREX7HFhlS0sbCUkGJr'} 2024-03-29T04:03:05.749 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629086', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 03:57:02.835208', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFjwJDMVarbHtjiUtXKIP+5u6ItZTbrUVu3PrfrDCa1N'} 2024-03-29T04:03:05.749 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-29T04:03:05.755 INFO:teuthology.task.internal:roles: ubuntu@smithi022.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-29T04:03:05.756 INFO:teuthology.task.internal:roles: ubuntu@smithi151.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-29T04:03:05.756 INFO:teuthology.run_tasks:Running task console_log... 2024-03-29T04:03:05.822 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd404b1d3a0>, signals=[15]) 2024-03-29T04:03:05.822 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-29T04:03:05.828 INFO:teuthology.task.internal:Opening connections... 2024-03-29T04:03:05.828 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi022.front.sepia.ceph.com 2024-03-29T04:03:05.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:03:05.910 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi151.front.sepia.ceph.com 2024-03-29T04:03:05.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:03:05.989 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-29T04:03:05.998 DEBUG:teuthology.orchestra.run.smithi022:> uname -m 2024-03-29T04:03:06.003 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2024-03-29T04:03:06.003 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2024-03-29T04:03:06.052 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-29T04:03:06.052 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="Ubuntu" 2024-03-29T04:03:06.052 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="22.04" 2024-03-29T04:03:06.052 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-29T04:03:06.052 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_CODENAME=jammy 2024-03-29T04:03:06.052 INFO:teuthology.orchestra.run.smithi022.stdout:ID=ubuntu 2024-03-29T04:03:06.052 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE=debian 2024-03-29T04:03:06.053 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-29T04:03:06.053 INFO:teuthology.orchestra.run.smithi022.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-29T04:03:06.053 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-29T04:03:06.053 INFO:teuthology.orchestra.run.smithi022.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-29T04:03:06.053 INFO:teuthology.orchestra.run.smithi022.stdout:UBUNTU_CODENAME=jammy 2024-03-29T04:03:06.053 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2024-03-29T04:03:06.078 DEBUG:teuthology.orchestra.run.smithi151:> uname -m 2024-03-29T04:03:06.083 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64 2024-03-29T04:03:06.083 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release 2024-03-29T04:03:06.132 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-29T04:03:06.133 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="Ubuntu" 2024-03-29T04:03:06.133 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="22.04" 2024-03-29T04:03:06.133 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-29T04:03:06.133 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_CODENAME=jammy 2024-03-29T04:03:06.133 INFO:teuthology.orchestra.run.smithi151.stdout:ID=ubuntu 2024-03-29T04:03:06.133 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE=debian 2024-03-29T04:03:06.133 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-29T04:03:06.133 INFO:teuthology.orchestra.run.smithi151.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-29T04:03:06.133 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-29T04:03:06.133 INFO:teuthology.orchestra.run.smithi151.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-29T04:03:06.133 INFO:teuthology.orchestra.run.smithi151.stdout:UBUNTU_CODENAME=jammy 2024-03-29T04:03:06.134 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2024-03-29T04:03:06.153 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-29T04:03:06.162 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-29T04:03:06.167 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-29T04:03:06.167 DEBUG:teuthology.orchestra.run.smithi022:> test '!' -e /home/ubuntu/cephtest 2024-03-29T04:03:06.170 DEBUG:teuthology.orchestra.run.smithi151:> test '!' -e /home/ubuntu/cephtest 2024-03-29T04:03:06.180 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-29T04:03:06.185 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-29T04:03:06.185 DEBUG:teuthology.orchestra.run.smithi022:> test -z $(ls -A /var/lib/ceph) 2024-03-29T04:03:06.217 DEBUG:teuthology.orchestra.run.smithi151:> test -z $(ls -A /var/lib/ceph) 2024-03-29T04:03:06.237 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-29T04:03:06.290 INFO:teuthology.run_tasks:Running task kernel... 2024-03-29T04:03:06.304 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-29T04:03:06.304 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-29T04:03:06.304 DEBUG:teuthology.orchestra.run.smithi022:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-29T04:03:06.305 DEBUG:teuthology.orchestra.run.smithi151:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-29T04:03:06.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:03:06.309 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-03-29T04:03:06.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:03:06.311 DEBUG:teuthology.orchestra.run.smithi151:> uname -r 2024-03-29T04:03:06.356 INFO:teuthology.orchestra.run.smithi022.stdout:5.15.0-56-generic 2024-03-29T04:03:06.357 INFO:teuthology.task.kernel:Running kernel on smithi022: 5.15.0-56-generic 2024-03-29T04:03:06.357 DEBUG:teuthology.orchestra.run.smithi022:> sudo apt-get clean 2024-03-29T04:03:06.358 INFO:teuthology.orchestra.run.smithi151.stdout:5.15.0-56-generic 2024-03-29T04:03:06.358 INFO:teuthology.task.kernel:Running kernel on smithi151: 5.15.0-56-generic 2024-03-29T04:03:06.358 DEBUG:teuthology.orchestra.run.smithi151:> sudo apt-get clean 2024-03-29T04:03:06.468 DEBUG:teuthology.orchestra.run.smithi151:> sudo apt-get update 2024-03-29T04:03:06.483 DEBUG:teuthology.orchestra.run.smithi022:> sudo apt-get update 2024-03-29T04:03:06.611 INFO:teuthology.orchestra.run.smithi151.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-29T04:03:06.643 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-29T04:03:06.665 INFO:teuthology.orchestra.run.smithi022.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-29T04:03:06.715 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-29T04:03:06.742 INFO:teuthology.orchestra.run.smithi022.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-29T04:03:06.754 INFO:teuthology.orchestra.run.smithi022.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-29T04:03:06.796 INFO:teuthology.orchestra.run.smithi151.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-29T04:03:06.823 INFO:teuthology.orchestra.run.smithi151.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [436 kB] 2024-03-29T04:03:06.885 INFO:teuthology.orchestra.run.smithi022.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,518 kB] 2024-03-29T04:03:06.906 INFO:teuthology.orchestra.run.smithi151.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,303 kB] 2024-03-29T04:03:06.967 INFO:teuthology.orchestra.run.smithi151.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [233 kB] 2024-03-29T04:03:06.970 INFO:teuthology.orchestra.run.smithi151.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-29T04:03:06.970 INFO:teuthology.orchestra.run.smithi151.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,616 kB] 2024-03-29T04:03:06.994 INFO:teuthology.orchestra.run.smithi151.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-29T04:03:06.994 INFO:teuthology.orchestra.run.smithi151.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [271 kB] 2024-03-29T04:03:06.998 INFO:teuthology.orchestra.run.smithi151.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-29T04:03:06.998 INFO:teuthology.orchestra.run.smithi151.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [599 kB] 2024-03-29T04:03:07.003 INFO:teuthology.orchestra.run.smithi151.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-03-29T04:03:07.012 INFO:teuthology.orchestra.run.smithi151.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-03-29T04:03:07.013 INFO:teuthology.orchestra.run.smithi151.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-29T04:03:07.013 INFO:teuthology.orchestra.run.smithi151.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-29T04:03:07.014 INFO:teuthology.orchestra.run.smithi151.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-29T04:03:07.018 INFO:teuthology.orchestra.run.smithi022.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [602 kB] 2024-03-29T04:03:07.028 INFO:teuthology.orchestra.run.smithi022.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [293 kB] 2024-03-29T04:03:07.032 INFO:teuthology.orchestra.run.smithi022.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-29T04:03:07.032 INFO:teuthology.orchestra.run.smithi022.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,644 kB] 2024-03-29T04:03:07.044 INFO:teuthology.orchestra.run.smithi151.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-29T04:03:07.044 INFO:teuthology.orchestra.run.smithi151.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-29T04:03:07.057 INFO:teuthology.orchestra.run.smithi022.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-29T04:03:07.057 INFO:teuthology.orchestra.run.smithi022.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [274 kB] 2024-03-29T04:03:07.061 INFO:teuthology.orchestra.run.smithi022.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-29T04:03:07.062 INFO:teuthology.orchestra.run.smithi022.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [698 kB] 2024-03-29T04:03:07.070 INFO:teuthology.orchestra.run.smithi022.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,060 kB] 2024-03-29T04:03:07.082 INFO:teuthology.orchestra.run.smithi022.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [241 kB] 2024-03-29T04:03:07.107 INFO:teuthology.orchestra.run.smithi022.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-29T04:03:07.108 INFO:teuthology.orchestra.run.smithi022.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [6,028 B] 2024-03-29T04:03:07.108 INFO:teuthology.orchestra.run.smithi022.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [49.6 kB] 2024-03-29T04:03:07.109 INFO:teuthology.orchestra.run.smithi022.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [12.0 kB] 2024-03-29T04:03:07.109 INFO:teuthology.orchestra.run.smithi022.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-29T04:03:07.110 INFO:teuthology.orchestra.run.smithi022.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-03-29T04:03:07.110 INFO:teuthology.orchestra.run.smithi022.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-03-29T04:03:07.114 INFO:teuthology.orchestra.run.smithi022.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-03-29T04:03:07.123 INFO:teuthology.orchestra.run.smithi022.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-29T04:03:07.141 INFO:teuthology.orchestra.run.smithi022.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-03-29T04:03:07.142 INFO:teuthology.orchestra.run.smithi022.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-03-29T04:03:07.142 INFO:teuthology.orchestra.run.smithi022.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-03-29T04:03:07.142 INFO:teuthology.orchestra.run.smithi022.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-29T04:03:07.158 INFO:teuthology.orchestra.run.smithi151.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-29T04:03:07.205 INFO:teuthology.orchestra.run.smithi022.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [436 kB] 2024-03-29T04:03:07.297 INFO:teuthology.orchestra.run.smithi151.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [602 kB] 2024-03-29T04:03:07.485 INFO:teuthology.orchestra.run.smithi022.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,303 kB] 2024-03-29T04:03:07.494 INFO:teuthology.orchestra.run.smithi151.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,518 kB] 2024-03-29T04:03:07.643 INFO:teuthology.orchestra.run.smithi151.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [293 kB] 2024-03-29T04:03:07.650 INFO:teuthology.orchestra.run.smithi022.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [233 kB] 2024-03-29T04:03:07.658 INFO:teuthology.orchestra.run.smithi151.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-29T04:03:07.659 INFO:teuthology.orchestra.run.smithi151.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-29T04:03:07.661 INFO:teuthology.orchestra.run.smithi151.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,644 kB] 2024-03-29T04:03:07.662 INFO:teuthology.orchestra.run.smithi022.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-29T04:03:07.663 INFO:teuthology.orchestra.run.smithi022.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-29T04:03:07.664 INFO:teuthology.orchestra.run.smithi022.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,616 kB] 2024-03-29T04:03:07.720 INFO:teuthology.orchestra.run.smithi151.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [274 kB] 2024-03-29T04:03:07.727 INFO:teuthology.orchestra.run.smithi151.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-29T04:03:07.744 INFO:teuthology.orchestra.run.smithi022.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [271 kB] 2024-03-29T04:03:07.752 INFO:teuthology.orchestra.run.smithi022.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-29T04:03:07.752 INFO:teuthology.orchestra.run.smithi022.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-03-29T04:03:07.775 INFO:teuthology.orchestra.run.smithi022.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [599 kB] 2024-03-29T04:03:07.891 INFO:teuthology.orchestra.run.smithi151.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [698 kB] 2024-03-29T04:03:07.953 INFO:teuthology.orchestra.run.smithi022.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-03-29T04:03:08.350 INFO:teuthology.orchestra.run.smithi022.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-29T04:03:08.356 INFO:teuthology.orchestra.run.smithi022.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-29T04:03:08.356 INFO:teuthology.orchestra.run.smithi022.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-29T04:03:08.371 INFO:teuthology.orchestra.run.smithi022.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-29T04:03:08.373 INFO:teuthology.orchestra.run.smithi022.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-29T04:03:08.474 INFO:teuthology.orchestra.run.smithi151.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,060 kB] 2024-03-29T04:03:08.585 INFO:teuthology.orchestra.run.smithi151.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [241 kB] 2024-03-29T04:03:08.605 INFO:teuthology.orchestra.run.smithi151.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-29T04:03:08.606 INFO:teuthology.orchestra.run.smithi151.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [49.6 kB] 2024-03-29T04:03:08.610 INFO:teuthology.orchestra.run.smithi151.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [6,028 B] 2024-03-29T04:03:08.610 INFO:teuthology.orchestra.run.smithi151.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [12.0 kB] 2024-03-29T04:03:08.611 INFO:teuthology.orchestra.run.smithi151.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-29T04:03:08.611 INFO:teuthology.orchestra.run.smithi151.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-03-29T04:03:08.616 INFO:teuthology.orchestra.run.smithi151.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-03-29T04:03:08.620 INFO:teuthology.orchestra.run.smithi151.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-03-29T04:03:08.785 INFO:teuthology.orchestra.run.smithi151.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-29T04:03:08.867 INFO:teuthology.orchestra.run.smithi151.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-03-29T04:03:08.974 INFO:teuthology.orchestra.run.smithi151.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-03-29T04:03:09.033 INFO:teuthology.orchestra.run.smithi151.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-03-29T04:03:09.067 INFO:teuthology.orchestra.run.smithi151.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-29T04:03:13.091 INFO:teuthology.orchestra.run.smithi022.stdout:Fetched 12.6 MB in 2s (6,725 kB/s) 2024-03-29T04:03:13.642 INFO:teuthology.orchestra.run.smithi151.stdout:Fetched 12.6 MB in 3s (4,961 kB/s) 2024-03-29T04:03:14.460 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2024-03-29T04:03:14.478 DEBUG:teuthology.orchestra.run.smithi022:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-29T04:03:14.541 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2024-03-29T04:03:14.742 INFO:teuthology.orchestra.run.smithi022.stdout:Building dependency tree... 2024-03-29T04:03:14.743 INFO:teuthology.orchestra.run.smithi022.stdout:Reading state information... 2024-03-29T04:03:14.922 INFO:teuthology.orchestra.run.smithi022.stdout:The following packages were automatically installed and are no longer required: 2024-03-29T04:03:14.922 INFO:teuthology.orchestra.run.smithi022.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-29T04:03:14.922 INFO:teuthology.orchestra.run.smithi022.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-29T04:03:14.923 INFO:teuthology.orchestra.run.smithi022.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-29T04:03:14.923 INFO:teuthology.orchestra.run.smithi022.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-29T04:03:14.923 INFO:teuthology.orchestra.run.smithi022.stdout: python2.7-minimal 2024-03-29T04:03:14.923 INFO:teuthology.orchestra.run.smithi022.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-29T04:03:14.924 INFO:teuthology.orchestra.run.smithi022.stdout:The following additional packages will be installed: 2024-03-29T04:03:14.924 INFO:teuthology.orchestra.run.smithi022.stdout: linux-generic linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-29T04:03:14.925 INFO:teuthology.orchestra.run.smithi022.stdout: linux-headers-generic linux-image-5.15.0-101-generic 2024-03-29T04:03:14.925 INFO:teuthology.orchestra.run.smithi022.stdout: linux-modules-5.15.0-101-generic linux-modules-extra-5.15.0-101-generic 2024-03-29T04:03:14.926 INFO:teuthology.orchestra.run.smithi022.stdout:Suggested packages: 2024-03-29T04:03:14.926 INFO:teuthology.orchestra.run.smithi022.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-29T04:03:14.926 INFO:teuthology.orchestra.run.smithi022.stdout:Recommended packages: 2024-03-29T04:03:14.927 INFO:teuthology.orchestra.run.smithi022.stdout: thermald 2024-03-29T04:03:14.968 INFO:teuthology.orchestra.run.smithi022.stdout:The following NEW packages will be installed: 2024-03-29T04:03:14.968 INFO:teuthology.orchestra.run.smithi022.stdout: linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-29T04:03:14.968 INFO:teuthology.orchestra.run.smithi022.stdout: linux-image-5.15.0-101-generic linux-modules-5.15.0-101-generic 2024-03-29T04:03:14.968 INFO:teuthology.orchestra.run.smithi022.stdout: linux-modules-extra-5.15.0-101-generic 2024-03-29T04:03:14.969 INFO:teuthology.orchestra.run.smithi022.stdout:The following packages will be upgraded: 2024-03-29T04:03:14.970 INFO:teuthology.orchestra.run.smithi022.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-29T04:03:15.029 INFO:teuthology.orchestra.run.smithi022.stdout:3 upgraded, 5 newly installed, 0 to remove and 311 not upgraded. 2024-03-29T04:03:15.030 INFO:teuthology.orchestra.run.smithi022.stdout:Need to get 113 MB of archives. 2024-03-29T04:03:15.030 INFO:teuthology.orchestra.run.smithi022.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-29T04:03:15.030 INFO:teuthology.orchestra.run.smithi022.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-101-generic amd64 5.15.0-101.111 [22.6 MB] 2024-03-29T04:03:15.100 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-03-29T04:03:15.120 DEBUG:teuthology.orchestra.run.smithi151:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-29T04:03:15.186 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-03-29T04:03:15.389 INFO:teuthology.orchestra.run.smithi151.stdout:Building dependency tree... 2024-03-29T04:03:15.390 INFO:teuthology.orchestra.run.smithi151.stdout:Reading state information... 2024-03-29T04:03:15.560 INFO:teuthology.orchestra.run.smithi151.stdout:The following packages were automatically installed and are no longer required: 2024-03-29T04:03:15.561 INFO:teuthology.orchestra.run.smithi151.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-29T04:03:15.561 INFO:teuthology.orchestra.run.smithi151.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-29T04:03:15.561 INFO:teuthology.orchestra.run.smithi151.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-29T04:03:15.561 INFO:teuthology.orchestra.run.smithi151.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-29T04:03:15.561 INFO:teuthology.orchestra.run.smithi151.stdout: python2.7-minimal 2024-03-29T04:03:15.561 INFO:teuthology.orchestra.run.smithi151.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-29T04:03:15.563 INFO:teuthology.orchestra.run.smithi151.stdout:The following additional packages will be installed: 2024-03-29T04:03:15.563 INFO:teuthology.orchestra.run.smithi151.stdout: linux-generic linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-29T04:03:15.563 INFO:teuthology.orchestra.run.smithi151.stdout: linux-headers-generic linux-image-5.15.0-101-generic 2024-03-29T04:03:15.563 INFO:teuthology.orchestra.run.smithi151.stdout: linux-modules-5.15.0-101-generic linux-modules-extra-5.15.0-101-generic 2024-03-29T04:03:15.564 INFO:teuthology.orchestra.run.smithi151.stdout:Suggested packages: 2024-03-29T04:03:15.564 INFO:teuthology.orchestra.run.smithi151.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-29T04:03:15.564 INFO:teuthology.orchestra.run.smithi151.stdout:Recommended packages: 2024-03-29T04:03:15.564 INFO:teuthology.orchestra.run.smithi151.stdout: thermald 2024-03-29T04:03:15.617 INFO:teuthology.orchestra.run.smithi151.stdout:The following NEW packages will be installed: 2024-03-29T04:03:15.617 INFO:teuthology.orchestra.run.smithi151.stdout: linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-29T04:03:15.617 INFO:teuthology.orchestra.run.smithi151.stdout: linux-image-5.15.0-101-generic linux-modules-5.15.0-101-generic 2024-03-29T04:03:15.618 INFO:teuthology.orchestra.run.smithi151.stdout: linux-modules-extra-5.15.0-101-generic 2024-03-29T04:03:15.619 INFO:teuthology.orchestra.run.smithi151.stdout:The following packages will be upgraded: 2024-03-29T04:03:15.620 INFO:teuthology.orchestra.run.smithi151.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-29T04:03:15.800 INFO:teuthology.orchestra.run.smithi151.stdout:3 upgraded, 5 newly installed, 0 to remove and 311 not upgraded. 2024-03-29T04:03:15.800 INFO:teuthology.orchestra.run.smithi151.stdout:Need to get 113 MB of archives. 2024-03-29T04:03:15.800 INFO:teuthology.orchestra.run.smithi151.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-29T04:03:15.800 INFO:teuthology.orchestra.run.smithi151.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-101-generic amd64 5.15.0-101.111 [22.6 MB] 2024-03-29T04:03:15.943 INFO:teuthology.orchestra.run.smithi022.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-101-generic amd64 5.15.0-101.111 [11.5 MB] 2024-03-29T04:03:16.256 INFO:teuthology.orchestra.run.smithi022.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-101-generic amd64 5.15.0-101.111 [63.9 MB] 2024-03-29T04:03:17.365 INFO:teuthology.orchestra.run.smithi022.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.101.98 [1,698 B] 2024-03-29T04:03:17.365 INFO:teuthology.orchestra.run.smithi022.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.101.98 [2,486 B] 2024-03-29T04:03:17.365 INFO:teuthology.orchestra.run.smithi022.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101 all 5.15.0-101.111 [12.3 MB] 2024-03-29T04:03:17.418 INFO:teuthology.orchestra.run.smithi151.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-101-generic amd64 5.15.0-101.111 [11.5 MB] 2024-03-29T04:03:17.503 INFO:teuthology.orchestra.run.smithi022.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101-generic amd64 5.15.0-101.111 [2,876 kB] 2024-03-29T04:03:17.531 INFO:teuthology.orchestra.run.smithi022.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.101.98 [2,332 B] 2024-03-29T04:03:17.770 INFO:teuthology.orchestra.run.smithi151.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-101-generic amd64 5.15.0-101.111 [63.9 MB] 2024-03-29T04:03:17.937 INFO:teuthology.orchestra.run.smithi022.stdout:Fetched 113 MB in 3s (44.3 MB/s) 2024-03-29T04:03:18.090 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package linux-modules-5.15.0-101-generic. 2024-03-29T04:03:20.125 INFO:teuthology.orchestra.run.smithi022.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-29T04:03:20.129 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../0-linux-modules-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-29T04:03:20.246 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-29T04:03:20.340 INFO:teuthology.orchestra.run.smithi151.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.101.98 [1,698 B] 2024-03-29T04:03:20.341 INFO:teuthology.orchestra.run.smithi151.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.101.98 [2,486 B] 2024-03-29T04:03:20.341 INFO:teuthology.orchestra.run.smithi151.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101 all 5.15.0-101.111 [12.3 MB] 2024-03-29T04:03:21.540 INFO:teuthology.orchestra.run.smithi151.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101-generic amd64 5.15.0-101.111 [2,876 kB] 2024-03-29T04:03:21.661 INFO:teuthology.orchestra.run.smithi151.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.101.98 [2,332 B] 2024-03-29T04:03:22.052 INFO:teuthology.orchestra.run.smithi151.stdout:Fetched 113 MB in 6s (18.7 MB/s) 2024-03-29T04:03:22.215 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package linux-modules-5.15.0-101-generic. 2024-03-29T04:03:22.440 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package linux-image-5.15.0-101-generic. 2024-03-29T04:03:22.462 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../1-linux-image-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-29T04:03:22.562 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-29T04:03:23.010 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-101-generic. 2024-03-29T04:03:23.031 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-29T04:03:23.060 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-29T04:03:24.255 INFO:teuthology.orchestra.run.smithi151.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-29T04:03:24.259 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../0-linux-modules-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-29T04:03:24.329 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-29T04:03:26.538 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package linux-image-5.15.0-101-generic. 2024-03-29T04:03:26.559 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../1-linux-image-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-29T04:03:26.650 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-29T04:03:27.100 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-101-generic. 2024-03-29T04:03:27.120 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-29T04:03:27.158 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-29T04:03:29.625 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../3-linux-generic_5.15.0.101.98_amd64.deb ... 2024-03-29T04:03:29.743 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-29T04:03:30.077 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.101.98_amd64.deb ... 2024-03-29T04:03:30.221 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-image-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-29T04:03:30.516 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package linux-headers-5.15.0-101. 2024-03-29T04:03:30.539 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../5-linux-headers-5.15.0-101_5.15.0-101.111_all.deb ... 2024-03-29T04:03:30.582 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-29T04:03:33.482 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../3-linux-generic_5.15.0.101.98_amd64.deb ... 2024-03-29T04:03:33.609 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-29T04:03:33.943 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.101.98_amd64.deb ... 2024-03-29T04:03:34.071 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-image-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-29T04:03:34.341 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package linux-headers-5.15.0-101. 2024-03-29T04:03:34.352 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../5-linux-headers-5.15.0-101_5.15.0-101.111_all.deb ... 2024-03-29T04:03:34.382 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-29T04:03:36.577 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package linux-headers-5.15.0-101-generic. 2024-03-29T04:03:36.614 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../6-linux-headers-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-29T04:03:36.651 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-29T04:03:39.440 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.101.98_amd64.deb ... 2024-03-29T04:03:39.570 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking linux-headers-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-29T04:03:39.912 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-29T04:03:40.033 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-29T04:03:40.159 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-headers-generic (5.15.0.101.98) ... 2024-03-29T04:03:40.184 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package linux-headers-5.15.0-101-generic. 2024-03-29T04:03:40.221 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../6-linux-headers-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-29T04:03:40.258 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-29T04:03:40.288 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-29T04:03:41.642 INFO:teuthology.orchestra.run.smithi022.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-29T04:03:41.642 INFO:teuthology.orchestra.run.smithi022.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-29T04:03:41.642 INFO:teuthology.orchestra.run.smithi022.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-101-generic 2024-03-29T04:03:41.642 INFO:teuthology.orchestra.run.smithi022.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-101-generic 2024-03-29T04:03:41.790 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-29T04:03:43.341 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-29T04:03:43.561 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.101.98_amd64.deb ... 2024-03-29T04:03:43.686 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-headers-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-29T04:03:44.020 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-29T04:03:44.124 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-29T04:03:44.225 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-headers-generic (5.15.0.101.98) ... 2024-03-29T04:03:44.346 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-29T04:03:44.776 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-image-generic (5.15.0.101.98) ... 2024-03-29T04:03:44.901 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up linux-generic (5.15.0.101.98) ... 2024-03-29T04:03:45.028 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-29T04:03:45.135 INFO:teuthology.orchestra.run.smithi022.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-29T04:03:45.135 INFO:teuthology.orchestra.run.smithi022.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-101-generic 2024-03-29T04:03:45.638 INFO:teuthology.orchestra.run.smithi151.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-29T04:03:45.638 INFO:teuthology.orchestra.run.smithi151.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-29T04:03:45.638 INFO:teuthology.orchestra.run.smithi151.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-101-generic 2024-03-29T04:03:45.638 INFO:teuthology.orchestra.run.smithi151.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-101-generic 2024-03-29T04:03:45.798 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-29T04:03:47.232 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-29T04:03:49.142 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-image-generic (5.15.0.101.98) ... 2024-03-29T04:03:49.251 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-generic (5.15.0.101.98) ... 2024-03-29T04:03:49.377 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-29T04:03:49.447 INFO:teuthology.orchestra.run.smithi151.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-29T04:03:49.447 INFO:teuthology.orchestra.run.smithi151.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-101-generic 2024-03-29T04:04:01.328 INFO:teuthology.orchestra.run.smithi022.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-29T04:04:01.329 INFO:teuthology.orchestra.run.smithi022.stdout:Sourcing file `/etc/default/grub' 2024-03-29T04:04:01.342 INFO:teuthology.orchestra.run.smithi022.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-29T04:04:01.358 INFO:teuthology.orchestra.run.smithi022.stdout:Generating grub configuration file ... 2024-03-29T04:04:01.690 INFO:teuthology.orchestra.run.smithi022.stdout:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-29T04:04:01.710 INFO:teuthology.orchestra.run.smithi022.stdout:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-29T04:04:01.986 INFO:teuthology.orchestra.run.smithi022.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-29T04:04:01.992 INFO:teuthology.orchestra.run.smithi022.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-29T04:04:02.064 INFO:teuthology.orchestra.run.smithi022.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-29T04:04:02.070 INFO:teuthology.orchestra.run.smithi022.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-29T04:04:02.234 INFO:teuthology.orchestra.run.smithi022.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-29T04:04:02.234 INFO:teuthology.orchestra.run.smithi022.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-29T04:04:02.235 INFO:teuthology.orchestra.run.smithi022.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-29T04:04:02.286 INFO:teuthology.orchestra.run.smithi022.stdout:done 2024-03-29T04:04:02.751 DEBUG:teuthology.orchestra.run.smithi022:> dpkg -s linux-image-generic 2024-03-29T04:04:02.779 INFO:teuthology.orchestra.run.smithi022.stdout:Package: linux-image-generic 2024-03-29T04:04:02.779 INFO:teuthology.orchestra.run.smithi022.stdout:Status: install ok installed 2024-03-29T04:04:02.779 INFO:teuthology.orchestra.run.smithi022.stdout:Priority: optional 2024-03-29T04:04:02.779 INFO:teuthology.orchestra.run.smithi022.stdout:Section: kernel 2024-03-29T04:04:02.780 INFO:teuthology.orchestra.run.smithi022.stdout:Installed-Size: 21 2024-03-29T04:04:02.780 INFO:teuthology.orchestra.run.smithi022.stdout:Maintainer: Ubuntu Kernel Team 2024-03-29T04:04:02.780 INFO:teuthology.orchestra.run.smithi022.stdout:Architecture: amd64 2024-03-29T04:04:02.780 INFO:teuthology.orchestra.run.smithi022.stdout:Source: linux-meta 2024-03-29T04:04:02.780 INFO:teuthology.orchestra.run.smithi022.stdout:Version: 5.15.0.101.98 2024-03-29T04:04:02.780 INFO:teuthology.orchestra.run.smithi022.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-29T04:04:02.780 INFO:teuthology.orchestra.run.smithi022.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-29T04:04:02.780 INFO:teuthology.orchestra.run.smithi022.stdout:Recommends: thermald 2024-03-29T04:04:02.780 INFO:teuthology.orchestra.run.smithi022.stdout:Description: Generic Linux kernel image 2024-03-29T04:04:02.780 INFO:teuthology.orchestra.run.smithi022.stdout: This package will always depend on the latest generic kernel image 2024-03-29T04:04:02.780 INFO:teuthology.orchestra.run.smithi022.stdout: available. 2024-03-29T04:04:02.780 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-101-generic 2024-03-29T04:04:02.781 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-29T04:04:02.781 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-29T04:04:02.781 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-29T04:04:02.781 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi022.front.sepia.ceph.com, path=None, version=distro) 2024-03-29T04:04:02.781 DEBUG:teuthology.orchestra.run.smithi022:> sudo apt-get clean 2024-03-29T04:04:02.971 DEBUG:teuthology.orchestra.run.smithi022:> sudo apt-get update 2024-03-29T04:04:03.132 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-29T04:04:03.301 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-29T04:04:03.396 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-29T04:04:03.453 INFO:teuthology.orchestra.run.smithi151.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-29T04:04:03.453 INFO:teuthology.orchestra.run.smithi151.stdout:Sourcing file `/etc/default/grub' 2024-03-29T04:04:03.462 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-29T04:04:03.470 INFO:teuthology.orchestra.run.smithi151.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-29T04:04:03.487 INFO:teuthology.orchestra.run.smithi151.stdout:Generating grub configuration file ... 2024-03-29T04:04:03.757 INFO:teuthology.orchestra.run.smithi151.stdout:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-29T04:04:03.770 INFO:teuthology.orchestra.run.smithi151.stdout:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-29T04:04:04.054 INFO:teuthology.orchestra.run.smithi151.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-29T04:04:04.059 INFO:teuthology.orchestra.run.smithi151.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-29T04:04:04.126 INFO:teuthology.orchestra.run.smithi151.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-29T04:04:04.132 INFO:teuthology.orchestra.run.smithi151.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-29T04:04:04.287 INFO:teuthology.orchestra.run.smithi151.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-29T04:04:04.288 INFO:teuthology.orchestra.run.smithi151.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-29T04:04:04.288 INFO:teuthology.orchestra.run.smithi151.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-29T04:04:04.336 INFO:teuthology.orchestra.run.smithi151.stdout:done 2024-03-29T04:04:04.560 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2024-03-29T04:04:04.579 DEBUG:teuthology.orchestra.run.smithi022:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-29T04:04:04.632 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2024-03-29T04:04:04.778 DEBUG:teuthology.orchestra.run.smithi151:> dpkg -s linux-image-generic 2024-03-29T04:04:04.802 INFO:teuthology.orchestra.run.smithi151.stdout:Package: linux-image-generic 2024-03-29T04:04:04.803 INFO:teuthology.orchestra.run.smithi151.stdout:Status: install ok installed 2024-03-29T04:04:04.803 INFO:teuthology.orchestra.run.smithi151.stdout:Priority: optional 2024-03-29T04:04:04.803 INFO:teuthology.orchestra.run.smithi151.stdout:Section: kernel 2024-03-29T04:04:04.803 INFO:teuthology.orchestra.run.smithi151.stdout:Installed-Size: 21 2024-03-29T04:04:04.803 INFO:teuthology.orchestra.run.smithi151.stdout:Maintainer: Ubuntu Kernel Team 2024-03-29T04:04:04.803 INFO:teuthology.orchestra.run.smithi151.stdout:Architecture: amd64 2024-03-29T04:04:04.803 INFO:teuthology.orchestra.run.smithi151.stdout:Source: linux-meta 2024-03-29T04:04:04.803 INFO:teuthology.orchestra.run.smithi151.stdout:Version: 5.15.0.101.98 2024-03-29T04:04:04.803 INFO:teuthology.orchestra.run.smithi151.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-29T04:04:04.803 INFO:teuthology.orchestra.run.smithi151.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-29T04:04:04.803 INFO:teuthology.orchestra.run.smithi151.stdout:Recommends: thermald 2024-03-29T04:04:04.803 INFO:teuthology.orchestra.run.smithi151.stdout:Description: Generic Linux kernel image 2024-03-29T04:04:04.803 INFO:teuthology.orchestra.run.smithi151.stdout: This package will always depend on the latest generic kernel image 2024-03-29T04:04:04.804 INFO:teuthology.orchestra.run.smithi151.stdout: available. 2024-03-29T04:04:04.804 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-101-generic 2024-03-29T04:04:04.804 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-29T04:04:04.804 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-29T04:04:04.804 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-29T04:04:04.804 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi151.front.sepia.ceph.com, path=None, version=distro) 2024-03-29T04:04:04.804 DEBUG:teuthology.orchestra.run.smithi151:> sudo apt-get clean 2024-03-29T04:04:04.819 INFO:teuthology.orchestra.run.smithi022.stdout:Building dependency tree... 2024-03-29T04:04:04.820 INFO:teuthology.orchestra.run.smithi022.stdout:Reading state information... 2024-03-29T04:04:04.944 DEBUG:teuthology.orchestra.run.smithi151:> sudo apt-get update 2024-03-29T04:04:04.979 INFO:teuthology.orchestra.run.smithi022.stdout:linux-image-generic is already the newest version (5.15.0.101.98). 2024-03-29T04:04:04.980 INFO:teuthology.orchestra.run.smithi022.stdout:The following packages were automatically installed and are no longer required: 2024-03-29T04:04:04.980 INFO:teuthology.orchestra.run.smithi022.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-29T04:04:04.980 INFO:teuthology.orchestra.run.smithi022.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-29T04:04:04.980 INFO:teuthology.orchestra.run.smithi022.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-29T04:04:04.980 INFO:teuthology.orchestra.run.smithi022.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-29T04:04:04.980 INFO:teuthology.orchestra.run.smithi022.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-29T04:04:04.980 INFO:teuthology.orchestra.run.smithi022.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-29T04:04:05.020 INFO:teuthology.orchestra.run.smithi022.stdout:0 upgraded, 0 newly installed, 0 to remove and 311 not upgraded. 2024-03-29T04:04:05.022 DEBUG:teuthology.orchestra.run.smithi022:> dpkg -s linux-image-generic 2024-03-29T04:04:05.062 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-29T04:04:05.062 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-29T04:04:05.072 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-29T04:04:05.079 INFO:teuthology.orchestra.run.smithi022.stdout:Package: linux-image-generic 2024-03-29T04:04:05.079 INFO:teuthology.orchestra.run.smithi022.stdout:Status: install ok installed 2024-03-29T04:04:05.079 INFO:teuthology.orchestra.run.smithi022.stdout:Priority: optional 2024-03-29T04:04:05.079 INFO:teuthology.orchestra.run.smithi022.stdout:Section: kernel 2024-03-29T04:04:05.079 INFO:teuthology.orchestra.run.smithi022.stdout:Installed-Size: 21 2024-03-29T04:04:05.079 INFO:teuthology.orchestra.run.smithi022.stdout:Maintainer: Ubuntu Kernel Team 2024-03-29T04:04:05.079 INFO:teuthology.orchestra.run.smithi022.stdout:Architecture: amd64 2024-03-29T04:04:05.079 INFO:teuthology.orchestra.run.smithi022.stdout:Source: linux-meta 2024-03-29T04:04:05.079 INFO:teuthology.orchestra.run.smithi022.stdout:Version: 5.15.0.101.98 2024-03-29T04:04:05.079 INFO:teuthology.orchestra.run.smithi022.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-29T04:04:05.079 INFO:teuthology.orchestra.run.smithi022.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-29T04:04:05.080 INFO:teuthology.orchestra.run.smithi022.stdout:Recommends: thermald 2024-03-29T04:04:05.080 INFO:teuthology.orchestra.run.smithi022.stdout:Description: Generic Linux kernel image 2024-03-29T04:04:05.080 INFO:teuthology.orchestra.run.smithi022.stdout: This package will always depend on the latest generic kernel image 2024-03-29T04:04:05.080 INFO:teuthology.orchestra.run.smithi022.stdout: available. 2024-03-29T04:04:05.080 DEBUG:teuthology.orchestra.run.smithi022:> mktemp 2024-03-29T04:04:05.091 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-29T04:04:05.127 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.dtxZyEhbZz 2024-03-29T04:04:05.128 DEBUG:teuthology.orchestra.run.smithi022:> sudo cp /boot/grub/grub.cfg /tmp/tmp.dtxZyEhbZz 2024-03-29T04:04:05.182 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.dtxZyEhbZz 2024-03-29T04:04:05.298 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.dtxZyEhbZz is 10KB 2024-03-29T04:04:05.315 DEBUG:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.dtxZyEhbZz 2024-03-29T04:04:05.321 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-29T04:04:05.376 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:04:05.376 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-29T04:04:05.376 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-29T04:04:05.450 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-101-generic 2024-03-29T04:04:05.450 DEBUG:teuthology.orchestra.run.smithi022:> sudo update-grub 2024-03-29T04:04:06.436 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-03-29T04:04:06.454 DEBUG:teuthology.orchestra.run.smithi151:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-29T04:04:06.514 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-03-29T04:04:06.675 INFO:teuthology.orchestra.run.smithi022.stderr:Sourcing file `/etc/default/grub' 2024-03-29T04:04:06.676 INFO:teuthology.orchestra.run.smithi022.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-29T04:04:06.678 INFO:teuthology.orchestra.run.smithi022.stderr:Generating grub configuration file ... 2024-03-29T04:04:06.712 INFO:teuthology.orchestra.run.smithi151.stdout:Building dependency tree... 2024-03-29T04:04:06.712 INFO:teuthology.orchestra.run.smithi151.stdout:Reading state information... 2024-03-29T04:04:06.855 INFO:teuthology.orchestra.run.smithi022.stderr:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-29T04:04:06.862 INFO:teuthology.orchestra.run.smithi022.stderr:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-29T04:04:06.880 INFO:teuthology.orchestra.run.smithi151.stdout:linux-image-generic is already the newest version (5.15.0.101.98). 2024-03-29T04:04:06.880 INFO:teuthology.orchestra.run.smithi151.stdout:The following packages were automatically installed and are no longer required: 2024-03-29T04:04:06.880 INFO:teuthology.orchestra.run.smithi151.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-29T04:04:06.880 INFO:teuthology.orchestra.run.smithi151.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-29T04:04:06.880 INFO:teuthology.orchestra.run.smithi151.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-29T04:04:06.881 INFO:teuthology.orchestra.run.smithi151.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-29T04:04:06.881 INFO:teuthology.orchestra.run.smithi151.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-29T04:04:06.881 INFO:teuthology.orchestra.run.smithi151.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-29T04:04:06.919 INFO:teuthology.orchestra.run.smithi151.stdout:0 upgraded, 0 newly installed, 0 to remove and 311 not upgraded. 2024-03-29T04:04:06.921 DEBUG:teuthology.orchestra.run.smithi151:> dpkg -s linux-image-generic 2024-03-29T04:04:06.933 INFO:teuthology.orchestra.run.smithi151.stdout:Package: linux-image-generic 2024-03-29T04:04:06.933 INFO:teuthology.orchestra.run.smithi151.stdout:Status: install ok installed 2024-03-29T04:04:06.933 INFO:teuthology.orchestra.run.smithi151.stdout:Priority: optional 2024-03-29T04:04:06.933 INFO:teuthology.orchestra.run.smithi151.stdout:Section: kernel 2024-03-29T04:04:06.934 INFO:teuthology.orchestra.run.smithi151.stdout:Installed-Size: 21 2024-03-29T04:04:06.934 INFO:teuthology.orchestra.run.smithi151.stdout:Maintainer: Ubuntu Kernel Team 2024-03-29T04:04:06.934 INFO:teuthology.orchestra.run.smithi151.stdout:Architecture: amd64 2024-03-29T04:04:06.934 INFO:teuthology.orchestra.run.smithi151.stdout:Source: linux-meta 2024-03-29T04:04:06.934 INFO:teuthology.orchestra.run.smithi151.stdout:Version: 5.15.0.101.98 2024-03-29T04:04:06.934 INFO:teuthology.orchestra.run.smithi151.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-29T04:04:06.934 INFO:teuthology.orchestra.run.smithi151.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-29T04:04:06.934 INFO:teuthology.orchestra.run.smithi151.stdout:Recommends: thermald 2024-03-29T04:04:06.934 INFO:teuthology.orchestra.run.smithi151.stdout:Description: Generic Linux kernel image 2024-03-29T04:04:06.934 INFO:teuthology.orchestra.run.smithi151.stdout: This package will always depend on the latest generic kernel image 2024-03-29T04:04:06.934 INFO:teuthology.orchestra.run.smithi151.stdout: available. 2024-03-29T04:04:06.935 DEBUG:teuthology.orchestra.run.smithi151:> mktemp 2024-03-29T04:04:06.979 INFO:teuthology.orchestra.run.smithi151.stdout:/tmp/tmp.KsJngLBH6r 2024-03-29T04:04:06.980 DEBUG:teuthology.orchestra.run.smithi151:> sudo cp /boot/grub/grub.cfg /tmp/tmp.KsJngLBH6r 2024-03-29T04:04:07.033 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0666 /tmp/tmp.KsJngLBH6r 2024-03-29T04:04:07.084 INFO:teuthology.orchestra.run.smithi022.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-29T04:04:07.090 INFO:teuthology.orchestra.run.smithi022.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-29T04:04:07.131 INFO:teuthology.orchestra.run.smithi022.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-29T04:04:07.137 INFO:teuthology.orchestra.run.smithi022.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-29T04:04:07.146 DEBUG:teuthology.orchestra.remote:smithi151:/tmp/tmp.KsJngLBH6r is 10KB 2024-03-29T04:04:07.163 DEBUG:teuthology.orchestra.run.smithi151:> rm -fr /tmp/tmp.KsJngLBH6r 2024-03-29T04:04:07.168 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-29T04:04:07.220 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:04:07.220 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-29T04:04:07.220 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-29T04:04:07.255 INFO:teuthology.orchestra.run.smithi022.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-29T04:04:07.255 INFO:teuthology.orchestra.run.smithi022.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-29T04:04:07.255 INFO:teuthology.orchestra.run.smithi022.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-29T04:04:07.266 INFO:teuthology.orchestra.run.smithi022.stderr:done 2024-03-29T04:04:07.277 DEBUG:teuthology.orchestra.run.smithi022:> sudo shutdown -r now 2024-03-29T04:04:07.287 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-101-generic 2024-03-29T04:04:07.287 DEBUG:teuthology.orchestra.run.smithi151:> sudo update-grub 2024-03-29T04:04:08.392 INFO:teuthology.orchestra.run.smithi151.stderr:Sourcing file `/etc/default/grub' 2024-03-29T04:04:08.393 INFO:teuthology.orchestra.run.smithi151.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-29T04:04:08.395 INFO:teuthology.orchestra.run.smithi151.stderr:Generating grub configuration file ... 2024-03-29T04:04:08.566 INFO:teuthology.orchestra.run.smithi151.stderr:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-29T04:04:08.572 INFO:teuthology.orchestra.run.smithi151.stderr:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-29T04:04:08.796 INFO:teuthology.orchestra.run.smithi151.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-29T04:04:08.802 INFO:teuthology.orchestra.run.smithi151.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-29T04:04:08.846 INFO:teuthology.orchestra.run.smithi151.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-29T04:04:08.851 INFO:teuthology.orchestra.run.smithi151.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-29T04:04:08.962 INFO:teuthology.orchestra.run.smithi151.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-29T04:04:08.963 INFO:teuthology.orchestra.run.smithi151.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-29T04:04:08.963 INFO:teuthology.orchestra.run.smithi151.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-29T04:04:08.984 INFO:teuthology.orchestra.run.smithi151.stderr:done 2024-03-29T04:04:08.993 DEBUG:teuthology.orchestra.run.smithi151:> sudo shutdown -r now 2024-03-29T04:04:37.306 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-29T04:04:37.307 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-03-29T04:04:37.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:04:38.997 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-29T04:04:38.998 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-03-29T04:04:38.998 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:05:13.631 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-03-29T04:05:22.638 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-03-29T04:05:22.639 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:05:37.322 DEBUG:teuthology.orchestra.remote:timed out 2024-03-29T04:05:46.330 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-03-29T04:05:46.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:05:46.787 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-03-29T04:05:47.590 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-03-29T04:05:47.590 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-101-generic"... 2024-03-29T04:05:47.590 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-03-29T04:05:47.639 INFO:teuthology.orchestra.run.smithi022.stdout:5.15.0-101-generic 2024-03-29T04:05:47.639 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-101-generic vs 5.15.0-101-generic 2024-03-29T04:05:47.639 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-29T04:05:47.639 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-29T04:05:48.640 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-29T04:05:48.640 DEBUG:teuthology.orchestra.run.smithi022:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-29T04:05:48.861 INFO:teuthology.orchestra.run.smithi022.stdout:ttyS1 2024-03-29T04:05:48.885 DEBUG:teuthology.parallel:result is None 2024-03-29T04:06:22.641 DEBUG:teuthology.orchestra.remote:timed out 2024-03-29T04:06:34.642 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-03-29T04:06:34.643 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:06:34.980 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-03-29T04:06:35.497 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-03-29T04:06:35.497 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-101-generic"... 2024-03-29T04:06:35.497 DEBUG:teuthology.orchestra.run.smithi151:> uname -r 2024-03-29T04:06:35.545 INFO:teuthology.orchestra.run.smithi151.stdout:5.15.0-101-generic 2024-03-29T04:06:35.546 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-101-generic vs 5.15.0-101-generic 2024-03-29T04:06:35.546 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-29T04:06:35.546 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-29T04:06:36.547 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-29T04:06:36.547 DEBUG:teuthology.orchestra.run.smithi151:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-29T04:06:36.913 INFO:teuthology.orchestra.run.smithi151.stdout:ttyS1 2024-03-29T04:06:36.948 DEBUG:teuthology.parallel:result is None 2024-03-29T04:06:36.948 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-29T04:06:36.955 INFO:teuthology.task.internal:Creating test directory... 2024-03-29T04:06:36.956 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-29T04:06:36.958 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-29T04:06:36.963 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-29T04:06:36.976 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-29T04:06:36.986 INFO:teuthology.task.internal:Creating archive directory... 2024-03-29T04:06:36.986 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-29T04:06:37.008 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-29T04:06:37.028 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-29T04:06:37.034 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-29T04:06:37.035 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-29T04:06:37.064 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-29T04:06:37.095 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T04:06:37.096 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T04:06:37.103 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T04:06:37.104 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T04:06:37.106 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-29T04:06:37.113 INFO:teuthology.task.internal:Configuring sudo... 2024-03-29T04:06:37.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-29T04:06:37.152 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-29T04:06:37.170 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-29T04:06:37.179 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-29T04:06:37.179 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-29T04:06:37.212 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-29T04:06:37.218 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T04:06:37.263 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T04:06:37.311 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:06:37.311 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-29T04:06:37.375 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T04:06:37.382 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T04:06:37.430 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:06:37.430 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-29T04:06:37.495 DEBUG:teuthology.orchestra.run.smithi022:> sudo service rsyslog restart 2024-03-29T04:06:37.497 DEBUG:teuthology.orchestra.run.smithi151:> sudo service rsyslog restart 2024-03-29T04:06:37.566 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-29T04:06:37.574 INFO:teuthology.task.internal:Starting timer... 2024-03-29T04:06:37.574 INFO:teuthology.run_tasks:Running task pcp... 2024-03-29T04:06:37.585 INFO:teuthology.run_tasks:Running task selinux... 2024-03-29T04:06:37.594 DEBUG:teuthology.task.selinux:Excluding smithi022: OS 'ubuntu' does not support SELinux 2024-03-29T04:06:37.594 DEBUG:teuthology.task.selinux:Excluding smithi151: OS 'ubuntu' does not support SELinux 2024-03-29T04:06:37.594 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-29T04:06:37.594 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-03-29T04:06:37.594 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-29T04:06:37.595 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-29T04:06:37.604 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-29T04:06:37.621 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-29T04:06:37.622 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi022.front.sepia.ceph.com,smithi151.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-29T04:13:39.993 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi022.front.sepia.ceph.com'), Remote(name='ubuntu@smithi151.front.sepia.ceph.com')] 2024-03-29T04:13:39.995 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-03-29T04:13:39.996 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:13:40.078 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-03-29T04:13:40.158 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-03-29T04:13:40.159 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-03-29T04:13:40.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:13:40.238 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-03-29T04:13:40.314 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-03-29T04:13:40.315 INFO:teuthology.run_tasks:Running task clock... 2024-03-29T04:13:40.325 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-29T04:13:40.325 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-29T04:13:40.326 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T04:13:40.328 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-29T04:13:40.328 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T04:13:40.356 INFO:teuthology.orchestra.run.smithi022.stdout:29 Mar 04:13:40 ntpd[17090]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-29T04:13:40.356 INFO:teuthology.orchestra.run.smithi022.stdout:29 Mar 04:13:40 ntpd[17090]: Command line: ntpd -gq 2024-03-29T04:13:40.356 INFO:teuthology.orchestra.run.smithi022.stdout:29 Mar 04:13:40 ntpd[17090]: ---------------------------------------------------- 2024-03-29T04:13:40.356 INFO:teuthology.orchestra.run.smithi022.stdout:29 Mar 04:13:40 ntpd[17090]: ntp-4 is maintained by Network Time Foundation, 2024-03-29T04:13:40.356 INFO:teuthology.orchestra.run.smithi022.stdout:29 Mar 04:13:40 ntpd[17090]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-29T04:13:40.356 INFO:teuthology.orchestra.run.smithi022.stdout:29 Mar 04:13:40 ntpd[17090]: corporation. Support and training for ntp-4 are 2024-03-29T04:13:40.356 INFO:teuthology.orchestra.run.smithi022.stdout:29 Mar 04:13:40 ntpd[17090]: available at https://www.nwtime.org/support 2024-03-29T04:13:40.356 INFO:teuthology.orchestra.run.smithi022.stdout:29 Mar 04:13:40 ntpd[17090]: ---------------------------------------------------- 2024-03-29T04:13:40.357 INFO:teuthology.orchestra.run.smithi022.stdout:29 Mar 04:13:40 ntpd[17090]: proto: precision = 0.070 usec (-24) 2024-03-29T04:13:40.357 INFO:teuthology.orchestra.run.smithi022.stdout:29 Mar 04:13:40 ntpd[17090]: basedate set to 2022-02-04 2024-03-29T04:13:40.357 INFO:teuthology.orchestra.run.smithi022.stdout:29 Mar 04:13:40 ntpd[17090]: gps base set to 2022-02-06 (week 2196) 2024-03-29T04:13:40.358 INFO:teuthology.orchestra.run.smithi022.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-29T04:13:40.358 INFO:teuthology.orchestra.run.smithi022.stdout:29 Mar 04:13:40 ntpd[17090]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-29T04:13:40.358 INFO:teuthology.orchestra.run.smithi022.stdout:29 Mar 04:13:40 ntpd[17090]: restrict ::: KOD does nothing without LIMITED. 2024-03-29T04:13:40.358 INFO:teuthology.orchestra.run.smithi022.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-29T04:13:40.359 INFO:teuthology.orchestra.run.smithi022.stdout:29 Mar 04:13:40 ntpd[17090]: Listen and drop on 0 v6wildcard [::]:123 2024-03-29T04:13:40.359 INFO:teuthology.orchestra.run.smithi022.stdout:29 Mar 04:13:40 ntpd[17090]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-29T04:13:40.359 INFO:teuthology.orchestra.run.smithi022.stdout:29 Mar 04:13:40 ntpd[17090]: Listen normally on 2 lo 127.0.0.1:123 2024-03-29T04:13:40.359 INFO:teuthology.orchestra.run.smithi022.stdout:29 Mar 04:13:40 ntpd[17090]: Listen normally on 3 ens1f0 172.21.15.22:123 2024-03-29T04:13:40.360 INFO:teuthology.orchestra.run.smithi022.stdout:29 Mar 04:13:40 ntpd[17090]: Listen normally on 4 lo [::1]:123 2024-03-29T04:13:40.360 INFO:teuthology.orchestra.run.smithi022.stdout:29 Mar 04:13:40 ntpd[17090]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:fe59:5de4%3]:123 2024-03-29T04:13:40.360 INFO:teuthology.orchestra.run.smithi022.stdout:29 Mar 04:13:40 ntpd[17090]: Listening on routing socket on fd #22 for interface updates 2024-03-29T04:13:40.389 INFO:teuthology.orchestra.run.smithi151.stdout:29 Mar 04:13:40 ntpd[17193]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-29T04:13:40.389 INFO:teuthology.orchestra.run.smithi151.stdout:29 Mar 04:13:40 ntpd[17193]: Command line: ntpd -gq 2024-03-29T04:13:40.389 INFO:teuthology.orchestra.run.smithi151.stdout:29 Mar 04:13:40 ntpd[17193]: ---------------------------------------------------- 2024-03-29T04:13:40.389 INFO:teuthology.orchestra.run.smithi151.stdout:29 Mar 04:13:40 ntpd[17193]: ntp-4 is maintained by Network Time Foundation, 2024-03-29T04:13:40.389 INFO:teuthology.orchestra.run.smithi151.stdout:29 Mar 04:13:40 ntpd[17193]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-29T04:13:40.389 INFO:teuthology.orchestra.run.smithi151.stdout:29 Mar 04:13:40 ntpd[17193]: corporation. Support and training for ntp-4 are 2024-03-29T04:13:40.389 INFO:teuthology.orchestra.run.smithi151.stdout:29 Mar 04:13:40 ntpd[17193]: available at https://www.nwtime.org/support 2024-03-29T04:13:40.390 INFO:teuthology.orchestra.run.smithi151.stdout:29 Mar 04:13:40 ntpd[17193]: ---------------------------------------------------- 2024-03-29T04:13:40.390 INFO:teuthology.orchestra.run.smithi151.stdout:29 Mar 04:13:40 ntpd[17193]: proto: precision = 0.070 usec (-24) 2024-03-29T04:13:40.390 INFO:teuthology.orchestra.run.smithi151.stdout:29 Mar 04:13:40 ntpd[17193]: basedate set to 2022-02-04 2024-03-29T04:13:40.391 INFO:teuthology.orchestra.run.smithi151.stdout:29 Mar 04:13:40 ntpd[17193]: gps base set to 2022-02-06 (week 2196) 2024-03-29T04:13:40.391 INFO:teuthology.orchestra.run.smithi151.stdout:29 Mar 04:13:40 ntpd[17193]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-29T04:13:40.391 INFO:teuthology.orchestra.run.smithi151.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-29T04:13:40.391 INFO:teuthology.orchestra.run.smithi151.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-29T04:13:40.392 INFO:teuthology.orchestra.run.smithi151.stdout:29 Mar 04:13:40 ntpd[17193]: restrict ::: KOD does nothing without LIMITED. 2024-03-29T04:13:40.392 INFO:teuthology.orchestra.run.smithi151.stdout:29 Mar 04:13:40 ntpd[17193]: Listen and drop on 0 v6wildcard [::]:123 2024-03-29T04:13:40.392 INFO:teuthology.orchestra.run.smithi151.stdout:29 Mar 04:13:40 ntpd[17193]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-29T04:13:40.392 INFO:teuthology.orchestra.run.smithi151.stdout:29 Mar 04:13:40 ntpd[17193]: Listen normally on 2 lo 127.0.0.1:123 2024-03-29T04:13:40.392 INFO:teuthology.orchestra.run.smithi151.stdout:29 Mar 04:13:40 ntpd[17193]: Listen normally on 3 enp3s0f1 172.21.15.151:123 2024-03-29T04:13:40.392 INFO:teuthology.orchestra.run.smithi151.stdout:29 Mar 04:13:40 ntpd[17193]: Listen normally on 4 lo [::1]:123 2024-03-29T04:13:40.392 INFO:teuthology.orchestra.run.smithi151.stdout:29 Mar 04:13:40 ntpd[17193]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6f63%5]:123 2024-03-29T04:13:40.392 INFO:teuthology.orchestra.run.smithi151.stdout:29 Mar 04:13:40 ntpd[17193]: Listening on routing socket on fd #22 for interface updates 2024-03-29T04:13:41.359 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:41 ntpd[17090]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:41.360 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:41 ntpd[17090]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:41.360 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:41 ntpd[17090]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:41.360 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:41 ntpd[17090]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:41.360 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:41 ntpd[17090]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:41.360 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:41 ntpd[17090]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:41.360 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:41 ntpd[17090]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:41.361 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:41 ntpd[17090]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:41.361 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:41 ntpd[17090]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T04:13:41.361 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:41 ntpd[17090]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T04:13:41.361 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:41 ntpd[17090]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:41.361 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:41 ntpd[17090]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:41.361 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:41 ntpd[17090]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T04:13:41.362 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:41 ntpd[17090]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T04:13:41.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:41 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:41.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:41 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:41.391 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:41 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:41.391 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:41 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:41.391 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:41 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:41.391 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:41 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:41.391 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:41 ntpd[17193]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T04:13:41.391 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:41 ntpd[17193]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T04:13:42.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:42 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:42.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:42 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:42.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:42 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:42.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:42 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:42.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:42 ntpd[17193]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T04:13:42.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:42 ntpd[17193]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T04:13:43.358 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:43 ntpd[17090]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:43.358 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:43 ntpd[17090]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:43.358 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:43 ntpd[17090]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:43.358 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:43 ntpd[17090]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:43.358 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:43 ntpd[17090]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:43.358 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:43 ntpd[17090]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:43.358 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:43 ntpd[17090]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T04:13:43.359 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:43 ntpd[17090]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T04:13:43.359 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:43 ntpd[17090]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:43.359 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:43 ntpd[17090]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:43.359 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:43 ntpd[17090]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T04:13:43.359 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:43 ntpd[17090]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T04:13:43.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:43 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:43.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:43 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:43.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:43 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:43.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:43 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:43.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:43 ntpd[17193]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T04:13:43.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:43 ntpd[17193]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T04:13:44.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:44 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:44.391 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:44 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:44.391 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:44 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:44.391 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:44 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:44.391 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:44 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:44.391 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:44 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:44.391 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:44 ntpd[17193]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T04:13:44.391 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:44 ntpd[17193]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T04:13:44.391 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:44 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:44.391 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:44 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:44.391 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:44 ntpd[17193]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T04:13:44.391 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:44 ntpd[17193]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T04:13:45.361 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:45 ntpd[17090]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:45.361 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:45 ntpd[17090]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:45.361 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:45 ntpd[17090]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:45.362 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:45 ntpd[17090]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:45.362 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:45 ntpd[17090]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:45.362 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:45 ntpd[17090]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:45.362 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:45 ntpd[17090]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:45.362 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:45 ntpd[17090]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:45.362 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:45 ntpd[17090]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T04:13:45.362 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:45 ntpd[17090]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T04:13:45.362 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:45 ntpd[17090]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:45.362 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:45 ntpd[17090]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:45.362 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:45 ntpd[17090]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T04:13:45.362 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:45 ntpd[17090]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T04:13:45.362 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:45 ntpd[17090]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:45.362 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:45 ntpd[17090]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:45.363 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:45 ntpd[17090]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T04:13:45.363 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:45 ntpd[17090]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T04:13:45.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:45 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:45.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:45 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:45.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:45 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:45.391 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:45 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:45.391 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:45 ntpd[17193]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T04:13:45.391 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:45 ntpd[17193]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T04:13:46.392 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:46 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:46.393 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:46 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:46.393 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:46 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:46.393 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:46 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:46.393 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:46 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:46.393 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:46 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:46.393 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:46 ntpd[17193]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T04:13:46.394 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:46 ntpd[17193]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T04:13:46.394 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:46 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:46.394 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:46 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:46.394 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:46 ntpd[17193]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T04:13:46.394 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:46 ntpd[17193]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T04:13:47.358 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:47 ntpd[17090]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:47.358 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:47 ntpd[17090]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:47.358 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:47 ntpd[17090]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:47.358 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:47 ntpd[17090]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:47.358 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:47 ntpd[17090]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:47.358 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:47 ntpd[17090]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:47.358 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:47 ntpd[17090]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:47.358 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:47 ntpd[17090]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:47.358 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:47 ntpd[17090]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T04:13:47.358 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:47 ntpd[17090]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T04:13:47.359 INFO:teuthology.orchestra.run.smithi022.stdout:29 Mar 04:13:47 ntpd[17090]: ntpd: time slew -0.000702 s 2024-03-29T04:13:47.359 INFO:teuthology.orchestra.run.smithi022.stdout:ntpd: time slew -0.000702s 2024-03-29T04:13:47.359 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:47 ntpd[17090]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-29T04:13:47.359 INFO:teuthology.orchestra.run.smithi022.stderr:29 Mar 04:13:47 ntpd[17090]: can't open /var/log/ntpstats/loopstats.20240329: Permission denied 2024-03-29T04:13:47.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:47 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:47.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:47 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:47.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:47 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T04:13:47.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:47 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T04:13:47.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:47 ntpd[17193]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T04:13:47.390 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:47 ntpd[17193]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T04:13:47.391 INFO:teuthology.orchestra.run.smithi151.stdout:29 Mar 04:13:47 ntpd[17193]: ntpd: time slew -0.001007 s 2024-03-29T04:13:47.391 INFO:teuthology.orchestra.run.smithi151.stdout:ntpd: time slew -0.001007s 2024-03-29T04:13:47.392 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:47 ntpd[17193]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-29T04:13:47.392 INFO:teuthology.orchestra.run.smithi151.stderr:29 Mar 04:13:47 ntpd[17193]: can't open /var/log/ntpstats/loopstats.20240329: Permission denied 2024-03-29T04:13:47.411 INFO:teuthology.orchestra.run.smithi022.stdout: remote refid st t when poll reach delay offset jitter 2024-03-29T04:13:47.411 INFO:teuthology.orchestra.run.smithi022.stdout:============================================================================== 2024-03-29T04:13:47.411 INFO:teuthology.orchestra.run.smithi022.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-29T04:13:47.411 INFO:teuthology.orchestra.run.smithi022.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-29T04:13:47.411 INFO:teuthology.orchestra.run.smithi022.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-29T04:13:47.411 INFO:teuthology.orchestra.run.smithi022.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-29T04:13:47.494 INFO:teuthology.orchestra.run.smithi151.stdout: remote refid st t when poll reach delay offset jitter 2024-03-29T04:13:47.494 INFO:teuthology.orchestra.run.smithi151.stdout:============================================================================== 2024-03-29T04:13:47.494 INFO:teuthology.orchestra.run.smithi151.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-29T04:13:47.495 INFO:teuthology.orchestra.run.smithi151.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-29T04:13:47.495 INFO:teuthology.orchestra.run.smithi151.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-29T04:13:47.495 INFO:teuthology.orchestra.run.smithi151.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-29T04:13:47.496 INFO:teuthology.run_tasks:Running task install... 2024-03-29T04:13:47.505 DEBUG:teuthology.task.install:project ceph 2024-03-29T04:13:47.505 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'}} 2024-03-29T04:13:47.505 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'} 2024-03-29T04:13:47.505 INFO:teuthology.task.install:Using flavor: default 2024-03-29T04:13:47.511 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-29T04:13:47.511 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-29T04:13:47.512 DEBUG:teuthology.orchestra.run.smithi022:> sudo apt-key list | grep Ceph 2024-03-29T04:13:47.512 DEBUG:teuthology.orchestra.run.smithi151:> sudo apt-key list | grep Ceph 2024-03-29T04:13:47.577 INFO:teuthology.orchestra.run.smithi022.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-29T04:13:47.602 INFO:teuthology.orchestra.run.smithi151.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-29T04:13:47.608 INFO:teuthology.orchestra.run.smithi022.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-29T04:13:47.608 INFO:teuthology.orchestra.run.smithi022.stdout:uid [ unknown] Ceph.com (release key) 2024-03-29T04:13:47.609 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-29T04:13:47.609 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-29T04:13:47.609 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=69e2cd73c52284dfd5096c66b60c6e6b26a3c447 2024-03-29T04:13:47.633 INFO:teuthology.orchestra.run.smithi151.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-29T04:13:47.633 INFO:teuthology.orchestra.run.smithi151.stdout:uid [ unknown] Ceph.com (release key) 2024-03-29T04:13:47.634 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-29T04:13:47.634 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-29T04:13:47.635 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=69e2cd73c52284dfd5096c66b60c6e6b26a3c447 2024-03-29T04:13:47.828 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default/ 2024-03-29T04:13:47.828 INFO:teuthology.task.install.deb:Package version is 19.0.0-2520-g69e2cd73-1jammy 2024-03-29T04:13:47.852 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default/ 2024-03-29T04:13:47.853 INFO:teuthology.task.install.deb:Package version is 19.0.0-2520-g69e2cd73-1jammy 2024-03-29T04:13:47.945 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:13:47.945 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-29T04:13:47.959 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:13:47.960 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-29T04:13:47.961 DEBUG:teuthology.orchestra.run.smithi022:> sudo apt-get update 2024-03-29T04:13:47.975 DEBUG:teuthology.orchestra.run.smithi151:> sudo apt-get update 2024-03-29T04:13:48.139 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-29T04:13:48.145 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-29T04:13:48.150 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-29T04:13:48.155 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-29T04:13:48.156 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-29T04:13:48.158 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-29T04:13:48.171 INFO:teuthology.orchestra.run.smithi022.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-29T04:13:48.177 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-29T04:13:48.186 INFO:teuthology.orchestra.run.smithi022.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy InRelease 2024-03-29T04:13:48.207 INFO:teuthology.orchestra.run.smithi022.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-29T04:13:48.231 INFO:teuthology.orchestra.run.smithi022.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-29T04:13:48.253 INFO:teuthology.orchestra.run.smithi022.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-03-29T04:13:48.254 INFO:teuthology.orchestra.run.smithi151.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy InRelease 2024-03-29T04:13:48.281 INFO:teuthology.orchestra.run.smithi022.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,406 B] 2024-03-29T04:13:48.290 INFO:teuthology.orchestra.run.smithi151.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-29T04:13:48.331 INFO:teuthology.orchestra.run.smithi151.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-29T04:13:48.370 INFO:teuthology.orchestra.run.smithi151.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-03-29T04:13:48.422 INFO:teuthology.orchestra.run.smithi151.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,406 B] 2024-03-29T04:13:48.493 INFO:teuthology.orchestra.run.smithi151.stdout:Fetched 29.2 kB in 0s (85.0 kB/s) 2024-03-29T04:13:48.509 INFO:teuthology.orchestra.run.smithi022.stdout:Fetched 29.2 kB in 0s (79.9 kB/s) 2024-03-29T04:13:49.451 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-03-29T04:13:49.463 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2024-03-29T04:13:49.468 DEBUG:teuthology.orchestra.run.smithi151:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2520-g69e2cd73-1jammy cephadm=19.0.0-2520-g69e2cd73-1jammy ceph-mds=19.0.0-2520-g69e2cd73-1jammy ceph-mgr=19.0.0-2520-g69e2cd73-1jammy ceph-common=19.0.0-2520-g69e2cd73-1jammy ceph-fuse=19.0.0-2520-g69e2cd73-1jammy ceph-test=19.0.0-2520-g69e2cd73-1jammy ceph-volume=19.0.0-2520-g69e2cd73-1jammy radosgw=19.0.0-2520-g69e2cd73-1jammy python3-rados=19.0.0-2520-g69e2cd73-1jammy python3-rgw=19.0.0-2520-g69e2cd73-1jammy python3-cephfs=19.0.0-2520-g69e2cd73-1jammy python3-rbd=19.0.0-2520-g69e2cd73-1jammy libcephfs2=19.0.0-2520-g69e2cd73-1jammy libcephfs-dev=19.0.0-2520-g69e2cd73-1jammy librados2=19.0.0-2520-g69e2cd73-1jammy librbd1=19.0.0-2520-g69e2cd73-1jammy rbd-fuse=19.0.0-2520-g69e2cd73-1jammy python3-cephfs=19.0.0-2520-g69e2cd73-1jammy cephfs-shell=19.0.0-2520-g69e2cd73-1jammy cephfs-top=19.0.0-2520-g69e2cd73-1jammy cephfs-mirror=19.0.0-2520-g69e2cd73-1jammy 2024-03-29T04:13:49.479 DEBUG:teuthology.orchestra.run.smithi022:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2520-g69e2cd73-1jammy cephadm=19.0.0-2520-g69e2cd73-1jammy ceph-mds=19.0.0-2520-g69e2cd73-1jammy ceph-mgr=19.0.0-2520-g69e2cd73-1jammy ceph-common=19.0.0-2520-g69e2cd73-1jammy ceph-fuse=19.0.0-2520-g69e2cd73-1jammy ceph-test=19.0.0-2520-g69e2cd73-1jammy ceph-volume=19.0.0-2520-g69e2cd73-1jammy radosgw=19.0.0-2520-g69e2cd73-1jammy python3-rados=19.0.0-2520-g69e2cd73-1jammy python3-rgw=19.0.0-2520-g69e2cd73-1jammy python3-cephfs=19.0.0-2520-g69e2cd73-1jammy python3-rbd=19.0.0-2520-g69e2cd73-1jammy libcephfs2=19.0.0-2520-g69e2cd73-1jammy libcephfs-dev=19.0.0-2520-g69e2cd73-1jammy librados2=19.0.0-2520-g69e2cd73-1jammy librbd1=19.0.0-2520-g69e2cd73-1jammy rbd-fuse=19.0.0-2520-g69e2cd73-1jammy python3-cephfs=19.0.0-2520-g69e2cd73-1jammy cephfs-shell=19.0.0-2520-g69e2cd73-1jammy cephfs-top=19.0.0-2520-g69e2cd73-1jammy cephfs-mirror=19.0.0-2520-g69e2cd73-1jammy 2024-03-29T04:13:49.530 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-03-29T04:13:49.540 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2024-03-29T04:13:49.723 INFO:teuthology.orchestra.run.smithi151.stdout:Building dependency tree... 2024-03-29T04:13:49.723 INFO:teuthology.orchestra.run.smithi151.stdout:Reading state information... 2024-03-29T04:13:49.739 INFO:teuthology.orchestra.run.smithi022.stdout:Building dependency tree... 2024-03-29T04:13:49.740 INFO:teuthology.orchestra.run.smithi022.stdout:Reading state information... 2024-03-29T04:13:49.904 INFO:teuthology.orchestra.run.smithi151.stdout:The following packages were automatically installed and are no longer required: 2024-03-29T04:13:49.904 INFO:teuthology.orchestra.run.smithi151.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-29T04:13:49.904 INFO:teuthology.orchestra.run.smithi151.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-29T04:13:49.904 INFO:teuthology.orchestra.run.smithi151.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-29T04:13:49.905 INFO:teuthology.orchestra.run.smithi151.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-29T04:13:49.905 INFO:teuthology.orchestra.run.smithi151.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-29T04:13:49.905 INFO:teuthology.orchestra.run.smithi151.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-29T04:13:49.906 INFO:teuthology.orchestra.run.smithi151.stdout:The following additional packages will be installed: 2024-03-29T04:13:49.906 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-29T04:13:49.906 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-29T04:13:49.906 INFO:teuthology.orchestra.run.smithi151.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-29T04:13:49.906 INFO:teuthology.orchestra.run.smithi151.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-29T04:13:49.907 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-29T04:13:49.907 INFO:teuthology.orchestra.run.smithi151.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-29T04:13:49.907 INFO:teuthology.orchestra.run.smithi151.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-29T04:13:49.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-29T04:13:49.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-29T04:13:49.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-29T04:13:49.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-29T04:13:49.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-29T04:13:49.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-29T04:13:49.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-29T04:13:49.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-29T04:13:49.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-29T04:13:49.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-29T04:13:49.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-29T04:13:49.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-29T04:13:49.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-29T04:13:49.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-29T04:13:49.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-29T04:13:49.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-29T04:13:49.909 INFO:teuthology.orchestra.run.smithi151.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-03-29T04:13:49.909 INFO:teuthology.orchestra.run.smithi151.stdout:Suggested packages: 2024-03-29T04:13:49.909 INFO:teuthology.orchestra.run.smithi151.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-29T04:13:49.909 INFO:teuthology.orchestra.run.smithi151.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-29T04:13:49.909 INFO:teuthology.orchestra.run.smithi151.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-29T04:13:49.909 INFO:teuthology.orchestra.run.smithi151.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-29T04:13:49.909 INFO:teuthology.orchestra.run.smithi151.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-29T04:13:49.909 INFO:teuthology.orchestra.run.smithi151.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-29T04:13:49.909 INFO:teuthology.orchestra.run.smithi151.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-29T04:13:49.910 INFO:teuthology.orchestra.run.smithi151.stdout:Recommended packages: 2024-03-29T04:13:49.910 INFO:teuthology.orchestra.run.smithi151.stdout: btrfs-tools 2024-03-29T04:13:49.932 INFO:teuthology.orchestra.run.smithi022.stdout:The following packages were automatically installed and are no longer required: 2024-03-29T04:13:49.932 INFO:teuthology.orchestra.run.smithi022.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-29T04:13:49.932 INFO:teuthology.orchestra.run.smithi022.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-29T04:13:49.932 INFO:teuthology.orchestra.run.smithi022.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-29T04:13:49.933 INFO:teuthology.orchestra.run.smithi022.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-29T04:13:49.933 INFO:teuthology.orchestra.run.smithi022.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-29T04:13:49.933 INFO:teuthology.orchestra.run.smithi022.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-29T04:13:49.934 INFO:teuthology.orchestra.run.smithi022.stdout:The following additional packages will be installed: 2024-03-29T04:13:49.934 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-29T04:13:49.934 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-29T04:13:49.934 INFO:teuthology.orchestra.run.smithi022.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-29T04:13:49.934 INFO:teuthology.orchestra.run.smithi022.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-29T04:13:49.935 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-29T04:13:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-29T04:13:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-29T04:13:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-29T04:13:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-29T04:13:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-29T04:13:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-29T04:13:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-29T04:13:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-29T04:13:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-29T04:13:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-29T04:13:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-29T04:13:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-29T04:13:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-29T04:13:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-29T04:13:49.937 INFO:teuthology.orchestra.run.smithi022.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-29T04:13:49.937 INFO:teuthology.orchestra.run.smithi022.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-29T04:13:49.937 INFO:teuthology.orchestra.run.smithi022.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-29T04:13:49.937 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-29T04:13:49.937 INFO:teuthology.orchestra.run.smithi022.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-03-29T04:13:49.937 INFO:teuthology.orchestra.run.smithi022.stdout:Suggested packages: 2024-03-29T04:13:49.937 INFO:teuthology.orchestra.run.smithi022.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-29T04:13:49.937 INFO:teuthology.orchestra.run.smithi022.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-29T04:13:49.937 INFO:teuthology.orchestra.run.smithi022.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-29T04:13:49.938 INFO:teuthology.orchestra.run.smithi022.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-29T04:13:49.938 INFO:teuthology.orchestra.run.smithi022.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-29T04:13:49.938 INFO:teuthology.orchestra.run.smithi022.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-29T04:13:49.938 INFO:teuthology.orchestra.run.smithi022.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-29T04:13:49.938 INFO:teuthology.orchestra.run.smithi022.stdout:Recommended packages: 2024-03-29T04:13:49.938 INFO:teuthology.orchestra.run.smithi022.stdout: btrfs-tools 2024-03-29T04:13:50.016 INFO:teuthology.orchestra.run.smithi151.stdout:The following NEW packages will be installed: 2024-03-29T04:13:50.016 INFO:teuthology.orchestra.run.smithi151.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-29T04:13:50.016 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-29T04:13:50.016 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-03-29T04:13:50.016 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-03-29T04:13:50.017 INFO:teuthology.orchestra.run.smithi151.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-29T04:13:50.017 INFO:teuthology.orchestra.run.smithi151.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-03-29T04:13:50.017 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-03-29T04:13:50.017 INFO:teuthology.orchestra.run.smithi151.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-03-29T04:13:50.017 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-03-29T04:13:50.017 INFO:teuthology.orchestra.run.smithi151.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-29T04:13:50.017 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-29T04:13:50.017 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-29T04:13:50.017 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-29T04:13:50.017 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-29T04:13:50.017 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-29T04:13:50.017 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-29T04:13:50.017 INFO:teuthology.orchestra.run.smithi151.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-29T04:13:50.017 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-29T04:13:50.018 INFO:teuthology.orchestra.run.smithi151.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-29T04:13:50.018 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-29T04:13:50.018 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-29T04:13:50.018 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-29T04:13:50.018 INFO:teuthology.orchestra.run.smithi151.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-29T04:13:50.018 INFO:teuthology.orchestra.run.smithi151.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-29T04:13:50.018 INFO:teuthology.orchestra.run.smithi151.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-29T04:13:50.018 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-29T04:13:50.018 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-03-29T04:13:50.018 INFO:teuthology.orchestra.run.smithi151.stdout: socat unzip xmlstarlet zip 2024-03-29T04:13:50.049 INFO:teuthology.orchestra.run.smithi022.stdout:The following NEW packages will be installed: 2024-03-29T04:13:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-29T04:13:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-29T04:13:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-03-29T04:13:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-03-29T04:13:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-29T04:13:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-03-29T04:13:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-03-29T04:13:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-03-29T04:13:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-03-29T04:13:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-29T04:13:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-29T04:13:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-29T04:13:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-29T04:13:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-29T04:13:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-29T04:13:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-29T04:13:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-29T04:13:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-29T04:13:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-29T04:13:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-29T04:13:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-29T04:13:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-29T04:13:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-29T04:13:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-29T04:13:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-29T04:13:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-29T04:13:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-03-29T04:13:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: socat unzip xmlstarlet zip 2024-03-29T04:13:50.116 INFO:teuthology.orchestra.run.smithi022.stdout:0 upgraded, 124 newly installed, 0 to remove and 298 not upgraded. 2024-03-29T04:13:50.117 INFO:teuthology.orchestra.run.smithi022.stdout:Need to get 179 MB of archives. 2024-03-29T04:13:50.117 INFO:teuthology.orchestra.run.smithi022.stdout:After this operation, 821 MB of additional disk space will be used. 2024-03-29T04:13:50.117 INFO:teuthology.orchestra.run.smithi022.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-29T04:13:50.138 INFO:teuthology.orchestra.run.smithi022.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-29T04:13:50.149 INFO:teuthology.orchestra.run.smithi022.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-29T04:13:50.198 INFO:teuthology.orchestra.run.smithi022.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-29T04:13:50.208 INFO:teuthology.orchestra.run.smithi151.stdout:0 upgraded, 124 newly installed, 0 to remove and 298 not upgraded. 2024-03-29T04:13:50.208 INFO:teuthology.orchestra.run.smithi151.stdout:Need to get 179 MB of archives. 2024-03-29T04:13:50.209 INFO:teuthology.orchestra.run.smithi151.stdout:After this operation, 821 MB of additional disk space will be used. 2024-03-29T04:13:50.209 INFO:teuthology.orchestra.run.smithi151.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-29T04:13:50.223 INFO:teuthology.orchestra.run.smithi022.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-29T04:13:50.227 INFO:teuthology.orchestra.run.smithi022.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-29T04:13:50.238 INFO:teuthology.orchestra.run.smithi151.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2520-g69e2cd73-1jammy [3,556 kB] 2024-03-29T04:13:50.240 INFO:teuthology.orchestra.run.smithi022.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-29T04:13:50.272 INFO:teuthology.orchestra.run.smithi022.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2520-g69e2cd73-1jammy [3,556 kB] 2024-03-29T04:13:50.294 INFO:teuthology.orchestra.run.smithi151.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-29T04:13:50.296 INFO:teuthology.orchestra.run.smithi022.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-29T04:13:50.301 INFO:teuthology.orchestra.run.smithi022.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-29T04:13:50.314 INFO:teuthology.orchestra.run.smithi022.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-29T04:13:50.319 INFO:teuthology.orchestra.run.smithi022.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-29T04:13:50.319 INFO:teuthology.orchestra.run.smithi022.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-29T04:13:50.328 INFO:teuthology.orchestra.run.smithi022.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-29T04:13:50.331 INFO:teuthology.orchestra.run.smithi022.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-29T04:13:50.333 INFO:teuthology.orchestra.run.smithi022.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-29T04:13:50.334 INFO:teuthology.orchestra.run.smithi022.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-29T04:13:50.335 INFO:teuthology.orchestra.run.smithi022.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-29T04:13:50.336 INFO:teuthology.orchestra.run.smithi151.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-29T04:13:50.346 INFO:teuthology.orchestra.run.smithi022.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-29T04:13:50.348 INFO:teuthology.orchestra.run.smithi022.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-29T04:13:50.350 INFO:teuthology.orchestra.run.smithi022.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-29T04:13:50.351 INFO:teuthology.orchestra.run.smithi022.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-29T04:13:50.366 INFO:teuthology.orchestra.run.smithi022.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-29T04:13:50.367 INFO:teuthology.orchestra.run.smithi022.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-29T04:13:50.368 INFO:teuthology.orchestra.run.smithi022.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-29T04:13:50.368 INFO:teuthology.orchestra.run.smithi022.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-29T04:13:50.368 INFO:teuthology.orchestra.run.smithi022.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-29T04:13:50.369 INFO:teuthology.orchestra.run.smithi022.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-29T04:13:50.387 INFO:teuthology.orchestra.run.smithi022.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-29T04:13:50.387 INFO:teuthology.orchestra.run.smithi022.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-29T04:13:50.390 INFO:teuthology.orchestra.run.smithi022.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-29T04:13:50.390 INFO:teuthology.orchestra.run.smithi022.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-29T04:13:50.407 INFO:teuthology.orchestra.run.smithi022.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-29T04:13:50.408 INFO:teuthology.orchestra.run.smithi022.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-29T04:13:50.409 INFO:teuthology.orchestra.run.smithi022.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-29T04:13:50.409 INFO:teuthology.orchestra.run.smithi022.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-29T04:13:50.409 INFO:teuthology.orchestra.run.smithi022.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-29T04:13:50.411 INFO:teuthology.orchestra.run.smithi022.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-29T04:13:50.428 INFO:teuthology.orchestra.run.smithi022.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-29T04:13:50.428 INFO:teuthology.orchestra.run.smithi022.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-29T04:13:50.435 INFO:teuthology.orchestra.run.smithi022.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-29T04:13:50.435 INFO:teuthology.orchestra.run.smithi022.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-29T04:13:50.449 INFO:teuthology.orchestra.run.smithi022.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-29T04:13:50.449 INFO:teuthology.orchestra.run.smithi022.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-29T04:13:50.450 INFO:teuthology.orchestra.run.smithi022.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-29T04:13:50.452 INFO:teuthology.orchestra.run.smithi022.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-29T04:13:50.453 INFO:teuthology.orchestra.run.smithi022.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-29T04:13:50.455 INFO:teuthology.orchestra.run.smithi022.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-29T04:13:50.469 INFO:teuthology.orchestra.run.smithi022.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-29T04:13:50.472 INFO:teuthology.orchestra.run.smithi022.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-29T04:13:50.473 INFO:teuthology.orchestra.run.smithi022.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-29T04:13:50.473 INFO:teuthology.orchestra.run.smithi022.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-29T04:13:50.490 INFO:teuthology.orchestra.run.smithi022.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-29T04:13:50.539 INFO:teuthology.orchestra.run.smithi151.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-29T04:13:50.547 INFO:teuthology.orchestra.run.smithi022.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-29T04:13:50.548 INFO:teuthology.orchestra.run.smithi022.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-29T04:13:50.549 INFO:teuthology.orchestra.run.smithi022.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-29T04:13:50.549 INFO:teuthology.orchestra.run.smithi022.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-29T04:13:50.551 INFO:teuthology.orchestra.run.smithi022.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-29T04:13:50.576 INFO:teuthology.orchestra.run.smithi022.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-29T04:13:50.591 INFO:teuthology.orchestra.run.smithi022.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-29T04:13:50.591 INFO:teuthology.orchestra.run.smithi022.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-29T04:13:50.612 INFO:teuthology.orchestra.run.smithi022.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-29T04:13:50.612 INFO:teuthology.orchestra.run.smithi022.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-29T04:13:50.623 INFO:teuthology.orchestra.run.smithi022.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-29T04:13:50.624 INFO:teuthology.orchestra.run.smithi022.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-29T04:13:50.624 INFO:teuthology.orchestra.run.smithi022.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-29T04:13:50.625 INFO:teuthology.orchestra.run.smithi022.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-29T04:13:50.626 INFO:teuthology.orchestra.run.smithi022.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-29T04:13:50.626 INFO:teuthology.orchestra.run.smithi022.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-29T04:13:50.631 INFO:teuthology.orchestra.run.smithi022.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-29T04:13:50.633 INFO:teuthology.orchestra.run.smithi151.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-29T04:13:50.644 INFO:teuthology.orchestra.run.smithi022.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-29T04:13:50.646 INFO:teuthology.orchestra.run.smithi022.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-29T04:13:50.647 INFO:teuthology.orchestra.run.smithi151.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-29T04:13:50.647 INFO:teuthology.orchestra.run.smithi022.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-29T04:13:50.665 INFO:teuthology.orchestra.run.smithi022.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-29T04:13:50.668 INFO:teuthology.orchestra.run.smithi022.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-29T04:13:50.668 INFO:teuthology.orchestra.run.smithi022.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-29T04:13:50.670 INFO:teuthology.orchestra.run.smithi022.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-29T04:13:50.670 INFO:teuthology.orchestra.run.smithi022.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-29T04:13:50.676 INFO:teuthology.orchestra.run.smithi022.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-29T04:13:50.677 INFO:teuthology.orchestra.run.smithi022.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-29T04:13:50.685 INFO:teuthology.orchestra.run.smithi022.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-29T04:13:50.686 INFO:teuthology.orchestra.run.smithi022.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-29T04:13:50.687 INFO:teuthology.orchestra.run.smithi022.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-29T04:13:50.698 INFO:teuthology.orchestra.run.smithi151.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-29T04:13:50.707 INFO:teuthology.orchestra.run.smithi022.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-29T04:13:50.710 INFO:teuthology.orchestra.run.smithi022.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-29T04:13:50.711 INFO:teuthology.orchestra.run.smithi022.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-29T04:13:50.720 INFO:teuthology.orchestra.run.smithi022.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-29T04:13:50.720 INFO:teuthology.orchestra.run.smithi022.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-29T04:13:50.720 INFO:teuthology.orchestra.run.smithi022.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-29T04:13:50.723 INFO:teuthology.orchestra.run.smithi022.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-29T04:13:50.728 INFO:teuthology.orchestra.run.smithi022.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-29T04:13:50.873 INFO:teuthology.orchestra.run.smithi151.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-29T04:13:50.882 INFO:teuthology.orchestra.run.smithi151.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-29T04:13:50.910 INFO:teuthology.orchestra.run.smithi151.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-29T04:13:50.917 INFO:teuthology.orchestra.run.smithi151.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-29T04:13:51.028 INFO:teuthology.orchestra.run.smithi151.stdout:Get:13 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2520-g69e2cd73-1jammy [3,226 kB] 2024-03-29T04:13:51.082 INFO:teuthology.orchestra.run.smithi151.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-29T04:13:51.158 INFO:teuthology.orchestra.run.smithi022.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2520-g69e2cd73-1jammy [3,226 kB] 2024-03-29T04:13:51.320 INFO:teuthology.orchestra.run.smithi151.stdout:Get:15 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2520-g69e2cd73-1jammy [712 kB] 2024-03-29T04:13:51.386 INFO:teuthology.orchestra.run.smithi151.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2520-g69e2cd73-1jammy [356 kB] 2024-03-29T04:13:51.393 INFO:teuthology.orchestra.run.smithi151.stdout:Get:17 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2520-g69e2cd73-1jammy [32.8 kB] 2024-03-29T04:13:51.394 INFO:teuthology.orchestra.run.smithi151.stdout:Get:18 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2520-g69e2cd73-1jammy [180 kB] 2024-03-29T04:13:51.423 INFO:teuthology.orchestra.run.smithi151.stdout:Get:19 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2520-g69e2cd73-1jammy [66.7 kB] 2024-03-29T04:13:51.425 INFO:teuthology.orchestra.run.smithi151.stdout:Get:20 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2520-g69e2cd73-1jammy [330 kB] 2024-03-29T04:13:51.454 INFO:teuthology.orchestra.run.smithi151.stdout:Get:21 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2520-g69e2cd73-1jammy [6,660 kB] 2024-03-29T04:13:51.487 INFO:teuthology.orchestra.run.smithi022.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2520-g69e2cd73-1jammy [712 kB] 2024-03-29T04:13:51.554 INFO:teuthology.orchestra.run.smithi022.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2520-g69e2cd73-1jammy [356 kB] 2024-03-29T04:13:51.562 INFO:teuthology.orchestra.run.smithi022.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2520-g69e2cd73-1jammy [32.8 kB] 2024-03-29T04:13:51.563 INFO:teuthology.orchestra.run.smithi022.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2520-g69e2cd73-1jammy [180 kB] 2024-03-29T04:13:51.592 INFO:teuthology.orchestra.run.smithi022.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2520-g69e2cd73-1jammy [66.7 kB] 2024-03-29T04:13:51.594 INFO:teuthology.orchestra.run.smithi022.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2520-g69e2cd73-1jammy [330 kB] 2024-03-29T04:13:51.626 INFO:teuthology.orchestra.run.smithi022.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2520-g69e2cd73-1jammy [6,660 kB] 2024-03-29T04:13:51.653 INFO:teuthology.orchestra.run.smithi151.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-29T04:13:51.668 INFO:teuthology.orchestra.run.smithi151.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-29T04:13:51.683 INFO:teuthology.orchestra.run.smithi151.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-29T04:13:51.692 INFO:teuthology.orchestra.run.smithi151.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-29T04:13:51.693 INFO:teuthology.orchestra.run.smithi151.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-29T04:13:51.709 INFO:teuthology.orchestra.run.smithi151.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-29T04:13:51.747 INFO:teuthology.orchestra.run.smithi151.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-29T04:13:51.761 INFO:teuthology.orchestra.run.smithi151.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-29T04:13:51.764 INFO:teuthology.orchestra.run.smithi151.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-29T04:13:51.765 INFO:teuthology.orchestra.run.smithi151.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-29T04:13:51.840 INFO:teuthology.orchestra.run.smithi151.stdout:Get:32 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2520-g69e2cd73-1jammy [112 kB] 2024-03-29T04:13:51.842 INFO:teuthology.orchestra.run.smithi151.stdout:Get:33 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2520-g69e2cd73-1jammy [466 kB] 2024-03-29T04:13:51.882 INFO:teuthology.orchestra.run.smithi151.stdout:Get:34 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2520-g69e2cd73-1jammy [25.4 MB] 2024-03-29T04:13:51.935 INFO:teuthology.orchestra.run.smithi151.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-29T04:13:52.018 INFO:teuthology.orchestra.run.smithi151.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-29T04:13:52.033 INFO:teuthology.orchestra.run.smithi151.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-29T04:13:52.100 INFO:teuthology.orchestra.run.smithi151.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-29T04:13:52.127 INFO:teuthology.orchestra.run.smithi151.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-29T04:13:52.145 INFO:teuthology.orchestra.run.smithi151.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-29T04:13:52.151 INFO:teuthology.orchestra.run.smithi022.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2520-g69e2cd73-1jammy [112 kB] 2024-03-29T04:13:52.153 INFO:teuthology.orchestra.run.smithi022.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2520-g69e2cd73-1jammy [466 kB] 2024-03-29T04:13:52.169 INFO:teuthology.orchestra.run.smithi151.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-29T04:13:52.173 INFO:teuthology.orchestra.run.smithi022.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2520-g69e2cd73-1jammy [25.4 MB] 2024-03-29T04:13:52.348 INFO:teuthology.orchestra.run.smithi151.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-29T04:13:52.364 INFO:teuthology.orchestra.run.smithi151.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-29T04:13:52.371 INFO:teuthology.orchestra.run.smithi151.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-29T04:13:52.376 INFO:teuthology.orchestra.run.smithi151.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-29T04:13:52.559 INFO:teuthology.orchestra.run.smithi151.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-29T04:13:52.643 INFO:teuthology.orchestra.run.smithi151.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-29T04:13:52.658 INFO:teuthology.orchestra.run.smithi151.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-29T04:13:52.844 INFO:teuthology.orchestra.run.smithi151.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-29T04:13:52.895 INFO:teuthology.orchestra.run.smithi151.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-29T04:13:52.909 INFO:teuthology.orchestra.run.smithi151.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-29T04:13:53.067 INFO:teuthology.orchestra.run.smithi151.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-29T04:13:53.068 INFO:teuthology.orchestra.run.smithi151.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-29T04:13:53.073 INFO:teuthology.orchestra.run.smithi151.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-29T04:13:53.078 INFO:teuthology.orchestra.run.smithi151.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-29T04:13:53.092 INFO:teuthology.orchestra.run.smithi151.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-29T04:13:53.277 INFO:teuthology.orchestra.run.smithi151.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-29T04:13:53.540 INFO:teuthology.orchestra.run.smithi151.stdout:Get:58 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2520-g69e2cd73-1jammy [5,085 kB] 2024-03-29T04:13:53.559 INFO:teuthology.orchestra.run.smithi151.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-29T04:13:53.674 INFO:teuthology.orchestra.run.smithi151.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-29T04:13:53.718 INFO:teuthology.orchestra.run.smithi151.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-29T04:13:53.766 INFO:teuthology.orchestra.run.smithi151.stdout:Get:62 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2520-g69e2cd73-1jammy [244 kB] 2024-03-29T04:13:53.789 INFO:teuthology.orchestra.run.smithi151.stdout:Get:63 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2520-g69e2cd73-1jammy [124 kB] 2024-03-29T04:13:53.793 INFO:teuthology.orchestra.run.smithi151.stdout:Get:64 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2520-g69e2cd73-1jammy [1,526 kB] 2024-03-29T04:13:53.801 INFO:teuthology.orchestra.run.smithi151.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-29T04:13:53.814 INFO:teuthology.orchestra.run.smithi151.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-29T04:13:53.827 INFO:teuthology.orchestra.run.smithi151.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-29T04:13:53.867 INFO:teuthology.orchestra.run.smithi151.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-29T04:13:53.880 INFO:teuthology.orchestra.run.smithi151.stdout:Get:69 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2520-g69e2cd73-1jammy [6,259 kB] 2024-03-29T04:13:54.131 INFO:teuthology.orchestra.run.smithi151.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-29T04:13:54.132 INFO:teuthology.orchestra.run.smithi151.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-29T04:13:54.135 INFO:teuthology.orchestra.run.smithi151.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-29T04:13:54.273 INFO:teuthology.orchestra.run.smithi151.stdout:Get:73 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2520-g69e2cd73-1jammy [22.8 MB] 2024-03-29T04:13:54.290 INFO:teuthology.orchestra.run.smithi022.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2520-g69e2cd73-1jammy [5,085 kB] 2024-03-29T04:13:54.299 INFO:teuthology.orchestra.run.smithi151.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-29T04:13:54.617 INFO:teuthology.orchestra.run.smithi022.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2520-g69e2cd73-1jammy [244 kB] 2024-03-29T04:13:54.623 INFO:teuthology.orchestra.run.smithi022.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2520-g69e2cd73-1jammy [124 kB] 2024-03-29T04:13:54.633 INFO:teuthology.orchestra.run.smithi151.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-29T04:13:54.654 INFO:teuthology.orchestra.run.smithi022.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2520-g69e2cd73-1jammy [1,526 kB] 2024-03-29T04:13:54.778 INFO:teuthology.orchestra.run.smithi022.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2520-g69e2cd73-1jammy [6,259 kB] 2024-03-29T04:13:55.046 INFO:teuthology.orchestra.run.smithi151.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-29T04:13:55.056 INFO:teuthology.orchestra.run.smithi151.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-29T04:13:55.057 INFO:teuthology.orchestra.run.smithi151.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-29T04:13:55.126 INFO:teuthology.orchestra.run.smithi151.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-29T04:13:55.127 INFO:teuthology.orchestra.run.smithi151.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-29T04:13:55.129 INFO:teuthology.orchestra.run.smithi151.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-29T04:13:55.130 INFO:teuthology.orchestra.run.smithi151.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-29T04:13:55.131 INFO:teuthology.orchestra.run.smithi151.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-29T04:13:55.289 INFO:teuthology.orchestra.run.smithi022.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2520-g69e2cd73-1jammy [22.8 MB] 2024-03-29T04:13:55.297 INFO:teuthology.orchestra.run.smithi151.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-29T04:13:55.396 INFO:teuthology.orchestra.run.smithi151.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-29T04:13:55.489 INFO:teuthology.orchestra.run.smithi151.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-29T04:13:55.794 INFO:teuthology.orchestra.run.smithi151.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-29T04:13:55.796 INFO:teuthology.orchestra.run.smithi151.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2520-g69e2cd73-1jammy [14.0 kB] 2024-03-29T04:13:55.797 INFO:teuthology.orchestra.run.smithi151.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2520-g69e2cd73-1jammy [870 kB] 2024-03-29T04:13:55.799 INFO:teuthology.orchestra.run.smithi151.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-29T04:13:55.803 INFO:teuthology.orchestra.run.smithi151.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-29T04:13:55.812 INFO:teuthology.orchestra.run.smithi151.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-29T04:13:55.814 INFO:teuthology.orchestra.run.smithi151.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2520-g69e2cd73-1jammy [2,451 kB] 2024-03-29T04:13:55.877 INFO:teuthology.orchestra.run.smithi151.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-29T04:13:55.893 INFO:teuthology.orchestra.run.smithi151.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2520-g69e2cd73-1jammy [791 kB] 2024-03-29T04:13:55.899 INFO:teuthology.orchestra.run.smithi151.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-29T04:13:55.900 INFO:teuthology.orchestra.run.smithi151.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-29T04:13:55.906 INFO:teuthology.orchestra.run.smithi151.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-29T04:13:55.962 INFO:teuthology.orchestra.run.smithi151.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2520-g69e2cd73-1jammy [147 kB] 2024-03-29T04:13:55.967 INFO:teuthology.orchestra.run.smithi151.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2520-g69e2cd73-1jammy [2,242 kB] 2024-03-29T04:13:56.073 INFO:teuthology.orchestra.run.smithi151.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-29T04:13:56.107 INFO:teuthology.orchestra.run.smithi151.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2520-g69e2cd73-1jammy [8,625 kB] 2024-03-29T04:13:56.605 INFO:teuthology.orchestra.run.smithi151.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-29T04:13:56.610 INFO:teuthology.orchestra.run.smithi151.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-29T04:13:56.635 INFO:teuthology.orchestra.run.smithi151.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2520-g69e2cd73-1jammy [14.2 kB] 2024-03-29T04:13:56.636 INFO:teuthology.orchestra.run.smithi151.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2520-g69e2cd73-1jammy [50.3 MB] 2024-03-29T04:13:56.649 INFO:teuthology.orchestra.run.smithi151.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-29T04:13:56.650 INFO:teuthology.orchestra.run.smithi151.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-29T04:13:56.657 INFO:teuthology.orchestra.run.smithi151.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-29T04:13:56.659 INFO:teuthology.orchestra.run.smithi151.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-29T04:13:56.700 INFO:teuthology.orchestra.run.smithi151.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-29T04:13:56.709 INFO:teuthology.orchestra.run.smithi151.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-29T04:13:56.782 INFO:teuthology.orchestra.run.smithi151.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-29T04:13:56.782 INFO:teuthology.orchestra.run.smithi151.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-29T04:13:56.947 INFO:teuthology.orchestra.run.smithi151.stdout:Get:115 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-29T04:13:57.169 INFO:teuthology.orchestra.run.smithi022.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2520-g69e2cd73-1jammy [14.0 kB] 2024-03-29T04:13:57.170 INFO:teuthology.orchestra.run.smithi022.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2520-g69e2cd73-1jammy [870 kB] 2024-03-29T04:13:57.187 INFO:teuthology.orchestra.run.smithi022.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2520-g69e2cd73-1jammy [2,451 kB] 2024-03-29T04:13:57.337 INFO:teuthology.orchestra.run.smithi022.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2520-g69e2cd73-1jammy [791 kB] 2024-03-29T04:13:57.381 INFO:teuthology.orchestra.run.smithi151.stdout:Get:116 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-29T04:13:57.396 INFO:teuthology.orchestra.run.smithi151.stdout:Get:117 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-29T04:13:57.415 INFO:teuthology.orchestra.run.smithi022.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2520-g69e2cd73-1jammy [147 kB] 2024-03-29T04:13:57.420 INFO:teuthology.orchestra.run.smithi022.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2520-g69e2cd73-1jammy [2,242 kB] 2024-03-29T04:13:57.574 INFO:teuthology.orchestra.run.smithi022.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2520-g69e2cd73-1jammy [8,625 kB] 2024-03-29T04:13:58.148 INFO:teuthology.orchestra.run.smithi022.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2520-g69e2cd73-1jammy [14.2 kB] 2024-03-29T04:13:58.148 INFO:teuthology.orchestra.run.smithi022.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2520-g69e2cd73-1jammy [50.3 MB] 2024-03-29T04:14:00.361 INFO:teuthology.orchestra.run.smithi151.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2520-g69e2cd73-1jammy [121 kB] 2024-03-29T04:14:00.363 INFO:teuthology.orchestra.run.smithi151.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2520-g69e2cd73-1jammy [230 kB] 2024-03-29T04:14:00.368 INFO:teuthology.orchestra.run.smithi151.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2520-g69e2cd73-1jammy [34.0 kB] 2024-03-29T04:14:00.369 INFO:teuthology.orchestra.run.smithi151.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2520-g69e2cd73-1jammy [23.4 kB] 2024-03-29T04:14:00.369 INFO:teuthology.orchestra.run.smithi151.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2520-g69e2cd73-1jammy [39.0 kB] 2024-03-29T04:14:00.370 INFO:teuthology.orchestra.run.smithi151.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2520-g69e2cd73-1jammy [13.2 MB] 2024-03-29T04:14:01.139 INFO:teuthology.orchestra.run.smithi151.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2520-g69e2cd73-1jammy [91.6 kB] 2024-03-29T04:14:01.613 INFO:teuthology.orchestra.run.smithi022.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2520-g69e2cd73-1jammy [121 kB] 2024-03-29T04:14:01.616 INFO:teuthology.orchestra.run.smithi022.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2520-g69e2cd73-1jammy [230 kB] 2024-03-29T04:14:01.620 INFO:teuthology.orchestra.run.smithi022.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2520-g69e2cd73-1jammy [34.0 kB] 2024-03-29T04:14:01.621 INFO:teuthology.orchestra.run.smithi022.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2520-g69e2cd73-1jammy [23.4 kB] 2024-03-29T04:14:01.622 INFO:teuthology.orchestra.run.smithi022.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2520-g69e2cd73-1jammy [39.0 kB] 2024-03-29T04:14:01.623 INFO:teuthology.orchestra.run.smithi022.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2520-g69e2cd73-1jammy [13.2 MB] 2024-03-29T04:14:01.757 INFO:teuthology.orchestra.run.smithi151.stdout:Fetched 179 MB in 11s (16.1 MB/s) 2024-03-29T04:14:01.874 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-29T04:14:01.967 INFO:teuthology.orchestra.run.smithi151.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-03-29T04:14:01.972 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-29T04:14:02.005 INFO:teuthology.orchestra.run.smithi022.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2520-g69e2cd73-1jammy [91.6 kB] 2024-03-29T04:14:02.006 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-29T04:14:02.277 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-29T04:14:02.303 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-29T04:14:02.344 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-29T04:14:02.619 INFO:teuthology.orchestra.run.smithi022.stdout:Fetched 179 MB in 12s (15.0 MB/s) 2024-03-29T04:14:02.725 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-29T04:14:02.824 INFO:teuthology.orchestra.run.smithi022.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-03-29T04:14:02.830 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-29T04:14:02.863 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-cmd2. 2024-03-29T04:14:02.873 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-29T04:14:02.890 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-29T04:14:02.930 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-29T04:14:03.167 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-29T04:14:03.194 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-29T04:14:03.234 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-29T04:14:03.284 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-29T04:14:03.310 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-29T04:14:03.393 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-29T04:14:03.589 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-cmd2. 2024-03-29T04:14:03.614 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-29T04:14:03.654 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-29T04:14:03.688 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-29T04:14:03.715 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-29T04:14:03.747 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-29T04:14:04.023 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-29T04:14:04.050 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-29T04:14:04.084 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-29T04:14:04.110 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-29T04:14:04.132 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-29T04:14:04.150 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-29T04:14:04.444 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-29T04:14:04.471 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-29T04:14:04.502 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-29T04:14:04.571 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-29T04:14:04.598 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-29T04:14:04.646 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T04:14:04.821 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-29T04:14:04.848 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-29T04:14:04.888 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-29T04:14:05.050 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-29T04:14:05.077 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-29T04:14:05.117 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T04:14:05.341 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-29T04:14:05.369 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-29T04:14:05.416 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T04:14:05.462 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-29T04:14:05.489 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-29T04:14:05.529 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T04:14:05.769 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-29T04:14:05.796 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-29T04:14:05.828 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T04:14:05.875 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-29T04:14:05.901 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-29T04:14:05.941 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-29T04:14:06.156 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-29T04:14:06.182 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-29T04:14:06.222 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T04:14:06.261 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package librados2. 2024-03-29T04:14:06.288 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../010-librados2_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:06.328 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking librados2 (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:06.709 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-29T04:14:06.735 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-29T04:14:06.775 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-29T04:14:06.781 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package librbd1. 2024-03-29T04:14:06.808 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../011-librbd1_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:06.848 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking librbd1 (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:07.103 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package librados2. 2024-03-29T04:14:07.129 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../010-librados2_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:07.169 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking librados2 (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:07.285 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libcephfs2. 2024-03-29T04:14:07.312 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:07.343 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libcephfs2 (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:07.630 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-rados. 2024-03-29T04:14:07.639 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package librbd1. 2024-03-29T04:14:07.656 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:07.666 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../011-librbd1_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:07.696 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-rados (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:07.705 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking librbd1 (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:08.075 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libcephfs2. 2024-03-29T04:14:08.088 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:08.116 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libcephfs2 (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:08.142 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-29T04:14:08.168 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2520-g69e2cd73-1jammy_all.deb ... 2024-03-29T04:14:08.208 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-ceph-argparse (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:08.394 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-rados. 2024-03-29T04:14:08.421 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:08.461 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-rados (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:08.495 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-cephfs. 2024-03-29T04:14:08.521 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:08.561 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-cephfs (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:08.856 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-29T04:14:08.883 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2520-g69e2cd73-1jammy_all.deb ... 2024-03-29T04:14:08.923 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-ceph-common (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:09.013 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-29T04:14:09.040 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2520-g69e2cd73-1jammy_all.deb ... 2024-03-29T04:14:09.080 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-ceph-argparse (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:09.210 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-prettytable. 2024-03-29T04:14:09.236 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-03-29T04:14:09.268 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-29T04:14:09.326 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-cephfs. 2024-03-29T04:14:09.340 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:09.374 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-cephfs (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:09.530 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-rbd. 2024-03-29T04:14:09.545 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:09.588 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-rbd (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:09.679 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-29T04:14:09.706 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2520-g69e2cd73-1jammy_all.deb ... 2024-03-29T04:14:09.737 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-ceph-common (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:09.934 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-29T04:14:09.960 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-29T04:14:10.000 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-29T04:14:10.023 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-prettytable. 2024-03-29T04:14:10.050 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-03-29T04:14:10.090 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-29T04:14:10.262 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-29T04:14:10.277 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-29T04:14:10.320 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-29T04:14:10.350 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-rbd. 2024-03-29T04:14:10.366 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:10.409 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-rbd (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:10.623 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-29T04:14:10.639 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-29T04:14:10.690 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-29T04:14:10.762 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-29T04:14:10.789 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-29T04:14:10.828 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-29T04:14:11.011 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package lua5.1. 2024-03-29T04:14:11.038 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-29T04:14:11.077 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-29T04:14:11.147 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-29T04:14:11.164 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-29T04:14:11.206 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-29T04:14:11.373 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package lua-any. 2024-03-29T04:14:11.388 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-03-29T04:14:11.431 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-29T04:14:11.500 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-29T04:14:11.517 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-29T04:14:11.550 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-29T04:14:11.667 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package zip. 2024-03-29T04:14:11.683 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-03-29T04:14:11.726 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking zip (3.0-12build2) ... 2024-03-29T04:14:11.828 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package lua5.1. 2024-03-29T04:14:11.855 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-29T04:14:11.894 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-29T04:14:12.071 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package unzip. 2024-03-29T04:14:12.098 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-29T04:14:12.138 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-29T04:14:12.214 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package lua-any. 2024-03-29T04:14:12.230 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-03-29T04:14:12.272 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-29T04:14:12.518 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package luarocks. 2024-03-29T04:14:12.525 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package zip. 2024-03-29T04:14:12.541 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-03-29T04:14:12.545 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-29T04:14:12.583 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking zip (3.0-12build2) ... 2024-03-29T04:14:12.584 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-29T04:14:12.878 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package unzip. 2024-03-29T04:14:12.894 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-29T04:14:12.929 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-29T04:14:13.094 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package librgw2. 2024-03-29T04:14:13.121 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../027-librgw2_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:13.161 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking librgw2 (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:13.332 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package luarocks. 2024-03-29T04:14:13.359 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-29T04:14:13.398 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-29T04:14:13.697 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-rgw. 2024-03-29T04:14:13.712 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:13.755 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-rgw (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:13.764 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package librgw2. 2024-03-29T04:14:13.791 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../027-librgw2_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:13.830 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking librgw2 (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:14.067 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-29T04:14:14.083 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-29T04:14:14.117 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-29T04:14:14.383 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-rgw. 2024-03-29T04:14:14.399 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:14.412 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libradosstriper1. 2024-03-29T04:14:14.439 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:14.441 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-rgw (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:14.479 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libradosstriper1 (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:14.819 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-29T04:14:14.846 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-29T04:14:14.885 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-29T04:14:14.908 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-common. 2024-03-29T04:14:14.935 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:14.975 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-common (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:15.205 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libradosstriper1. 2024-03-29T04:14:15.232 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:15.271 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libradosstriper1 (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:15.641 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-common. 2024-03-29T04:14:15.669 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:15.699 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-common (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:16.356 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-base. 2024-03-29T04:14:16.383 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:16.432 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-base (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:17.020 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-29T04:14:17.047 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-29T04:14:17.086 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-29T04:14:17.171 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-base. 2024-03-29T04:14:17.198 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:17.247 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-base (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:17.348 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-cheroot. 2024-03-29T04:14:17.375 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-29T04:14:17.406 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-29T04:14:17.702 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-29T04:14:17.729 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-29T04:14:17.769 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-29T04:14:17.784 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-29T04:14:17.811 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-29T04:14:17.850 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-29T04:14:18.072 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-29T04:14:18.098 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-29T04:14:18.139 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-29T04:14:18.153 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-cheroot. 2024-03-29T04:14:18.180 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-29T04:14:18.219 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-29T04:14:18.522 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-29T04:14:18.539 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-29T04:14:18.572 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-29T04:14:18.634 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-29T04:14:18.661 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-29T04:14:18.701 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-29T04:14:18.833 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-29T04:14:18.849 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-29T04:14:18.892 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-29T04:14:18.971 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-tempora. 2024-03-29T04:14:18.987 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-03-29T04:14:19.047 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-29T04:14:19.349 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-portend. 2024-03-29T04:14:19.376 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-03-29T04:14:19.416 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-29T04:14:19.508 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-29T04:14:19.535 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-29T04:14:19.577 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-29T04:14:19.720 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-29T04:14:19.747 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-29T04:14:19.786 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-29T04:14:19.889 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-tempora. 2024-03-29T04:14:19.916 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-03-29T04:14:19.955 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-29T04:14:20.073 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-29T04:14:20.100 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-29T04:14:20.140 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-29T04:14:20.216 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-portend. 2024-03-29T04:14:20.233 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-03-29T04:14:20.266 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-29T04:14:20.470 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-natsort. 2024-03-29T04:14:20.485 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-03-29T04:14:20.519 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-29T04:14:20.527 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-29T04:14:20.554 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-29T04:14:20.594 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-29T04:14:20.823 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-logutils. 2024-03-29T04:14:20.850 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-03-29T04:14:20.890 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-29T04:14:20.905 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-29T04:14:20.932 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-29T04:14:20.971 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-29T04:14:21.202 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-29T04:14:21.228 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-29T04:14:21.258 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-natsort. 2024-03-29T04:14:21.268 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-29T04:14:21.285 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-03-29T04:14:21.325 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-29T04:14:21.538 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-mako. 2024-03-29T04:14:21.565 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-29T04:14:21.597 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-29T04:14:21.636 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-logutils. 2024-03-29T04:14:21.663 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-03-29T04:14:21.702 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-29T04:14:21.926 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-29T04:14:21.953 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-29T04:14:21.993 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-29T04:14:21.997 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-29T04:14:22.025 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-29T04:14:22.064 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-29T04:14:22.296 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-29T04:14:22.323 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-29T04:14:22.341 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-mako. 2024-03-29T04:14:22.362 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-29T04:14:22.369 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-29T04:14:22.408 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-29T04:14:22.674 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-webob. 2024-03-29T04:14:22.701 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-29T04:14:22.741 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-29T04:14:22.752 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-29T04:14:22.780 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-29T04:14:22.811 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-29T04:14:23.053 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-waitress. 2024-03-29T04:14:23.080 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-29T04:14:23.105 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-29T04:14:23.114 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-29T04:14:23.133 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-29T04:14:23.172 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-29T04:14:23.823 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-tempita. 2024-03-29T04:14:23.851 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-29T04:14:24.175 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-webob. 2024-03-29T04:14:24.202 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-29T04:14:24.240 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-29T04:14:24.241 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-29T04:14:24.544 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-paste. 2024-03-29T04:14:24.571 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-29T04:14:24.594 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-waitress. 2024-03-29T04:14:24.610 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-29T04:14:24.621 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-29T04:14:24.664 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-29T04:14:24.931 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-29T04:14:24.946 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-29T04:14:24.964 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-tempita. 2024-03-29T04:14:24.989 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-29T04:14:24.991 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-29T04:14:25.030 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-29T04:14:25.301 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-29T04:14:25.308 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-paste. 2024-03-29T04:14:25.324 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-29T04:14:25.328 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-29T04:14:25.358 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-29T04:14:25.368 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-29T04:14:25.647 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-webtest. 2024-03-29T04:14:25.673 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-03-29T04:14:25.677 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-29T04:14:25.705 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-29T04:14:25.705 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-29T04:14:25.744 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-29T04:14:25.992 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pecan. 2024-03-29T04:14:26.019 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-29T04:14:26.030 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-29T04:14:26.058 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-29T04:14:26.058 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-29T04:14:26.088 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-29T04:14:26.366 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-webtest. 2024-03-29T04:14:26.394 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-03-29T04:14:26.422 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-certifi. 2024-03-29T04:14:26.433 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-29T04:14:26.450 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-03-29T04:14:26.489 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-29T04:14:26.736 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pecan. 2024-03-29T04:14:26.763 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-29T04:14:26.802 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-29T04:14:26.818 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-idna. 2024-03-29T04:14:26.845 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-03-29T04:14:26.884 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-29T04:14:27.157 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-certifi. 2024-03-29T04:14:27.179 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-urllib3. 2024-03-29T04:14:27.184 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-03-29T04:14:27.207 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-29T04:14:27.223 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-29T04:14:27.238 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-29T04:14:27.533 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-requests. 2024-03-29T04:14:27.549 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-29T04:14:27.559 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-idna. 2024-03-29T04:14:27.583 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-29T04:14:27.587 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-03-29T04:14:27.626 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-29T04:14:27.879 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-29T04:14:27.906 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-29T04:14:27.920 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-urllib3. 2024-03-29T04:14:27.945 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-29T04:14:27.948 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-29T04:14:27.987 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-29T04:14:28.224 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-29T04:14:28.251 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2520-g69e2cd73-1jammy_all.deb ... 2024-03-29T04:14:28.290 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-requests. 2024-03-29T04:14:28.291 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:28.318 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-29T04:14:28.348 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-29T04:14:28.626 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-29T04:14:28.638 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-29T04:14:28.642 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-29T04:14:28.665 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:28.676 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-29T04:14:28.705 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:28.962 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-29T04:14:28.989 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2520-g69e2cd73-1jammy_all.deb ... 2024-03-29T04:14:29.028 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:29.175 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-29T04:14:29.191 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-29T04:14:29.225 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-29T04:14:29.392 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-29T04:14:29.419 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:29.458 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:29.537 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-openssl. 2024-03-29T04:14:29.563 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-03-29T04:14:29.603 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-29T04:14:29.953 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-29T04:14:29.974 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr. 2024-03-29T04:14:29.979 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-29T04:14:30.002 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:30.019 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-29T04:14:30.040 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:30.297 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-openssl. 2024-03-29T04:14:30.313 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-03-29T04:14:30.347 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-29T04:14:30.444 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mon. 2024-03-29T04:14:30.471 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:30.511 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mon (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:30.683 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mgr. 2024-03-29T04:14:30.711 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:30.749 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mgr (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:31.082 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-osd. 2024-03-29T04:14:31.109 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:31.136 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mon. 2024-03-29T04:14:31.148 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-osd (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:31.163 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:31.202 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mon (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:31.748 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-osd. 2024-03-29T04:14:31.775 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:31.805 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-osd (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:32.177 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph. 2024-03-29T04:14:32.193 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../068-ceph_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:32.236 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:32.514 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-fuse. 2024-03-29T04:14:32.541 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:32.580 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-fuse (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:32.883 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph. 2024-03-29T04:14:32.899 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../068-ceph_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:32.942 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:32.959 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mds. 2024-03-29T04:14:32.986 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:33.025 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mds (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:33.277 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-fuse. 2024-03-29T04:14:33.305 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:33.344 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-fuse (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:33.454 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package cephadm. 2024-03-29T04:14:33.481 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../071-cephadm_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:33.521 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking cephadm (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:33.689 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mds. 2024-03-29T04:14:33.716 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:33.747 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mds (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:33.807 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-29T04:14:33.835 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-29T04:14:33.866 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-29T04:14:34.133 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package cephadm. 2024-03-29T04:14:34.160 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../071-cephadm_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:34.177 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-29T04:14:34.193 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-29T04:14:34.199 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking cephadm (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:34.227 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-29T04:14:34.552 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-29T04:14:34.580 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-29T04:14:34.618 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-29T04:14:34.930 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-29T04:14:34.945 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-29T04:14:34.988 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-29T04:14:35.209 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-babel. 2024-03-29T04:14:35.236 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-29T04:14:35.275 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-29T04:14:35.606 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-jinja2. 2024-03-29T04:14:35.633 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-29T04:14:35.672 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-29T04:14:36.018 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-29T04:14:36.045 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2520-g69e2cd73-1jammy_all.deb ... 2024-03-29T04:14:36.076 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-babel. 2024-03-29T04:14:36.084 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:36.104 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-29T04:14:36.142 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-29T04:14:36.405 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-29T04:14:36.421 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-03-29T04:14:36.462 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-jinja2. 2024-03-29T04:14:36.463 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-29T04:14:36.489 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-29T04:14:36.528 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-29T04:14:36.742 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-routes. 2024-03-29T04:14:36.769 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-29T04:14:36.808 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-29T04:14:36.873 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-29T04:14:36.901 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2520-g69e2cd73-1jammy_all.deb ... 2024-03-29T04:14:36.939 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:37.171 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-29T04:14:37.198 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2520-g69e2cd73-1jammy_all.deb ... 2024-03-29T04:14:37.237 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:37.259 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-29T04:14:37.287 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-03-29T04:14:37.326 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-29T04:14:37.629 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-routes. 2024-03-29T04:14:37.645 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-29T04:14:37.686 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-29T04:14:38.057 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-29T04:14:38.084 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2520-g69e2cd73-1jammy_all.deb ... 2024-03-29T04:14:38.123 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:38.560 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-29T04:14:38.589 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-29T04:14:38.626 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-29T04:14:39.231 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-joblib. 2024-03-29T04:14:39.259 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-29T04:14:39.298 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-29T04:14:39.344 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-29T04:14:39.361 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-29T04:14:39.394 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-29T04:14:39.652 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-29T04:14:39.681 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-29T04:14:39.796 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-29T04:14:39.890 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-joblib. 2024-03-29T04:14:39.917 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-29T04:14:39.956 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-29T04:14:40.083 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-sklearn. 2024-03-29T04:14:40.111 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-29T04:14:40.150 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-29T04:14:40.326 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-29T04:14:40.354 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-29T04:14:40.486 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-29T04:14:40.764 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-sklearn. 2024-03-29T04:14:40.792 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-29T04:14:40.822 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-29T04:14:40.870 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-29T04:14:40.898 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2520-g69e2cd73-1jammy_all.deb ... 2024-03-29T04:14:40.936 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:41.523 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-29T04:14:41.551 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2520-g69e2cd73-1jammy_all.deb ... 2024-03-29T04:14:41.589 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:41.865 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-cachetools. 2024-03-29T04:14:41.893 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-03-29T04:14:41.931 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-29T04:14:42.218 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-29T04:14:42.246 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-29T04:14:42.277 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-29T04:14:42.567 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-cachetools. 2024-03-29T04:14:42.589 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-29T04:14:42.595 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-03-29T04:14:42.617 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-29T04:14:42.633 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-29T04:14:42.656 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-29T04:14:42.920 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-29T04:14:42.948 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-29T04:14:42.984 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-rsa. 2024-03-29T04:14:42.986 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-29T04:14:43.012 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-03-29T04:14:43.051 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-29T04:14:43.331 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-29T04:14:43.355 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-google-auth. 2024-03-29T04:14:43.359 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-29T04:14:43.383 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-03-29T04:14:43.398 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-29T04:14:43.422 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-29T04:14:43.709 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-rsa. 2024-03-29T04:14:43.709 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-29T04:14:43.737 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-29T04:14:43.737 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-03-29T04:14:43.775 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-29T04:14:43.775 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-29T04:14:44.079 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-google-auth. 2024-03-29T04:14:44.088 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-websocket. 2024-03-29T04:14:44.107 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-03-29T04:14:44.115 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-03-29T04:14:44.137 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-29T04:14:44.171 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-29T04:14:44.415 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-29T04:14:44.443 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-29T04:14:44.475 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-29T04:14:44.482 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-29T04:14:44.503 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-29T04:14:44.542 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-29T04:14:44.818 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-websocket. 2024-03-29T04:14:44.847 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-03-29T04:14:44.884 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-29T04:14:45.205 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-29T04:14:45.232 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-29T04:14:45.316 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-29T04:14:45.344 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2520-g69e2cd73-1jammy_all.deb ... 2024-03-29T04:14:45.382 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:45.517 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-29T04:14:45.660 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-29T04:14:45.688 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-29T04:14:45.719 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-29T04:14:46.064 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-29T04:14:46.092 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-29T04:14:46.131 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-29T04:14:46.226 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-29T04:14:46.243 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2520-g69e2cd73-1jammy_all.deb ... 2024-03-29T04:14:46.284 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:46.392 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package jq. 2024-03-29T04:14:46.408 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-29T04:14:46.450 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-29T04:14:46.604 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-29T04:14:46.632 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-29T04:14:46.670 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-29T04:14:46.686 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package socat. 2024-03-29T04:14:46.703 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-29T04:14:46.745 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-29T04:14:46.990 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-29T04:14:47.018 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-29T04:14:47.049 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package xmlstarlet. 2024-03-29T04:14:47.056 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-29T04:14:47.065 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-29T04:14:47.107 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-29T04:14:47.300 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package jq. 2024-03-29T04:14:47.317 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-29T04:14:47.358 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-29T04:14:47.479 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-test. 2024-03-29T04:14:47.507 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:47.546 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-test (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:47.611 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package socat. 2024-03-29T04:14:47.628 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-29T04:14:47.669 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-29T04:14:47.941 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package xmlstarlet. 2024-03-29T04:14:47.958 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-29T04:14:47.991 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-29T04:14:48.362 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-test. 2024-03-29T04:14:48.390 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:48.429 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-test (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:50.035 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-volume. 2024-03-29T04:14:50.062 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2520-g69e2cd73-1jammy_all.deb ... 2024-03-29T04:14:50.101 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-volume (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:50.465 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-29T04:14:50.492 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:50.531 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking cephfs-mirror (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:50.725 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ceph-volume. 2024-03-29T04:14:50.753 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2520-g69e2cd73-1jammy_all.deb ... 2024-03-29T04:14:50.791 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ceph-volume (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:50.843 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-colorama. 2024-03-29T04:14:50.870 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-03-29T04:14:50.909 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-29T04:14:51.163 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package cephfs-shell. 2024-03-29T04:14:51.179 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-29T04:14:51.190 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2520-g69e2cd73-1jammy_all.deb ... 2024-03-29T04:14:51.207 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:51.238 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking cephfs-shell (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:51.245 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking cephfs-mirror (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:51.515 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-colorama. 2024-03-29T04:14:51.531 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-03-29T04:14:51.533 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package cephfs-top. 2024-03-29T04:14:51.560 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2520-g69e2cd73-1jammy_all.deb ... 2024-03-29T04:14:51.565 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-29T04:14:51.599 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking cephfs-top (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:51.792 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package cephfs-shell. 2024-03-29T04:14:51.809 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2520-g69e2cd73-1jammy_all.deb ... 2024-03-29T04:14:51.851 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking cephfs-shell (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:51.852 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-29T04:14:51.876 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:51.919 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libcephfs-dev (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:52.120 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package cephfs-top. 2024-03-29T04:14:52.136 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2520-g69e2cd73-1jammy_all.deb ... 2024-03-29T04:14:52.178 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking cephfs-top (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:52.231 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-29T04:14:52.258 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-29T04:14:52.297 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-29T04:14:52.431 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-29T04:14:52.447 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:52.489 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libcephfs-dev (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:52.603 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-29T04:14:52.630 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-03-29T04:14:52.669 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-29T04:14:52.817 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-29T04:14:52.845 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-29T04:14:52.883 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-29T04:14:53.016 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package nvme-cli. 2024-03-29T04:14:53.043 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-29T04:14:53.082 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-29T04:14:53.238 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-29T04:14:53.266 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-03-29T04:14:53.304 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-29T04:14:53.489 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package pkg-config. 2024-03-29T04:14:53.517 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-29T04:14:53.547 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-29T04:14:53.634 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package nvme-cli. 2024-03-29T04:14:53.662 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-29T04:14:53.692 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-29T04:14:53.801 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-29T04:14:53.817 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-29T04:14:53.859 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-29T04:14:54.064 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package pkg-config. 2024-03-29T04:14:54.092 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-29T04:14:54.131 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-29T04:14:54.288 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-29T04:14:54.315 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-29T04:14:54.354 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-29T04:14:54.383 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-29T04:14:54.400 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-29T04:14:54.442 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-29T04:14:54.658 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pastescript. 2024-03-29T04:14:54.685 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-03-29T04:14:54.724 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-29T04:14:54.895 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-29T04:14:54.923 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-29T04:14:54.961 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-29T04:14:55.079 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pluggy. 2024-03-29T04:14:55.106 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-29T04:14:55.137 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-29T04:14:55.281 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pastescript. 2024-03-29T04:14:55.309 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-03-29T04:14:55.347 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-29T04:14:55.441 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-psutil. 2024-03-29T04:14:55.468 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-29T04:14:55.507 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-29T04:14:55.668 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pluggy. 2024-03-29T04:14:55.696 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-29T04:14:55.743 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-29T04:14:55.836 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-py. 2024-03-29T04:14:55.864 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-03-29T04:14:55.902 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-29T04:14:56.012 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-psutil. 2024-03-29T04:14:56.029 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-29T04:14:56.070 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-29T04:14:56.240 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pygments. 2024-03-29T04:14:56.268 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-29T04:14:56.306 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-29T04:14:56.382 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-py. 2024-03-29T04:14:56.410 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-03-29T04:14:56.457 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-29T04:14:56.735 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pygments. 2024-03-29T04:14:56.764 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-29T04:14:56.769 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-29T04:14:56.798 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-29T04:14:56.801 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-29T04:14:56.836 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-29T04:14:57.123 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-toml. 2024-03-29T04:14:57.151 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-03-29T04:14:57.189 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-29T04:14:57.339 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-29T04:14:57.368 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-29T04:14:57.405 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-29T04:14:57.485 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pytest. 2024-03-29T04:14:57.512 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-29T04:14:57.551 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-29T04:14:57.725 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-toml. 2024-03-29T04:14:57.753 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-03-29T04:14:57.792 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-29T04:14:57.897 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-simplejson. 2024-03-29T04:14:57.925 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-29T04:14:57.963 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-29T04:14:58.053 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-pytest. 2024-03-29T04:14:58.081 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-29T04:14:58.111 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-29T04:14:58.225 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-29T04:14:58.241 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-29T04:14:58.283 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-29T04:14:58.465 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package python3-simplejson. 2024-03-29T04:14:58.493 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-29T04:14:58.531 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-29T04:14:58.826 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-29T04:14:58.851 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-29T04:14:58.892 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-29T04:14:59.005 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package radosgw. 2024-03-29T04:14:59.033 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../122-radosgw_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:59.063 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking radosgw (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:59.588 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package radosgw. 2024-03-29T04:14:59.616 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../122-radosgw_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:59.654 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking radosgw (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:14:59.851 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package rbd-fuse. 2024-03-29T04:14:59.879 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:14:59.917 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking rbd-fuse (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:00.231 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-29T04:15:00.482 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-29T04:15:00.541 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package rbd-fuse. 2024-03-29T04:15:00.558 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2520-g69e2cd73-1jammy_amd64.deb ... 2024-03-29T04:15:00.599 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking rbd-fuse (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:00.608 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-29T04:15:00.868 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-29T04:15:00.954 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-29T04:15:01.184 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-29T04:15:01.204 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-29T04:15:01.330 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-29T04:15:01.593 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-29T04:15:01.613 INFO:teuthology.orchestra.run.smithi151.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-29T04:15:01.677 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up cephadm (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:01.930 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-29T04:15:02.220 INFO:teuthology.orchestra.run.smithi151.stdout:Adding system user cephadm....done 2024-03-29T04:15:02.376 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-29T04:15:02.378 INFO:teuthology.orchestra.run.smithi022.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-29T04:15:02.449 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up cephadm (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:02.653 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-29T04:15:02.888 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-29T04:15:02.965 INFO:teuthology.orchestra.run.smithi022.stdout:Adding system user cephadm....done 2024-03-29T04:15:03.014 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-29T04:15:03.113 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-29T04:15:03.274 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-29T04:15:03.431 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-29T04:15:03.534 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-29T04:15:03.652 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-29T04:15:03.699 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-29T04:15:03.817 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-29T04:15:03.937 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-29T04:15:04.076 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-29T04:15:04.298 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-29T04:15:04.336 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-29T04:15:04.454 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-29T04:15:04.558 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-29T04:15:04.780 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-29T04:15:04.826 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-29T04:15:05.102 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-29T04:15:05.141 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-29T04:15:05.400 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-29T04:15:05.412 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-29T04:15:05.651 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-29T04:15:05.672 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-29T04:15:05.928 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-29T04:15:05.932 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-29T04:15:06.221 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-29T04:15:06.275 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-ceph-argparse (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:06.472 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-29T04:15:06.535 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-29T04:15:06.636 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-29T04:15:06.732 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-29T04:15:06.754 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-29T04:15:07.005 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-29T04:15:07.017 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-ceph-argparse (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:07.140 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-29T04:15:07.268 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-29T04:15:07.369 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-29T04:15:07.399 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-29T04:15:07.470 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-29T04:15:07.754 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-29T04:15:07.855 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-29T04:15:07.900 INFO:teuthology.orchestra.run.smithi151.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-29T04:15:07.932 INFO:teuthology.orchestra.run.smithi151.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-29T04:15:07.995 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-29T04:15:08.106 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-29T04:15:08.113 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-29T04:15:08.462 INFO:teuthology.orchestra.run.smithi022.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-29T04:15:08.473 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-29T04:15:08.495 INFO:teuthology.orchestra.run.smithi022.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-29T04:15:08.585 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-29T04:15:08.599 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-29T04:15:08.710 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-29T04:15:08.867 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-29T04:15:09.079 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-29T04:15:09.127 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-29T04:15:09.204 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-29T04:15:09.428 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up zip (3.0-12build2) ... 2024-03-29T04:15:09.506 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-29T04:15:09.546 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-29T04:15:09.799 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-29T04:15:10.108 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up zip (3.0-12build2) ... 2024-03-29T04:15:10.199 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-29T04:15:10.234 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-29T04:15:10.459 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-29T04:15:10.593 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-29T04:15:10.711 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-29T04:15:10.978 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-29T04:15:11.004 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-29T04:15:11.237 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-29T04:15:11.339 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-29T04:15:11.381 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-29T04:15:11.448 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-29T04:15:11.624 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-29T04:15:11.758 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-29T04:15:12.009 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-29T04:15:12.151 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-29T04:15:12.294 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-29T04:15:12.418 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-29T04:15:12.595 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-29T04:15:12.803 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-29T04:15:12.922 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-29T04:15:13.104 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-29T04:15:13.182 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-29T04:15:13.308 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-ceph-common (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:13.397 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-29T04:15:13.602 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-idna (3.3-1) ... 2024-03-29T04:15:13.749 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-29T04:15:13.903 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-29T04:15:14.000 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-29T04:15:14.101 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-ceph-common (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:14.411 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-idna (3.3-1) ... 2024-03-29T04:15:14.728 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-29T04:15:15.102 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-29T04:15:15.354 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T04:15:15.480 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-29T04:15:15.750 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-29T04:15:15.876 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-29T04:15:15.910 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-29T04:15:16.002 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-29T04:15:16.312 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-29T04:15:16.570 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T04:15:16.604 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-29T04:15:16.687 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-29T04:15:16.847 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T04:15:16.947 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-29T04:15:16.973 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-29T04:15:17.073 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-29T04:15:17.199 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-29T04:15:17.282 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-29T04:15:17.484 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-29T04:15:17.534 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-29T04:15:17.727 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-29T04:15:17.776 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-29T04:15:17.978 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T04:15:18.027 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-29T04:15:18.104 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-29T04:15:18.278 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-29T04:15:18.406 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-29T04:15:18.546 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-29T04:15:18.647 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-29T04:15:18.665 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-29T04:15:18.924 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-29T04:15:18.931 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-29T04:15:19.057 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-29T04:15:19.176 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-29T04:15:19.367 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-29T04:15:19.410 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-29T04:15:19.684 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-29T04:15:19.686 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-29T04:15:19.795 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-29T04:15:19.927 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-29T04:15:20.061 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-29T04:15:20.072 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-29T04:15:20.187 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-29T04:15:20.198 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-29T04:15:20.276 INFO:teuthology.orchestra.run.smithi151.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-29T04:15:20.499 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-29T04:15:20.792 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-29T04:15:20.805 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-29T04:15:21.006 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-29T04:15:21.060 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-29T04:15:21.186 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-29T04:15:21.312 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-29T04:15:21.383 INFO:teuthology.orchestra.run.smithi022.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-29T04:15:21.399 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-29T04:15:21.639 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-29T04:15:21.667 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-29T04:15:21.765 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-29T04:15:21.776 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-29T04:15:22.019 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T04:15:22.145 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-29T04:15:22.166 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-29T04:15:22.442 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-29T04:15:22.471 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-29T04:15:22.544 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-29T04:15:22.764 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-29T04:15:22.803 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T04:15:22.912 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-29T04:15:23.031 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-29T04:15:23.222 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-29T04:15:23.243 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-29T04:15:23.502 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-29T04:15:23.548 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-29T04:15:23.795 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-29T04:15:23.841 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-29T04:15:24.055 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-29T04:15:24.086 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-29T04:15:24.379 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-29T04:15:24.448 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-29T04:15:24.647 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-29T04:15:24.741 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-29T04:15:24.915 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-29T04:15:25.110 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-29T04:15:25.236 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up librados2 (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:25.308 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-29T04:15:25.354 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:25.455 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-29T04:15:25.601 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-29T04:15:25.791 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libcephfs2 (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:25.934 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libradosstriper1 (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:25.979 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-29T04:15:26.060 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-29T04:15:26.105 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up librados2 (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:26.231 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:26.336 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up librbd1 (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:26.357 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-29T04:15:26.462 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr-modules-core (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:26.588 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-fuse (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:26.635 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libcephfs2 (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:26.761 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libradosstriper1 (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:26.800 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-29T04:15:26.800 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-29T04:15:26.887 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-29T04:15:27.163 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up librbd1 (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:27.289 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mgr-modules-core (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:27.331 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libcephfs-dev (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:27.424 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-fuse (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:27.456 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-rados (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:27.591 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up librgw2 (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:27.646 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-29T04:15:27.646 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-29T04:15:27.734 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up cephfs-top (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:27.851 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-29T04:15:28.175 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libcephfs-dev (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:28.301 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-rados (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:28.427 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up librgw2 (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:28.545 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up cephfs-top (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:28.646 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-29T04:15:29.046 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-rbd (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:29.172 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up rbd-fuse (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:29.298 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-rgw (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:29.424 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-cephfs (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:29.550 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up cephfs-shell (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:29.676 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-common (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:29.850 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-rbd (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:29.976 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up rbd-fuse (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:30.102 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-rgw (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:30.210 INFO:teuthology.orchestra.run.smithi151.stdout:Adding group ceph....done 2024-03-29T04:15:30.228 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up python3-cephfs (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:30.329 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up cephfs-shell (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:30.430 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-common (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:30.570 INFO:teuthology.orchestra.run.smithi151.stdout:Adding system user ceph....done 2024-03-29T04:15:30.696 INFO:teuthology.orchestra.run.smithi151.stdout:Setting system user ceph properties....done 2024-03-29T04:15:30.710 INFO:teuthology.orchestra.run.smithi151.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-29T04:15:30.845 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-29T04:15:30.965 INFO:teuthology.orchestra.run.smithi022.stdout:Adding group ceph....done 2024-03-29T04:15:31.161 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-29T04:15:31.366 INFO:teuthology.orchestra.run.smithi022.stdout:Adding system user ceph....done 2024-03-29T04:15:31.527 INFO:teuthology.orchestra.run.smithi022.stdout:Setting system user ceph properties....done 2024-03-29T04:15:31.539 INFO:teuthology.orchestra.run.smithi022.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-29T04:15:31.660 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up cephfs-mirror (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:31.686 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-29T04:15:31.886 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-29T04:15:31.886 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-29T04:15:32.059 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-29T04:15:32.548 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up cephfs-mirror (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:32.554 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-test (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:32.680 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up radosgw (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:32.764 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-29T04:15:32.764 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-29T04:15:33.142 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-29T04:15:33.142 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-29T04:15:33.233 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-test (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:33.334 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up radosgw (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:33.598 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-base (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:33.788 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-29T04:15:33.788 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-29T04:15:33.962 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-29T04:15:34.261 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-base (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:34.425 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mds (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:34.634 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-29T04:15:34.655 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-29T04:15:34.655 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-29T04:15:35.118 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:35.258 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mds (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:35.335 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-29T04:15:35.335 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-29T04:15:35.531 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-29T04:15:35.531 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-29T04:15:35.802 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-osd (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:36.043 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mgr (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:36.073 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-29T04:15:36.073 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-29T04:15:36.338 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-29T04:15:36.338 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-29T04:15:36.837 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-osd (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:36.864 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:36.976 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:37.102 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mon (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:37.290 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-29T04:15:37.291 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-29T04:15:37.314 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-29T04:15:37.314 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-29T04:15:37.751 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:37.770 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr-cephadm (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:37.877 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:37.912 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:38.028 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mon (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:38.038 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr-dashboard (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:38.189 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-volume (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:38.243 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-29T04:15:38.243 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-29T04:15:38.551 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-29T04:15:38.668 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-29T04:15:38.729 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mgr-cephadm (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:38.820 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-29T04:15:38.888 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:38.989 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-mgr-dashboard (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:39.141 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ceph-volume (19.0.0-2520-g69e2cd73-1jammy) ... 2024-03-29T04:15:39.511 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-29T04:15:39.611 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-29T04:15:39.764 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-29T04:15:39.854 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-29T04:15:40.497 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-29T04:15:40.499 INFO:teuthology.orchestra.run.smithi151.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-29T04:15:40.502 DEBUG:teuthology.orchestra.run.smithi151:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-29T04:15:40.554 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-03-29T04:15:40.751 INFO:teuthology.orchestra.run.smithi151.stdout:Building dependency tree... 2024-03-29T04:15:40.751 INFO:teuthology.orchestra.run.smithi151.stdout:Reading state information... 2024-03-29T04:15:40.921 INFO:teuthology.orchestra.run.smithi151.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-29T04:15:40.921 INFO:teuthology.orchestra.run.smithi151.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-29T04:15:40.921 INFO:teuthology.orchestra.run.smithi151.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-29T04:15:40.921 INFO:teuthology.orchestra.run.smithi151.stdout:The following packages were automatically installed and are no longer required: 2024-03-29T04:15:40.922 INFO:teuthology.orchestra.run.smithi151.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-29T04:15:40.922 INFO:teuthology.orchestra.run.smithi151.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-29T04:15:40.922 INFO:teuthology.orchestra.run.smithi151.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-29T04:15:40.922 INFO:teuthology.orchestra.run.smithi151.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-29T04:15:40.922 INFO:teuthology.orchestra.run.smithi151.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-29T04:15:40.922 INFO:teuthology.orchestra.run.smithi151.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-29T04:15:40.923 INFO:teuthology.orchestra.run.smithi151.stdout:The following additional packages will be installed: 2024-03-29T04:15:40.923 INFO:teuthology.orchestra.run.smithi151.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-29T04:15:40.924 INFO:teuthology.orchestra.run.smithi151.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-29T04:15:40.924 INFO:teuthology.orchestra.run.smithi151.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-03-29T04:15:40.924 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-29T04:15:40.924 INFO:teuthology.orchestra.run.smithi151.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-29T04:15:40.925 INFO:teuthology.orchestra.run.smithi151.stdout:Suggested packages: 2024-03-29T04:15:40.925 INFO:teuthology.orchestra.run.smithi151.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-29T04:15:40.925 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-03-29T04:15:40.925 INFO:teuthology.orchestra.run.smithi151.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-03-29T04:15:40.962 INFO:teuthology.orchestra.run.smithi151.stdout:The following NEW packages will be installed: 2024-03-29T04:15:40.963 INFO:teuthology.orchestra.run.smithi151.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-29T04:15:40.963 INFO:teuthology.orchestra.run.smithi151.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-29T04:15:40.964 INFO:teuthology.orchestra.run.smithi151.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-29T04:15:40.964 INFO:teuthology.orchestra.run.smithi151.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-29T04:15:40.964 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-29T04:15:40.964 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-29T04:15:40.964 INFO:teuthology.orchestra.run.smithi151.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-29T04:15:40.964 INFO:teuthology.orchestra.run.smithi151.stdout:The following packages will be upgraded: 2024-03-29T04:15:40.965 INFO:teuthology.orchestra.run.smithi151.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-03-29T04:15:41.145 INFO:teuthology.orchestra.run.smithi151.stdout:4 upgraded, 39 newly installed, 0 to remove and 294 not upgraded. 2024-03-29T04:15:41.145 INFO:teuthology.orchestra.run.smithi151.stdout:Need to get 55.7 MB of archives. 2024-03-29T04:15:41.145 INFO:teuthology.orchestra.run.smithi151.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-29T04:15:41.145 INFO:teuthology.orchestra.run.smithi151.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.3 [1,063 kB] 2024-03-29T04:15:41.153 INFO:teuthology.orchestra.run.smithi022.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-29T04:15:41.156 DEBUG:teuthology.orchestra.run.smithi022:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-29T04:15:41.262 INFO:teuthology.orchestra.run.smithi022.stdout:Reading package lists... 2024-03-29T04:15:41.464 INFO:teuthology.orchestra.run.smithi022.stdout:Building dependency tree... 2024-03-29T04:15:41.464 INFO:teuthology.orchestra.run.smithi022.stdout:Reading state information... 2024-03-29T04:15:41.646 INFO:teuthology.orchestra.run.smithi022.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-29T04:15:41.647 INFO:teuthology.orchestra.run.smithi022.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-29T04:15:41.647 INFO:teuthology.orchestra.run.smithi022.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-29T04:15:41.647 INFO:teuthology.orchestra.run.smithi022.stdout:The following packages were automatically installed and are no longer required: 2024-03-29T04:15:41.647 INFO:teuthology.orchestra.run.smithi022.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-29T04:15:41.647 INFO:teuthology.orchestra.run.smithi022.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-29T04:15:41.647 INFO:teuthology.orchestra.run.smithi022.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-29T04:15:41.647 INFO:teuthology.orchestra.run.smithi022.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-29T04:15:41.647 INFO:teuthology.orchestra.run.smithi022.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-29T04:15:41.648 INFO:teuthology.orchestra.run.smithi022.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-29T04:15:41.648 INFO:teuthology.orchestra.run.smithi022.stdout:The following additional packages will be installed: 2024-03-29T04:15:41.648 INFO:teuthology.orchestra.run.smithi022.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-29T04:15:41.648 INFO:teuthology.orchestra.run.smithi022.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-29T04:15:41.649 INFO:teuthology.orchestra.run.smithi022.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-03-29T04:15:41.649 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-29T04:15:41.649 INFO:teuthology.orchestra.run.smithi022.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-29T04:15:41.650 INFO:teuthology.orchestra.run.smithi022.stdout:Suggested packages: 2024-03-29T04:15:41.650 INFO:teuthology.orchestra.run.smithi022.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-29T04:15:41.650 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-03-29T04:15:41.651 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-03-29T04:15:41.694 INFO:teuthology.orchestra.run.smithi022.stdout:The following NEW packages will be installed: 2024-03-29T04:15:41.694 INFO:teuthology.orchestra.run.smithi022.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-29T04:15:41.694 INFO:teuthology.orchestra.run.smithi022.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-29T04:15:41.695 INFO:teuthology.orchestra.run.smithi022.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-29T04:15:41.695 INFO:teuthology.orchestra.run.smithi022.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-29T04:15:41.695 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-29T04:15:41.695 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-29T04:15:41.695 INFO:teuthology.orchestra.run.smithi022.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-29T04:15:41.696 INFO:teuthology.orchestra.run.smithi022.stdout:The following packages will be upgraded: 2024-03-29T04:15:41.696 INFO:teuthology.orchestra.run.smithi022.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-03-29T04:15:41.754 INFO:teuthology.orchestra.run.smithi022.stdout:4 upgraded, 39 newly installed, 0 to remove and 294 not upgraded. 2024-03-29T04:15:41.754 INFO:teuthology.orchestra.run.smithi022.stdout:Need to get 55.7 MB of archives. 2024-03-29T04:15:41.754 INFO:teuthology.orchestra.run.smithi022.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-29T04:15:41.754 INFO:teuthology.orchestra.run.smithi022.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.3 [1,063 kB] 2024-03-29T04:15:41.779 INFO:teuthology.orchestra.run.smithi151.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-29T04:15:41.795 INFO:teuthology.orchestra.run.smithi151.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-29T04:15:41.907 INFO:teuthology.orchestra.run.smithi022.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-29T04:15:41.914 INFO:teuthology.orchestra.run.smithi022.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-29T04:15:41.920 INFO:teuthology.orchestra.run.smithi151.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.3 [33.1 kB] 2024-03-29T04:15:41.921 INFO:teuthology.orchestra.run.smithi151.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.3 [24.4 kB] 2024-03-29T04:15:41.922 INFO:teuthology.orchestra.run.smithi151.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-29T04:15:41.951 INFO:teuthology.orchestra.run.smithi151.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-29T04:15:41.953 INFO:teuthology.orchestra.run.smithi151.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-29T04:15:41.954 INFO:teuthology.orchestra.run.smithi022.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.3 [33.1 kB] 2024-03-29T04:15:41.954 INFO:teuthology.orchestra.run.smithi022.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.3 [24.4 kB] 2024-03-29T04:15:41.955 INFO:teuthology.orchestra.run.smithi022.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-29T04:15:41.956 INFO:teuthology.orchestra.run.smithi151.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-29T04:15:41.959 INFO:teuthology.orchestra.run.smithi151.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-29T04:15:41.960 INFO:teuthology.orchestra.run.smithi151.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-29T04:15:41.967 INFO:teuthology.orchestra.run.smithi022.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-29T04:15:41.968 INFO:teuthology.orchestra.run.smithi022.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-29T04:15:41.970 INFO:teuthology.orchestra.run.smithi022.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-29T04:15:41.971 INFO:teuthology.orchestra.run.smithi022.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-29T04:15:41.972 INFO:teuthology.orchestra.run.smithi022.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-29T04:15:41.973 INFO:teuthology.orchestra.run.smithi022.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-29T04:15:41.975 INFO:teuthology.orchestra.run.smithi022.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-29T04:15:41.994 INFO:teuthology.orchestra.run.smithi022.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-29T04:15:41.995 INFO:teuthology.orchestra.run.smithi022.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-29T04:15:41.996 INFO:teuthology.orchestra.run.smithi022.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-29T04:15:42.126 INFO:teuthology.orchestra.run.smithi151.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-29T04:15:42.332 INFO:teuthology.orchestra.run.smithi022.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-29T04:15:42.362 INFO:teuthology.orchestra.run.smithi022.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-29T04:15:42.362 INFO:teuthology.orchestra.run.smithi022.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-29T04:15:42.373 INFO:teuthology.orchestra.run.smithi022.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-29T04:15:42.377 INFO:teuthology.orchestra.run.smithi022.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-29T04:15:42.384 INFO:teuthology.orchestra.run.smithi022.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-29T04:15:42.391 INFO:teuthology.orchestra.run.smithi022.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-29T04:15:42.407 INFO:teuthology.orchestra.run.smithi022.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-29T04:15:42.418 INFO:teuthology.orchestra.run.smithi151.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-29T04:15:42.427 INFO:teuthology.orchestra.run.smithi151.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-29T04:15:42.432 INFO:teuthology.orchestra.run.smithi151.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-29T04:15:42.445 INFO:teuthology.orchestra.run.smithi022.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-29T04:15:42.508 INFO:teuthology.orchestra.run.smithi022.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-29T04:15:42.516 INFO:teuthology.orchestra.run.smithi151.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-29T04:15:42.534 INFO:teuthology.orchestra.run.smithi022.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-29T04:15:42.558 INFO:teuthology.orchestra.run.smithi022.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-29T04:15:42.592 INFO:teuthology.orchestra.run.smithi022.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-29T04:15:42.596 INFO:teuthology.orchestra.run.smithi022.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-29T04:15:42.597 INFO:teuthology.orchestra.run.smithi022.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-29T04:15:42.597 INFO:teuthology.orchestra.run.smithi022.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-29T04:15:42.598 INFO:teuthology.orchestra.run.smithi022.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-29T04:15:42.611 INFO:teuthology.orchestra.run.smithi022.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-29T04:15:42.613 INFO:teuthology.orchestra.run.smithi022.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-29T04:15:42.910 INFO:teuthology.orchestra.run.smithi022.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-29T04:15:42.911 INFO:teuthology.orchestra.run.smithi022.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-29T04:15:42.911 INFO:teuthology.orchestra.run.smithi022.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-29T04:15:42.911 INFO:teuthology.orchestra.run.smithi022.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-29T04:15:42.911 INFO:teuthology.orchestra.run.smithi022.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-29T04:15:42.911 INFO:teuthology.orchestra.run.smithi022.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-29T04:15:42.923 INFO:teuthology.orchestra.run.smithi022.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-29T04:15:42.923 INFO:teuthology.orchestra.run.smithi022.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-29T04:15:43.346 INFO:teuthology.orchestra.run.smithi022.stdout:Preconfiguring packages ... 2024-03-29T04:15:43.529 INFO:teuthology.orchestra.run.smithi022.stdout:Fetched 55.7 MB in 1s (45.7 MB/s) 2024-03-29T04:15:43.601 INFO:teuthology.orchestra.run.smithi151.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-29T04:15:43.605 INFO:teuthology.orchestra.run.smithi151.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-29T04:15:43.606 INFO:teuthology.orchestra.run.smithi151.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-29T04:15:43.612 INFO:teuthology.orchestra.run.smithi151.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-29T04:15:43.613 INFO:teuthology.orchestra.run.smithi151.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-29T04:15:43.624 INFO:teuthology.orchestra.run.smithi151.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-29T04:15:43.790 INFO:teuthology.orchestra.run.smithi151.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-29T04:15:43.858 INFO:teuthology.orchestra.run.smithi022.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173988 files and directories currently installed.) 2024-03-29T04:15:43.863 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-29T04:15:44.006 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking util-linux (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-29T04:15:44.383 INFO:teuthology.orchestra.run.smithi151.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-29T04:15:44.384 INFO:teuthology.orchestra.run.smithi151.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-29T04:15:44.563 INFO:teuthology.orchestra.run.smithi151.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-29T04:15:44.563 INFO:teuthology.orchestra.run.smithi151.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-29T04:15:44.602 INFO:teuthology.orchestra.run.smithi151.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-29T04:15:44.637 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up util-linux (2.37.2-4ubuntu3.3) ... 2024-03-29T04:15:44.662 INFO:teuthology.orchestra.run.smithi151.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-29T04:15:44.669 INFO:teuthology.orchestra.run.smithi151.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-29T04:15:44.670 INFO:teuthology.orchestra.run.smithi151.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-29T04:15:44.671 INFO:teuthology.orchestra.run.smithi151.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-29T04:15:44.672 INFO:teuthology.orchestra.run.smithi151.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-29T04:15:44.877 INFO:teuthology.orchestra.run.smithi151.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-29T04:15:45.291 INFO:teuthology.orchestra.run.smithi151.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-29T04:15:45.576 INFO:teuthology.orchestra.run.smithi022.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-03-29T04:15:45.871 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package flex. 2024-03-29T04:15:45.976 INFO:teuthology.orchestra.run.smithi022.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173988 files and directories currently installed.) 2024-03-29T04:15:45.981 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-29T04:15:46.027 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-29T04:15:46.103 INFO:teuthology.orchestra.run.smithi151.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-29T04:15:46.103 INFO:teuthology.orchestra.run.smithi151.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-29T04:15:46.103 INFO:teuthology.orchestra.run.smithi151.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-29T04:15:46.103 INFO:teuthology.orchestra.run.smithi151.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-29T04:15:46.135 INFO:teuthology.orchestra.run.smithi151.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-29T04:15:46.136 INFO:teuthology.orchestra.run.smithi151.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-29T04:15:46.166 INFO:teuthology.orchestra.run.smithi151.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-29T04:15:46.167 INFO:teuthology.orchestra.run.smithi151.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-29T04:15:46.552 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-29T04:15:46.570 INFO:teuthology.orchestra.run.smithi151.stdout:Preconfiguring packages ... 2024-03-29T04:15:46.724 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-29T04:15:46.729 INFO:teuthology.orchestra.run.smithi151.stdout:Fetched 55.7 MB in 5s (10.7 MB/s) 2024-03-29T04:15:47.048 INFO:teuthology.orchestra.run.smithi151.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173988 files and directories currently installed.) 2024-03-29T04:15:47.054 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-29T04:15:47.186 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-29T04:15:47.192 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking util-linux (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-29T04:15:47.715 INFO:teuthology.orchestra.run.smithi022.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174061 files and directories currently installed.) 2024-03-29T04:15:47.720 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-29T04:15:47.804 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up util-linux (2.37.2-4ubuntu3.3) ... 2024-03-29T04:15:47.824 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-29T04:15:48.284 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-29T04:15:48.422 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-29T04:15:48.769 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.3) ... 2024-03-29T04:15:48.795 INFO:teuthology.orchestra.run.smithi151.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-03-29T04:15:49.047 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package flex. 2024-03-29T04:15:49.141 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package bison. 2024-03-29T04:15:49.147 INFO:teuthology.orchestra.run.smithi151.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173988 files and directories currently installed.) 2024-03-29T04:15:49.153 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-29T04:15:49.194 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-29T04:15:49.247 INFO:teuthology.orchestra.run.smithi022.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174061 files and directories currently installed.) 2024-03-29T04:15:49.254 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-29T04:15:49.301 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-29T04:15:49.702 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-29T04:15:49.702 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-29T04:15:49.730 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-29T04:15:49.768 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-29T04:15:49.850 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-29T04:15:50.113 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package dump. 2024-03-29T04:15:50.142 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-29T04:15:50.180 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking dump (0.4b47-3) ... 2024-03-29T04:15:50.243 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-29T04:15:50.450 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package indent. 2024-03-29T04:15:50.479 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-29T04:15:50.509 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-29T04:15:50.738 INFO:teuthology.orchestra.run.smithi151.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174061 files and directories currently installed.) 2024-03-29T04:15:50.743 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-29T04:15:50.791 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-29T04:15:50.819 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-29T04:15:50.857 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-29T04:15:50.861 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-29T04:15:51.191 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-29T04:15:51.221 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-29T04:15:51.257 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-29T04:15:51.274 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-29T04:15:51.405 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-29T04:15:51.502 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-29T04:15:51.518 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-29T04:15:51.551 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-29T04:15:51.723 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.3) ... 2024-03-29T04:15:51.905 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-29T04:15:51.934 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-29T04:15:51.972 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-29T04:15:52.080 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package bison. 2024-03-29T04:15:52.181 INFO:teuthology.orchestra.run.smithi151.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174061 files and directories currently installed.) 2024-03-29T04:15:52.186 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-29T04:15:52.199 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-29T04:15:52.216 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-29T04:15:52.249 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-29T04:15:52.265 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-29T04:15:52.494 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libjson-perl. 2024-03-29T04:15:52.522 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-29T04:15:52.561 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-29T04:15:52.658 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-29T04:15:52.686 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-29T04:15:52.724 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-29T04:15:52.880 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-29T04:15:52.909 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-29T04:15:52.947 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-29T04:15:53.044 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package dump. 2024-03-29T04:15:53.071 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-29T04:15:53.110 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking dump (0.4b47-3) ... 2024-03-29T04:15:53.397 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package indent. 2024-03-29T04:15:53.425 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-29T04:15:53.464 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-29T04:15:53.767 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-29T04:15:53.794 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-29T04:15:53.833 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-29T04:15:54.162 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-29T04:15:54.189 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-29T04:15:54.228 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-29T04:15:54.497 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-29T04:15:54.514 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-29T04:15:54.556 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-29T04:15:54.700 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-29T04:15:54.728 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-29T04:15:54.766 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-29T04:15:54.859 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-29T04:15:54.875 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-29T04:15:54.909 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-29T04:15:55.036 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-29T04:15:55.053 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-29T04:15:55.094 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-29T04:15:55.145 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-29T04:15:55.161 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-29T04:15:55.203 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-29T04:15:55.389 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-29T04:15:55.417 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-29T04:15:55.422 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libjson-perl. 2024-03-29T04:15:55.439 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-29T04:15:55.447 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-29T04:15:55.472 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-29T04:15:55.792 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-29T04:15:55.810 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-29T04:15:55.820 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-29T04:15:55.837 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-29T04:15:55.858 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-29T04:15:55.884 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-29T04:15:56.187 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-29T04:15:56.215 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-29T04:15:56.253 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-29T04:15:56.565 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-29T04:15:56.593 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-29T04:15:56.623 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T04:15:56.968 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-29T04:15:56.996 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-29T04:15:57.034 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-29T04:15:57.388 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-29T04:15:57.404 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-29T04:15:57.446 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-29T04:15:57.645 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-29T04:15:57.665 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-29T04:15:57.673 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-29T04:15:57.682 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-29T04:15:57.711 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-29T04:15:57.714 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-29T04:15:57.964 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-29T04:15:57.989 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-29T04:15:58.056 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-29T04:15:58.227 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-29T04:15:58.255 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-29T04:15:58.294 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-29T04:15:58.358 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-29T04:15:58.386 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-29T04:15:58.425 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-29T04:15:58.672 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package modemmanager. 2024-03-29T04:15:58.700 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-29T04:15:58.738 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-29T04:15:58.778 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-29T04:15:58.805 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-29T04:15:58.844 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-29T04:15:59.172 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-29T04:15:59.200 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-29T04:15:59.238 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-29T04:15:59.470 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package network-manager. 2024-03-29T04:15:59.499 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-29T04:15:59.554 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-29T04:15:59.608 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-29T04:15:59.636 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-29T04:15:59.675 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T04:16:00.011 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-29T04:16:00.039 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-29T04:16:00.077 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-29T04:16:00.270 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package ppp. 2024-03-29T04:16:00.298 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-29T04:16:00.363 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-29T04:16:00.379 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-29T04:16:00.413 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-29T04:16:00.571 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-29T04:16:00.615 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-29T04:16:00.631 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-29T04:16:00.665 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-29T04:16:00.916 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package pptp-linux. 2024-03-29T04:16:00.975 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-29T04:16:00.983 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-29T04:16:01.169 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-29T04:16:01.197 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-29T04:16:01.228 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-29T04:16:01.236 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-29T04:16:01.257 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-29T04:16:01.294 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-29T04:16:01.589 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package modemmanager. 2024-03-29T04:16:01.597 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-29T04:16:01.617 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-29T04:16:01.626 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-29T04:16:01.655 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-29T04:16:01.665 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking postgresql-client-common (238) ... 2024-03-29T04:16:01.926 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-29T04:16:01.943 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-29T04:16:01.976 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T04:16:02.304 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package network-manager. 2024-03-29T04:16:02.332 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-29T04:16:02.379 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-29T04:16:02.606 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package postgresql-common. 2024-03-29T04:16:02.635 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-29T04:16:02.717 INFO:teuthology.orchestra.run.smithi022.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-29T04:16:02.746 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking postgresql-common (238) ... 2024-03-29T04:16:02.911 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ppp. 2024-03-29T04:16:02.939 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-29T04:16:03.242 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-29T04:16:03.253 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package postgresql-14. 2024-03-29T04:16:03.273 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-29T04:16:03.467 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T04:16:03.506 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package pptp-linux. 2024-03-29T04:16:03.523 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-29T04:16:03.557 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-29T04:16:03.784 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-29T04:16:03.812 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-29T04:16:03.851 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-29T04:16:04.154 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-29T04:16:04.182 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-29T04:16:04.220 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking postgresql-client-common (238) ... 2024-03-29T04:16:04.522 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-29T04:16:04.549 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-29T04:16:04.624 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T04:16:04.751 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package postgresql. 2024-03-29T04:16:04.768 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-29T04:16:04.801 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking postgresql (14+238) ... 2024-03-29T04:16:05.045 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package postgresql-client. 2024-03-29T04:16:05.062 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-29T04:16:05.103 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking postgresql-client (14+238) ... 2024-03-29T04:16:05.205 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package postgresql-common. 2024-03-29T04:16:05.234 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-29T04:16:05.291 INFO:teuthology.orchestra.run.smithi151.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-29T04:16:05.320 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking postgresql-common (238) ... 2024-03-29T04:16:05.347 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-29T04:16:05.364 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-29T04:16:05.405 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-29T04:16:05.725 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package tcl8.6. 2024-03-29T04:16:05.729 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package postgresql-14. 2024-03-29T04:16:05.753 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-29T04:16:05.758 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-29T04:16:05.791 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T04:16:05.791 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-29T04:16:06.044 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package tcl. 2024-03-29T04:16:06.073 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-29T04:16:06.116 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-29T04:16:06.489 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package wpasupplicant. 2024-03-29T04:16:06.518 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-29T04:16:06.555 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-29T04:16:06.952 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-29T04:16:06.981 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-29T04:16:07.018 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-29T04:16:07.066 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package postgresql. 2024-03-29T04:16:07.082 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-29T04:16:07.124 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking postgresql (14+238) ... 2024-03-29T04:16:07.405 INFO:teuthology.orchestra.run.smithi022.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-29T04:16:07.434 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-29T04:16:07.480 INFO:teuthology.orchestra.run.smithi022.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-29T04:16:07.539 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package postgresql-client. 2024-03-29T04:16:07.567 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-29T04:16:07.760 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up postgresql-client-common (238) ... 2024-03-29T04:16:07.801 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking postgresql-client (14+238) ... 2024-03-29T04:16:07.969 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-29T04:16:08.028 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-29T04:16:08.044 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-29T04:16:08.086 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-29T04:16:08.095 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-29T04:16:08.221 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T04:16:08.347 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-29T04:16:08.406 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package tcl8.6. 2024-03-29T04:16:08.434 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-29T04:16:08.472 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-29T04:16:08.473 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-29T04:16:08.574 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T04:16:08.758 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package tcl. 2024-03-29T04:16:08.786 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-29T04:16:08.829 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-29T04:16:09.144 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package wpasupplicant. 2024-03-29T04:16:09.155 INFO:teuthology.orchestra.run.smithi022.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-29T04:16:09.172 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-29T04:16:09.210 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-29T04:16:09.255 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-29T04:16:09.573 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-29T04:16:09.589 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-29T04:16:09.631 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-29T04:16:09.635 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-29T04:16:09.635 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-29T04:16:10.010 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-29T04:16:10.038 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-29T04:16:10.084 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-29T04:16:10.100 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.3) ... 2024-03-29T04:16:10.201 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-29T04:16:10.327 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-29T04:16:10.372 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up postgresql-client-common (238) ... 2024-03-29T04:16:10.453 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-29T04:16:10.579 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-29T04:16:10.590 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-29T04:16:10.705 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-29T04:16:10.716 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-29T04:16:10.831 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-29T04:16:10.842 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T04:16:10.957 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-29T04:16:10.968 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-29T04:16:11.095 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-29T04:16:11.212 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T04:16:11.554 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-29T04:16:11.680 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up dump (0.4b47-3) ... 2024-03-29T04:16:11.682 INFO:teuthology.orchestra.run.smithi151.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-29T04:16:11.774 INFO:teuthology.orchestra.run.smithi022.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-29T04:16:11.776 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-29T04:16:11.832 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-29T04:16:11.941 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-29T04:16:12.042 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-29T04:16:12.132 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-29T04:16:12.132 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-29T04:16:12.143 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-29T04:16:12.271 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-29T04:16:12.362 INFO:teuthology.orchestra.run.smithi022.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-29T04:16:12.431 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-29T04:16:12.557 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-29T04:16:12.638 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.3) ... 2024-03-29T04:16:12.733 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-29T04:16:12.834 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-29T04:16:12.935 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-29T04:16:13.089 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-29T04:16:13.103 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up postgresql-client (14+238) ... 2024-03-29T04:16:13.215 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-29T04:16:13.238 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-29T04:16:13.341 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-29T04:16:13.467 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-29T04:16:13.593 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-29T04:16:13.744 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-29T04:16:13.853 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-29T04:16:14.540 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-29T04:16:14.563 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:14.563 INFO:teuthology.orchestra.run.smithi022.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-29T04:16:14.563 INFO:teuthology.orchestra.run.smithi022.stdout:which means they are currently configured by ifupdown: 2024-03-29T04:16:14.563 INFO:teuthology.orchestra.run.smithi022.stdout:- ens1f0 2024-03-29T04:16:14.564 INFO:teuthology.orchestra.run.smithi022.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-29T04:16:14.564 INFO:teuthology.orchestra.run.smithi022.stdout:remove their configuration from /etc/network/interfaces. 2024-03-29T04:16:14.564 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:14.694 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-29T04:16:14.718 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up dump (0.4b47-3) ... 2024-03-29T04:16:14.813 INFO:teuthology.orchestra.run.smithi151.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-29T04:16:14.869 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-29T04:16:14.970 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-29T04:16:15.021 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-29T04:16:15.088 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-29T04:16:15.214 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-29T04:16:15.340 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-29T04:16:15.356 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-29T04:16:15.446 INFO:teuthology.orchestra.run.smithi151.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-29T04:16:15.516 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-29T04:16:15.642 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-29T04:16:15.819 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-29T04:16:15.890 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-29T04:16:15.945 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-29T04:16:16.017 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-29T04:16:16.046 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-29T04:16:16.118 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-29T04:16:16.214 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up postgresql-client (14+238) ... 2024-03-29T04:16:16.227 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up postgresql-common (238) ... 2024-03-29T04:16:16.340 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-29T04:16:16.844 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:16.844 INFO:teuthology.orchestra.run.smithi151.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-29T04:16:16.844 INFO:teuthology.orchestra.run.smithi151.stdout:which means they are currently configured by ifupdown: 2024-03-29T04:16:16.844 INFO:teuthology.orchestra.run.smithi151.stdout:- enp3s0f1 2024-03-29T04:16:16.844 INFO:teuthology.orchestra.run.smithi151.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-29T04:16:16.844 INFO:teuthology.orchestra.run.smithi151.stdout:remove their configuration from /etc/network/interfaces. 2024-03-29T04:16:16.844 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:16.973 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-29T04:16:17.141 INFO:teuthology.orchestra.run.smithi022.stdout:Adding user postgres to group ssl-cert 2024-03-29T04:16:17.289 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-29T04:16:17.405 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:17.405 INFO:teuthology.orchestra.run.smithi022.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-29T04:16:17.610 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-29T04:16:17.613 INFO:teuthology.orchestra.run.smithi022.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-29T04:16:17.614 INFO:teuthology.orchestra.run.smithi022.stdout:Removing obsolete dictionary files: 2024-03-29T04:16:18.092 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-29T04:16:18.155 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-29T04:16:18.245 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-29T04:16:18.370 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-29T04:16:18.479 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up postgresql-common (238) ... 2024-03-29T04:16:18.545 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T04:16:18.878 INFO:teuthology.orchestra.run.smithi022.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-29T04:16:18.898 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-29T04:16:18.953 INFO:teuthology.orchestra.run.smithi022.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-29T04:16:18.953 INFO:teuthology.orchestra.run.smithi022.stdout:This user must also own the server process. 2024-03-29T04:16:18.953 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:18.953 INFO:teuthology.orchestra.run.smithi022.stdout:The database cluster will be initialized with locale "en_US". 2024-03-29T04:16:18.953 INFO:teuthology.orchestra.run.smithi022.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-29T04:16:18.953 INFO:teuthology.orchestra.run.smithi022.stdout:The default text search configuration will be set to "english". 2024-03-29T04:16:18.953 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:18.953 INFO:teuthology.orchestra.run.smithi022.stdout:Data page checksums are disabled. 2024-03-29T04:16:18.953 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:18.954 INFO:teuthology.orchestra.run.smithi022.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-29T04:16:18.954 INFO:teuthology.orchestra.run.smithi022.stdout:creating subdirectories ... ok 2024-03-29T04:16:18.954 INFO:teuthology.orchestra.run.smithi022.stdout:selecting dynamic shared memory implementation ... posix 2024-03-29T04:16:18.969 INFO:teuthology.orchestra.run.smithi022.stdout:selecting default max_connections ... 100 2024-03-29T04:16:18.990 INFO:teuthology.orchestra.run.smithi022.stdout:selecting default shared_buffers ... 128MB 2024-03-29T04:16:19.001 INFO:teuthology.orchestra.run.smithi022.stdout:selecting default time zone ... Etc/UTC 2024-03-29T04:16:19.001 INFO:teuthology.orchestra.run.smithi022.stdout:creating configuration files ... ok 2024-03-29T04:16:19.120 INFO:teuthology.orchestra.run.smithi022.stdout:running bootstrap script ... ok 2024-03-29T04:16:19.372 INFO:teuthology.orchestra.run.smithi151.stdout:Adding user postgres to group ssl-cert 2024-03-29T04:16:19.612 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:19.612 INFO:teuthology.orchestra.run.smithi151.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-29T04:16:19.777 INFO:teuthology.orchestra.run.smithi022.stdout:performing post-bootstrap initialization ... ok 2024-03-29T04:16:19.834 INFO:teuthology.orchestra.run.smithi151.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-29T04:16:19.834 INFO:teuthology.orchestra.run.smithi151.stdout:Removing obsolete dictionary files: 2024-03-29T04:16:20.370 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-29T04:16:20.632 INFO:teuthology.orchestra.run.smithi022.stdout:syncing data to disk ... ok 2024-03-29T04:16:20.730 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T04:16:21.051 INFO:teuthology.orchestra.run.smithi151.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-29T04:16:21.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-29T04:16:21.074 INFO:teuthology.orchestra.run.smithi022.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-29T04:16:21.123 INFO:teuthology.orchestra.run.smithi151.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-29T04:16:21.123 INFO:teuthology.orchestra.run.smithi151.stdout:This user must also own the server process. 2024-03-29T04:16:21.123 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:21.123 INFO:teuthology.orchestra.run.smithi151.stdout:The database cluster will be initialized with locale "en_US". 2024-03-29T04:16:21.123 INFO:teuthology.orchestra.run.smithi151.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-29T04:16:21.123 INFO:teuthology.orchestra.run.smithi151.stdout:The default text search configuration will be set to "english". 2024-03-29T04:16:21.123 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:21.123 INFO:teuthology.orchestra.run.smithi151.stdout:Data page checksums are disabled. 2024-03-29T04:16:21.124 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:21.124 INFO:teuthology.orchestra.run.smithi151.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-29T04:16:21.124 INFO:teuthology.orchestra.run.smithi151.stdout:creating subdirectories ... ok 2024-03-29T04:16:21.124 INFO:teuthology.orchestra.run.smithi151.stdout:selecting dynamic shared memory implementation ... posix 2024-03-29T04:16:21.138 INFO:teuthology.orchestra.run.smithi151.stdout:selecting default max_connections ... 100 2024-03-29T04:16:21.159 INFO:teuthology.orchestra.run.smithi151.stdout:selecting default shared_buffers ... 128MB 2024-03-29T04:16:21.170 INFO:teuthology.orchestra.run.smithi151.stdout:selecting default time zone ... Etc/UTC 2024-03-29T04:16:21.172 INFO:teuthology.orchestra.run.smithi151.stdout:creating configuration files ... ok 2024-03-29T04:16:21.290 INFO:teuthology.orchestra.run.smithi151.stdout:running bootstrap script ... ok 2024-03-29T04:16:22.001 INFO:teuthology.orchestra.run.smithi151.stdout:performing post-bootstrap initialization ... ok 2024-03-29T04:16:22.709 INFO:teuthology.orchestra.run.smithi151.stdout:syncing data to disk ... ok 2024-03-29T04:16:23.093 INFO:teuthology.orchestra.run.smithi151.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-29T04:16:23.910 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-29T04:16:24.036 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-29T04:16:24.171 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up postgresql (14+238) ... 2024-03-29T04:16:24.305 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-29T04:16:24.456 INFO:teuthology.orchestra.run.smithi022.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-29T04:16:24.738 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-29T04:16:24.739 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-29T04:16:25.258 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-29T04:16:25.393 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-29T04:16:25.686 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-29T04:16:25.822 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-29T04:16:25.836 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-29T04:16:25.954 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-29T04:16:26.080 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up postgresql (14+238) ... 2024-03-29T04:16:26.206 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-29T04:16:26.332 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-29T04:16:26.589 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-29T04:16:26.590 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-29T04:16:27.107 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-29T04:16:27.118 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-29T04:16:27.207 INFO:teuthology.orchestra.run.smithi022.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-29T04:16:27.210 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-29T04:16:27.522 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-29T04:16:27.673 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-29T04:16:27.724 INFO:teuthology.orchestra.run.smithi022.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-29T04:16:27.730 DEBUG:teuthology.parallel:result is None 2024-03-29T04:16:29.026 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-29T04:16:29.126 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-29T04:16:29.608 INFO:teuthology.orchestra.run.smithi151.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-29T04:16:29.612 DEBUG:teuthology.parallel:result is None 2024-03-29T04:16:29.613 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=69e2cd73c52284dfd5096c66b60c6e6b26a3c447 2024-03-29T04:16:29.835 DEBUG:teuthology.orchestra.run.smithi022:> dpkg-query -W -f '${Version}' ceph 2024-03-29T04:16:29.855 INFO:teuthology.orchestra.run.smithi022.stdout:19.0.0-2520-g69e2cd73-1jammy 2024-03-29T04:16:29.856 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2520-g69e2cd73-1jammy 2024-03-29T04:16:29.856 INFO:teuthology.task.install:The correct ceph version 19.0.0-2520-g69e2cd73-1jammy is installed. 2024-03-29T04:16:29.858 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=69e2cd73c52284dfd5096c66b60c6e6b26a3c447 2024-03-29T04:16:30.075 DEBUG:teuthology.orchestra.run.smithi151:> dpkg-query -W -f '${Version}' ceph 2024-03-29T04:16:30.094 INFO:teuthology.orchestra.run.smithi151.stdout:19.0.0-2520-g69e2cd73-1jammy 2024-03-29T04:16:30.094 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2520-g69e2cd73-1jammy 2024-03-29T04:16:30.095 INFO:teuthology.task.install:The correct ceph version 19.0.0-2520-g69e2cd73-1jammy is installed. 2024-03-29T04:16:30.096 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-29T04:16:30.097 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:16:30.097 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-29T04:16:30.112 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:16:30.112 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-29T04:16:30.156 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-29T04:16:30.157 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:16:30.157 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/daemon-helper 2024-03-29T04:16:30.170 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-29T04:16:30.224 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:16:30.224 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/daemon-helper 2024-03-29T04:16:30.238 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-29T04:16:30.294 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-29T04:16:30.296 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:16:30.296 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-29T04:16:30.309 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-29T04:16:30.364 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:16:30.364 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-29T04:16:30.376 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-29T04:16:30.433 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-29T04:16:30.434 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:16:30.434 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/stdin-killer 2024-03-29T04:16:30.448 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-29T04:16:30.506 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:16:30.506 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/stdin-killer 2024-03-29T04:16:30.519 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-29T04:16:30.574 INFO:teuthology.run_tasks:Running task ceph... 2024-03-29T04:16:30.660 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-29T04:16:30.660 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 777 /var/log/ceph 2024-03-29T04:16:30.663 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 777 /var/log/ceph 2024-03-29T04:16:30.675 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-29T04:16:30.675 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-29T04:16:30.718 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-29T04:16:30.729 INFO:tasks.ceph:Creating extra log directories... 2024-03-29T04:16:30.730 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-29T04:16:30.774 DEBUG:teuthology.orchestra.run.smithi151:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-29T04:16:30.793 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-29T04:16:30.793 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-29T04:16:30.794 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-29T04:16:30.794 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:16:30.794 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-29T04:16:30.833 DEBUG:teuthology.orchestra.run.smithi022:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-29T04:16:30.909 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:16:30.909 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-29T04:16:30.960 DEBUG:teuthology.orchestra.run.smithi151:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-29T04:16:31.039 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-29T04:16:31.039 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-29T04:16:31.040 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629086', 'branch': 'wip-mchangir-testing1-main-20240328-072412', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/uninlining}', 'email': 'mchangir@redhat.com', 'first_in_suite': False, 'job_id': '7629086', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-mchangir-testing1-main-20240328-072412'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'}}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mchangir-testing1-main-20240328-072412', 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'}}, 'owner': 'scheduled_mchangir@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8207, 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-mchangir-testing1-main-20240328-072412', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi022.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKCXo6Fu6f+uCk70spbH9SnoOQpq10bOkMvJsyVbHpskB9nuO/VnU7ECSDvwyeyjtUXCiDdkBiSlBW101K0GTAo=', 'smithi151.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHDWNwZHqvmHe//FSX8MiigxaJK++ZfMmopww64yeuTn8TzDtScnW51x2HSUHK62BijHfDGyhbfDRhgiE8FiUoo='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_uninlining']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-29_02:01:30', 'tube': 'smithi', 'user': 'mchangir', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-29T04:16:31.040 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-29T04:16:31.043 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-29T04:16:31.083 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m0777 -- /var/run/ceph 2024-03-29T04:16:31.089 DEBUG:teuthology.orchestra.run.smithi151:> sudo install -d -m0777 -- /var/run/ceph 2024-03-29T04:16:31.139 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:16:31.139 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/scratch_devs of=/dev/stdout 2024-03-29T04:16:31.149 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-29T04:16:31.150 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_1 2024-03-29T04:16:31.197 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-29T04:16:31.197 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:16:31.197 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 874 Links: 1 2024-03-29T04:16:31.197 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:16:31.198 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-03-29 04:10:20.782852084 +0000 2024-03-29T04:16:31.198 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-03-29 04:10:20.530854334 +0000 2024-03-29T04:16:31.198 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-03-29 04:10:20.530854334 +0000 2024-03-29T04:16:31.198 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2024-03-29T04:16:31.198 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-29T04:16:31.251 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-03-29T04:16:31.252 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-03-29T04:16:31.252 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000313164 s, 1.6 MB/s 2024-03-29T04:16:31.253 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-29T04:16:31.302 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_2 2024-03-29T04:16:31.349 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-29T04:16:31.349 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:16:31.349 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 892 Links: 1 2024-03-29T04:16:31.349 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:16:31.349 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-03-29 04:10:21.246847940 +0000 2024-03-29T04:16:31.349 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-03-29 04:10:20.982850298 +0000 2024-03-29T04:16:31.349 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-03-29 04:10:20.982850298 +0000 2024-03-29T04:16:31.349 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2024-03-29T04:16:31.349 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-29T04:16:31.403 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-03-29T04:16:31.404 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-03-29T04:16:31.404 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000309388 s, 1.7 MB/s 2024-03-29T04:16:31.405 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-29T04:16:31.454 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_3 2024-03-29T04:16:31.501 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-29T04:16:31.501 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:16:31.501 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 904 Links: 1 2024-03-29T04:16:31.502 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:16:31.502 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-03-29 04:10:21.738843544 +0000 2024-03-29T04:16:31.502 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-03-29 04:10:21.466845975 +0000 2024-03-29T04:16:31.502 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-03-29 04:10:21.466845975 +0000 2024-03-29T04:16:31.502 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2024-03-29T04:16:31.502 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-29T04:16:31.556 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-03-29T04:16:31.556 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-03-29T04:16:31.556 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000201119 s, 2.5 MB/s 2024-03-29T04:16:31.557 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-29T04:16:31.606 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_4 2024-03-29T04:16:31.653 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-29T04:16:31.653 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:16:31.653 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 926 Links: 1 2024-03-29T04:16:31.653 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:16:31.653 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-03-29 04:10:22.226839187 +0000 2024-03-29T04:16:31.653 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-03-29 04:10:21.982841366 +0000 2024-03-29T04:16:31.653 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-03-29 04:10:21.982841366 +0000 2024-03-29T04:16:31.653 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2024-03-29T04:16:31.654 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-29T04:16:31.707 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-03-29T04:16:31.708 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-03-29T04:16:31.708 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000311159 s, 1.6 MB/s 2024-03-29T04:16:31.709 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-29T04:16:31.758 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T04:16:31.758 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:16:31.758 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/scratch_devs of=/dev/stdout 2024-03-29T04:16:31.764 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-29T04:16:31.764 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_1 2024-03-29T04:16:31.812 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-29T04:16:31.855 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:16:31.855 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 883 Links: 1 2024-03-29T04:16:31.855 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:16:31.855 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-03-29 04:10:18.976065693 +0000 2024-03-29T04:16:31.856 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-03-29 04:10:18.708068083 +0000 2024-03-29T04:16:31.856 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-03-29 04:10:18.708068083 +0000 2024-03-29T04:16:31.856 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: - 2024-03-29T04:16:31.856 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-29T04:16:31.868 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-03-29T04:16:31.869 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-03-29T04:16:31.869 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000415622 s, 1.2 MB/s 2024-03-29T04:16:31.870 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-29T04:16:31.917 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_2 2024-03-29T04:16:31.963 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-29T04:16:31.963 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:16:31.964 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 897 Links: 1 2024-03-29T04:16:31.964 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:16:31.964 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-03-29 04:10:19.460061375 +0000 2024-03-29T04:16:31.964 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-03-29 04:10:19.208063622 +0000 2024-03-29T04:16:31.964 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-03-29 04:10:19.208063622 +0000 2024-03-29T04:16:31.964 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: - 2024-03-29T04:16:31.964 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-29T04:16:32.018 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-03-29T04:16:32.018 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-03-29T04:16:32.018 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000444042 s, 1.2 MB/s 2024-03-29T04:16:32.019 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-29T04:16:32.068 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_3 2024-03-29T04:16:32.115 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-29T04:16:32.115 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:16:32.115 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 918 Links: 1 2024-03-29T04:16:32.115 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:16:32.115 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-03-29 04:10:19.960056915 +0000 2024-03-29T04:16:32.115 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-03-29 04:10:19.720059056 +0000 2024-03-29T04:16:32.115 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-03-29 04:10:19.720059056 +0000 2024-03-29T04:16:32.115 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: - 2024-03-29T04:16:32.116 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-29T04:16:32.170 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-03-29T04:16:32.170 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-03-29T04:16:32.170 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000405607 s, 1.3 MB/s 2024-03-29T04:16:32.171 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-29T04:16:32.221 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_4 2024-03-29T04:16:32.268 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-29T04:16:32.268 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:16:32.268 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 932 Links: 1 2024-03-29T04:16:32.268 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:16:32.268 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-03-29 04:10:20.460052454 +0000 2024-03-29T04:16:32.268 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-03-29 04:10:20.208054702 +0000 2024-03-29T04:16:32.268 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-03-29 04:10:20.208054702 +0000 2024-03-29T04:16:32.268 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: - 2024-03-29T04:16:32.268 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-29T04:16:32.321 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-03-29T04:16:32.321 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-03-29T04:16:32.321 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000279872 s, 1.8 MB/s 2024-03-29T04:16:32.322 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-29T04:16:32.369 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T04:16:32.369 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi022.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi151.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-29T04:16:32.369 INFO:tasks.ceph:Generating config... 2024-03-29T04:16:32.370 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-29T04:16:32.370 INFO:tasks.ceph:[client] debug client = 20 2024-03-29T04:16:32.370 INFO:tasks.ceph:[client] debug ms = 1 2024-03-29T04:16:32.370 INFO:tasks.ceph:[client] pid file = 2024-03-29T04:16:32.370 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-29T04:16:32.371 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-29T04:16:32.371 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-29T04:16:32.371 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-29T04:16:32.371 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-29T04:16:32.371 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-29T04:16:32.371 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-29T04:16:32.371 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-29T04:16:32.371 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-29T04:16:32.371 INFO:tasks.ceph:[mds] mds_bal_replicate_threshold = 1 2024-03-29T04:16:32.371 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-29T04:16:32.371 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-29T04:16:32.371 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-29T04:16:32.371 INFO:tasks.ceph:[mgr] debug client = 20 2024-03-29T04:16:32.371 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-03-29T04:16:32.372 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-29T04:16:32.372 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-29T04:16:32.372 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-29T04:16:32.372 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-29T04:16:32.372 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-29T04:16:32.372 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-03-29T04:16:32.372 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-29T04:16:32.372 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-29T04:16:32.372 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-29T04:16:32.372 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-03-29T04:16:32.372 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-03-29T04:16:32.372 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-29T04:16:32.372 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-29T04:16:32.372 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-03-29T04:16:32.373 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-29T04:16:32.373 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-29T04:16:32.373 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-29T04:16:32.373 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-29T04:16:32.373 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-29T04:16:32.373 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-29T04:16:32.373 INFO:tasks.ceph:Setting up mon.a... 2024-03-29T04:16:32.373 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-29T04:16:32.408 INFO:teuthology.orchestra.run.smithi022.stdout:creating /etc/ceph/ceph.keyring 2024-03-29T04:16:32.422 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-29T04:16:32.500 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-29T04:16:32.555 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.22'), ('mon.b', '172.21.15.151'), ('mon.c', '[v2:172.21.15.151:3301,v1:172.21.15.151:6790]')] 2024-03-29T04:16:32.556 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.22,172.21.15.151,[v2:172.21.15.151:3301,v1:172.21.15.151:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-29T04:16:32.557 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:16:32.557 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-29T04:16:32.600 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.22 --add b 172.21.15.151 --addv c '[v2:172.21.15.151:3301,v1:172.21.15.151:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:16:32.687 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:16:32.687 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool: generated fsid 9f2f9779-fe00-4763-9aa1-fd3768e2f135 2024-03-29T04:16:32.687 INFO:teuthology.orchestra.run.smithi022.stdout:setting min_mon_release = quincy 2024-03-29T04:16:32.687 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 0 2024-03-29T04:16:32.687 INFO:teuthology.orchestra.run.smithi022.stdout:fsid 9f2f9779-fe00-4763-9aa1-fd3768e2f135 2024-03-29T04:16:32.687 INFO:teuthology.orchestra.run.smithi022.stdout:last_changed 2024-03-29T04:16:32.709408+0000 2024-03-29T04:16:32.687 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-03-29T04:16:32.709408+0000 2024-03-29T04:16:32.687 INFO:teuthology.orchestra.run.smithi022.stdout:min_mon_release 17 (quincy) 2024-03-29T04:16:32.687 INFO:teuthology.orchestra.run.smithi022.stdout:election_strategy: 1 2024-03-29T04:16:32.687 INFO:teuthology.orchestra.run.smithi022.stdout:0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-03-29T04:16:32.687 INFO:teuthology.orchestra.run.smithi022.stdout:1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.b 2024-03-29T04:16:32.688 INFO:teuthology.orchestra.run.smithi022.stdout:2: [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] mon.c 2024-03-29T04:16:32.688 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-29T04:16:32.689 DEBUG:teuthology.orchestra.run.smithi022:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-29T04:16:32.737 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9f2f9779-fe00-4763-9aa1-fd3768e2f135... 2024-03-29T04:16:32.738 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-29T04:16:32.782 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-29T04:16:32.811 INFO:teuthology.orchestra.run.smithi022.stdout:[global] 2024-03-29T04:16:32.811 INFO:teuthology.orchestra.run.smithi022.stdout: chdir = "" 2024-03-29T04:16:32.811 INFO:teuthology.orchestra.run.smithi022.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-29T04:16:32.811 INFO:teuthology.orchestra.run.smithi022.stdout: auth supported = cephx 2024-03-29T04:16:32.811 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.811 INFO:teuthology.orchestra.run.smithi022.stdout: filestore xattr use omap = true 2024-03-29T04:16:32.811 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.811 INFO:teuthology.orchestra.run.smithi022.stdout: mon clock drift allowed = 1.000 2024-03-29T04:16:32.811 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.812 INFO:teuthology.orchestra.run.smithi022.stdout: osd crush chooseleaf type = 0 2024-03-29T04:16:32.812 INFO:teuthology.orchestra.run.smithi022.stdout: auth debug = true 2024-03-29T04:16:32.812 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.812 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on old message = true 2024-03-29T04:16:32.812 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on bug = true 2024-03-29T04:16:32.812 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.812 INFO:teuthology.orchestra.run.smithi022.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-29T04:16:32.812 INFO:teuthology.orchestra.run.smithi022.stdout: mon pg warn max object skew = 0 2024-03-29T04:16:32.812 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.812 INFO:teuthology.orchestra.run.smithi022.stdout: # disable pg_autoscaler by default for new pools 2024-03-29T04:16:32.812 INFO:teuthology.orchestra.run.smithi022.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-29T04:16:32.812 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.812 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default size = 2 2024-03-29T04:16:32.812 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.812 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow primary affinity = true 2024-03-29T04:16:32.812 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow pg remap = true 2024-03-29T04:16:32.813 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on legacy crush tunables = false 2024-03-29T04:16:32.813 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on crush straw calc version zero = false 2024-03-29T04:16:32.813 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on no sortbitwise = false 2024-03-29T04:16:32.813 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on osd down out interval zero = false 2024-03-29T04:16:32.813 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on too few osds = false 2024-03-29T04:16:32.813 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-29T04:16:32.813 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-29T04:16:32.813 INFO:teuthology.orchestra.run.smithi022.stdout: mon_allow_pool_size_one = true 2024-03-29T04:16:32.813 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.813 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-29T04:16:32.813 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.813 INFO:teuthology.orchestra.run.smithi022.stdout: osd default data pool replay window = 5 2024-03-29T04:16:32.813 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.814 INFO:teuthology.orchestra.run.smithi022.stdout: mon allow pool delete = true 2024-03-29T04:16:32.814 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.814 INFO:teuthology.orchestra.run.smithi022.stdout: mon cluster log file level = debug 2024-03-29T04:16:32.814 INFO:teuthology.orchestra.run.smithi022.stdout: debug asserts on shutdown = true 2024-03-29T04:16:32.814 INFO:teuthology.orchestra.run.smithi022.stdout: mon health detail to clog = false 2024-03-29T04:16:32.814 INFO:teuthology.orchestra.run.smithi022.stdout: mon host = "172.21.15.22,172.21.15.151,[v2:172.21.15.151:3301,v1:172.21.15.151:6790]" 2024-03-29T04:16:32.814 INFO:teuthology.orchestra.run.smithi022.stdout: fsid = 9f2f9779-fe00-4763-9aa1-fd3768e2f135 2024-03-29T04:16:32.814 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.814 INFO:teuthology.orchestra.run.smithi022.stdout:[osd] 2024-03-29T04:16:32.814 INFO:teuthology.orchestra.run.smithi022.stdout: osd journal size = 100 2024-03-29T04:16:32.814 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.814 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub load threshold = 5.0 2024-03-29T04:16:32.814 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub max interval = 600 2024-03-29T04:16:32.815 INFO:teuthology.orchestra.run.smithi022.stdout: osd mclock profile = high_recovery_ops 2024-03-29T04:16:32.815 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.815 INFO:teuthology.orchestra.run.smithi022.stdout: osd recover clone overlap = true 2024-03-29T04:16:32.815 INFO:teuthology.orchestra.run.smithi022.stdout: osd recovery max chunk = 1048576 2024-03-29T04:16:32.815 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.815 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug shutdown = true 2024-03-29T04:16:32.815 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug op order = true 2024-03-29T04:16:32.815 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug verify stray on activate = true 2024-03-29T04:16:32.815 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug trim objects = true 2024-03-29T04:16:32.815 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.815 INFO:teuthology.orchestra.run.smithi022.stdout: osd open classes on start = true 2024-03-29T04:16:32.815 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug pg log writeout = true 2024-03-29T04:16:32.815 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.815 INFO:teuthology.orchestra.run.smithi022.stdout: osd deep scrub update digest min age = 30 2024-03-29T04:16:32.815 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.816 INFO:teuthology.orchestra.run.smithi022.stdout: osd map max advance = 10 2024-03-29T04:16:32.816 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.816 INFO:teuthology.orchestra.run.smithi022.stdout: journal zero on create = true 2024-03-29T04:16:32.816 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.816 INFO:teuthology.orchestra.run.smithi022.stdout: filestore ondisk finisher threads = 3 2024-03-29T04:16:32.816 INFO:teuthology.orchestra.run.smithi022.stdout: filestore apply finisher threads = 3 2024-03-29T04:16:32.816 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.816 INFO:teuthology.orchestra.run.smithi022.stdout: bdev debug aio = true 2024-03-29T04:16:32.816 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug misdirected ops = true 2024-03-29T04:16:32.816 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore block size = 96636764160 2024-03-29T04:16:32.816 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore fsck on mount = True 2024-03-29T04:16:32.816 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluefs = 20 2024-03-29T04:16:32.816 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluestore = 20 2024-03-29T04:16:32.816 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-03-29T04:16:32.817 INFO:teuthology.orchestra.run.smithi022.stdout: debug osd = 20 2024-03-29T04:16:32.817 INFO:teuthology.orchestra.run.smithi022.stdout: debug rocksdb = 10 2024-03-29T04:16:32.817 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd backfillfull_ratio = 0.85 2024-03-29T04:16:32.817 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd full ratio = 0.9 2024-03-29T04:16:32.817 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd nearfull ratio = 0.8 2024-03-29T04:16:32.817 INFO:teuthology.orchestra.run.smithi022.stdout: osd failsafe full ratio = 0.95 2024-03-29T04:16:32.817 INFO:teuthology.orchestra.run.smithi022.stdout: osd objectstore = bluestore 2024-03-29T04:16:32.817 INFO:teuthology.orchestra.run.smithi022.stdout: osd op complaint time = 180 2024-03-29T04:16:32.817 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.817 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr] 2024-03-29T04:16:32.817 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-03-29T04:16:32.817 INFO:teuthology.orchestra.run.smithi022.stdout: debug mgr = 20 2024-03-29T04:16:32.817 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2024-03-29T04:16:32.817 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2024-03-29T04:16:32.817 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2024-03-29T04:16:32.818 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2024-03-29T04:16:32.818 INFO:teuthology.orchestra.run.smithi022.stdout: mgr/telemetry/nag = false 2024-03-29T04:16:32.818 INFO:teuthology.orchestra.run.smithi022.stdout: debug client = 20 2024-03-29T04:16:32.818 INFO:teuthology.orchestra.run.smithi022.stdout: debug finisher = 20 2024-03-29T04:16:32.818 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.818 INFO:teuthology.orchestra.run.smithi022.stdout:[mon] 2024-03-29T04:16:32.818 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-03-29T04:16:32.818 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2024-03-29T04:16:32.818 INFO:teuthology.orchestra.run.smithi022.stdout: debug paxos = 20 2024-03-29T04:16:32.818 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2024-03-29T04:16:32.818 INFO:teuthology.orchestra.run.smithi022.stdout: mon data avail warn = 5 2024-03-29T04:16:32.818 INFO:teuthology.orchestra.run.smithi022.stdout: mon mgr mkfs grace = 240 2024-03-29T04:16:32.818 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2024-03-29T04:16:32.818 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd reporter subtree level = osd 2024-03-29T04:16:32.819 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd prime pg temp = true 2024-03-29T04:16:32.819 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2024-03-29T04:16:32.819 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.819 INFO:teuthology.orchestra.run.smithi022.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-29T04:16:32.819 INFO:teuthology.orchestra.run.smithi022.stdout: auth mon ticket ttl = 660 # 11m 2024-03-29T04:16:32.819 INFO:teuthology.orchestra.run.smithi022.stdout: auth service ticket ttl = 240 # 4m 2024-03-29T04:16:32.819 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.819 INFO:teuthology.orchestra.run.smithi022.stdout: # don't complain about insecure global_id in the test suite 2024-03-29T04:16:32.819 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-29T04:16:32.819 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-29T04:16:32.819 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.819 INFO:teuthology.orchestra.run.smithi022.stdout: # 1m isn't quite enough 2024-03-29T04:16:32.819 INFO:teuthology.orchestra.run.smithi022.stdout: mon_down_mkfs_grace = 2m 2024-03-29T04:16:32.819 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.820 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_filestore_osds = false 2024-03-29T04:16:32.820 INFO:teuthology.orchestra.run.smithi022.stdout: mon down mkfs grace = 300 2024-03-29T04:16:32.820 INFO:teuthology.orchestra.run.smithi022.stdout: mon op complaint time = 120 2024-03-29T04:16:32.820 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:16:32.820 INFO:teuthology.orchestra.run.smithi022.stdout:[client] 2024-03-29T04:16:32.820 INFO:teuthology.orchestra.run.smithi022.stdout: rgw cache enabled = true 2024-03-29T04:16:32.820 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable ops log = true 2024-03-29T04:16:32.820 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable usage log = true 2024-03-29T04:16:32.820 INFO:teuthology.orchestra.run.smithi022.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-29T04:16:32.820 INFO:teuthology.orchestra.run.smithi022.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-29T04:16:32.820 INFO:teuthology.orchestra.run.smithi022.stdout: client mount timeout = 600 2024-03-29T04:16:32.820 INFO:teuthology.orchestra.run.smithi022.stdout: debug client = 20 2024-03-29T04:16:32.820 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-03-29T04:16:32.821 INFO:teuthology.orchestra.run.smithi022.stdout: pid file = "" 2024-03-29T04:16:32.821 INFO:teuthology.orchestra.run.smithi022.stdout: rados mon op timeout = 900 2024-03-29T04:16:32.821 INFO:teuthology.orchestra.run.smithi022.stdout: rados osd op timeout = 900 2024-03-29T04:16:32.821 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.a] 2024-03-29T04:16:32.821 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.b] 2024-03-29T04:16:32.821 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.c] 2024-03-29T04:16:32.821 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.a] 2024-03-29T04:16:32.821 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.c] 2024-03-29T04:16:32.821 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.b] 2024-03-29T04:16:32.821 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.d] 2024-03-29T04:16:32.821 INFO:teuthology.orchestra.run.smithi022.stdout:[mds] 2024-03-29T04:16:32.821 INFO:teuthology.orchestra.run.smithi022.stdout: debug mds = 20 2024-03-29T04:16:32.821 INFO:teuthology.orchestra.run.smithi022.stdout: debug mds balancer = 20 2024-03-29T04:16:32.821 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-03-29T04:16:32.822 INFO:teuthology.orchestra.run.smithi022.stdout: mds debug frag = True 2024-03-29T04:16:32.822 INFO:teuthology.orchestra.run.smithi022.stdout: mds debug scatterstat = True 2024-03-29T04:16:32.822 INFO:teuthology.orchestra.run.smithi022.stdout: mds op complaint time = 180 2024-03-29T04:16:32.822 INFO:teuthology.orchestra.run.smithi022.stdout: mds verify scatter = True 2024-03-29T04:16:32.822 INFO:teuthology.orchestra.run.smithi022.stdout: mds_bal_replicate_threshold = 1 2024-03-29T04:16:32.822 INFO:teuthology.orchestra.run.smithi022.stdout: osd op complaint time = 180 2024-03-29T04:16:32.822 INFO:teuthology.orchestra.run.smithi022.stdout: rados mon op timeout = 900 2024-03-29T04:16:32.822 INFO:teuthology.orchestra.run.smithi022.stdout: rados osd op timeout = 900 2024-03-29T04:16:32.823 INFO:teuthology.orchestra.run.smithi151.stdout:[global] 2024-03-29T04:16:32.823 INFO:teuthology.orchestra.run.smithi151.stdout: chdir = "" 2024-03-29T04:16:32.823 INFO:teuthology.orchestra.run.smithi151.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-29T04:16:32.823 INFO:teuthology.orchestra.run.smithi151.stdout: auth supported = cephx 2024-03-29T04:16:32.823 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.823 INFO:teuthology.orchestra.run.smithi151.stdout: filestore xattr use omap = true 2024-03-29T04:16:32.823 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.823 INFO:teuthology.orchestra.run.smithi151.stdout: mon clock drift allowed = 1.000 2024-03-29T04:16:32.823 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.823 INFO:teuthology.orchestra.run.smithi151.stdout: osd crush chooseleaf type = 0 2024-03-29T04:16:32.823 INFO:teuthology.orchestra.run.smithi151.stdout: auth debug = true 2024-03-29T04:16:32.823 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.823 INFO:teuthology.orchestra.run.smithi151.stdout: ms die on old message = true 2024-03-29T04:16:32.824 INFO:teuthology.orchestra.run.smithi151.stdout: ms die on bug = true 2024-03-29T04:16:32.824 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.824 INFO:teuthology.orchestra.run.smithi151.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-29T04:16:32.824 INFO:teuthology.orchestra.run.smithi151.stdout: mon pg warn max object skew = 0 2024-03-29T04:16:32.824 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.824 INFO:teuthology.orchestra.run.smithi151.stdout: # disable pg_autoscaler by default for new pools 2024-03-29T04:16:32.824 INFO:teuthology.orchestra.run.smithi151.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-29T04:16:32.824 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.824 INFO:teuthology.orchestra.run.smithi151.stdout: osd pool default size = 2 2024-03-29T04:16:32.824 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.824 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd allow primary affinity = true 2024-03-29T04:16:32.824 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd allow pg remap = true 2024-03-29T04:16:32.824 INFO:teuthology.orchestra.run.smithi151.stdout: mon warn on legacy crush tunables = false 2024-03-29T04:16:32.824 INFO:teuthology.orchestra.run.smithi151.stdout: mon warn on crush straw calc version zero = false 2024-03-29T04:16:32.824 INFO:teuthology.orchestra.run.smithi151.stdout: mon warn on no sortbitwise = false 2024-03-29T04:16:32.825 INFO:teuthology.orchestra.run.smithi151.stdout: mon warn on osd down out interval zero = false 2024-03-29T04:16:32.825 INFO:teuthology.orchestra.run.smithi151.stdout: mon warn on too few osds = false 2024-03-29T04:16:32.825 INFO:teuthology.orchestra.run.smithi151.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-29T04:16:32.825 INFO:teuthology.orchestra.run.smithi151.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-29T04:16:32.825 INFO:teuthology.orchestra.run.smithi151.stdout: mon_allow_pool_size_one = true 2024-03-29T04:16:32.825 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.825 INFO:teuthology.orchestra.run.smithi151.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-29T04:16:32.825 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.825 INFO:teuthology.orchestra.run.smithi151.stdout: osd default data pool replay window = 5 2024-03-29T04:16:32.825 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.825 INFO:teuthology.orchestra.run.smithi151.stdout: mon allow pool delete = true 2024-03-29T04:16:32.825 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.825 INFO:teuthology.orchestra.run.smithi151.stdout: mon cluster log file level = debug 2024-03-29T04:16:32.826 INFO:teuthology.orchestra.run.smithi151.stdout: debug asserts on shutdown = true 2024-03-29T04:16:32.826 INFO:teuthology.orchestra.run.smithi151.stdout: mon health detail to clog = false 2024-03-29T04:16:32.826 INFO:teuthology.orchestra.run.smithi151.stdout: mon host = "172.21.15.22,172.21.15.151,[v2:172.21.15.151:3301,v1:172.21.15.151:6790]" 2024-03-29T04:16:32.826 INFO:teuthology.orchestra.run.smithi151.stdout: fsid = 9f2f9779-fe00-4763-9aa1-fd3768e2f135 2024-03-29T04:16:32.826 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.826 INFO:teuthology.orchestra.run.smithi151.stdout:[osd] 2024-03-29T04:16:32.826 INFO:teuthology.orchestra.run.smithi151.stdout: osd journal size = 100 2024-03-29T04:16:32.826 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.826 INFO:teuthology.orchestra.run.smithi151.stdout: osd scrub load threshold = 5.0 2024-03-29T04:16:32.826 INFO:teuthology.orchestra.run.smithi151.stdout: osd scrub max interval = 600 2024-03-29T04:16:32.826 INFO:teuthology.orchestra.run.smithi151.stdout: osd mclock profile = high_recovery_ops 2024-03-29T04:16:32.826 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.826 INFO:teuthology.orchestra.run.smithi151.stdout: osd recover clone overlap = true 2024-03-29T04:16:32.826 INFO:teuthology.orchestra.run.smithi151.stdout: osd recovery max chunk = 1048576 2024-03-29T04:16:32.827 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.827 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug shutdown = true 2024-03-29T04:16:32.827 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug op order = true 2024-03-29T04:16:32.827 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug verify stray on activate = true 2024-03-29T04:16:32.827 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug trim objects = true 2024-03-29T04:16:32.827 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.827 INFO:teuthology.orchestra.run.smithi151.stdout: osd open classes on start = true 2024-03-29T04:16:32.827 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug pg log writeout = true 2024-03-29T04:16:32.827 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.827 INFO:teuthology.orchestra.run.smithi151.stdout: osd deep scrub update digest min age = 30 2024-03-29T04:16:32.827 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.827 INFO:teuthology.orchestra.run.smithi151.stdout: osd map max advance = 10 2024-03-29T04:16:32.827 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.828 INFO:teuthology.orchestra.run.smithi151.stdout: journal zero on create = true 2024-03-29T04:16:32.828 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.828 INFO:teuthology.orchestra.run.smithi151.stdout: filestore ondisk finisher threads = 3 2024-03-29T04:16:32.828 INFO:teuthology.orchestra.run.smithi151.stdout: filestore apply finisher threads = 3 2024-03-29T04:16:32.828 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.828 INFO:teuthology.orchestra.run.smithi151.stdout: bdev debug aio = true 2024-03-29T04:16:32.828 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug misdirected ops = true 2024-03-29T04:16:32.828 INFO:teuthology.orchestra.run.smithi151.stdout: bluestore block size = 96636764160 2024-03-29T04:16:32.828 INFO:teuthology.orchestra.run.smithi151.stdout: bluestore fsck on mount = True 2024-03-29T04:16:32.828 INFO:teuthology.orchestra.run.smithi151.stdout: debug bluefs = 20 2024-03-29T04:16:32.828 INFO:teuthology.orchestra.run.smithi151.stdout: debug bluestore = 20 2024-03-29T04:16:32.828 INFO:teuthology.orchestra.run.smithi151.stdout: debug ms = 1 2024-03-29T04:16:32.829 INFO:teuthology.orchestra.run.smithi151.stdout: debug osd = 20 2024-03-29T04:16:32.829 INFO:teuthology.orchestra.run.smithi151.stdout: debug rocksdb = 10 2024-03-29T04:16:32.829 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd backfillfull_ratio = 0.85 2024-03-29T04:16:32.829 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd full ratio = 0.9 2024-03-29T04:16:32.829 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd nearfull ratio = 0.8 2024-03-29T04:16:32.829 INFO:teuthology.orchestra.run.smithi151.stdout: osd failsafe full ratio = 0.95 2024-03-29T04:16:32.829 INFO:teuthology.orchestra.run.smithi151.stdout: osd objectstore = bluestore 2024-03-29T04:16:32.829 INFO:teuthology.orchestra.run.smithi151.stdout: osd op complaint time = 180 2024-03-29T04:16:32.829 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.829 INFO:teuthology.orchestra.run.smithi151.stdout:[mgr] 2024-03-29T04:16:32.829 INFO:teuthology.orchestra.run.smithi151.stdout: debug ms = 1 2024-03-29T04:16:32.829 INFO:teuthology.orchestra.run.smithi151.stdout: debug mgr = 20 2024-03-29T04:16:32.829 INFO:teuthology.orchestra.run.smithi151.stdout: debug mon = 20 2024-03-29T04:16:32.829 INFO:teuthology.orchestra.run.smithi151.stdout: debug auth = 20 2024-03-29T04:16:32.830 INFO:teuthology.orchestra.run.smithi151.stdout: mon reweight min pgs per osd = 4 2024-03-29T04:16:32.830 INFO:teuthology.orchestra.run.smithi151.stdout: mon reweight min bytes per osd = 10 2024-03-29T04:16:32.830 INFO:teuthology.orchestra.run.smithi151.stdout: mgr/telemetry/nag = false 2024-03-29T04:16:32.830 INFO:teuthology.orchestra.run.smithi151.stdout: debug client = 20 2024-03-29T04:16:32.830 INFO:teuthology.orchestra.run.smithi151.stdout: debug finisher = 20 2024-03-29T04:16:32.830 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.830 INFO:teuthology.orchestra.run.smithi151.stdout:[mon] 2024-03-29T04:16:32.830 INFO:teuthology.orchestra.run.smithi151.stdout: debug ms = 1 2024-03-29T04:16:32.830 INFO:teuthology.orchestra.run.smithi151.stdout: debug mon = 20 2024-03-29T04:16:32.830 INFO:teuthology.orchestra.run.smithi151.stdout: debug paxos = 20 2024-03-29T04:16:32.830 INFO:teuthology.orchestra.run.smithi151.stdout: debug auth = 20 2024-03-29T04:16:32.830 INFO:teuthology.orchestra.run.smithi151.stdout: mon data avail warn = 5 2024-03-29T04:16:32.830 INFO:teuthology.orchestra.run.smithi151.stdout: mon mgr mkfs grace = 240 2024-03-29T04:16:32.831 INFO:teuthology.orchestra.run.smithi151.stdout: mon reweight min pgs per osd = 4 2024-03-29T04:16:32.831 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd reporter subtree level = osd 2024-03-29T04:16:32.831 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd prime pg temp = true 2024-03-29T04:16:32.831 INFO:teuthology.orchestra.run.smithi151.stdout: mon reweight min bytes per osd = 10 2024-03-29T04:16:32.831 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.831 INFO:teuthology.orchestra.run.smithi151.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-29T04:16:32.831 INFO:teuthology.orchestra.run.smithi151.stdout: auth mon ticket ttl = 660 # 11m 2024-03-29T04:16:32.831 INFO:teuthology.orchestra.run.smithi151.stdout: auth service ticket ttl = 240 # 4m 2024-03-29T04:16:32.831 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.831 INFO:teuthology.orchestra.run.smithi151.stdout: # don't complain about insecure global_id in the test suite 2024-03-29T04:16:32.831 INFO:teuthology.orchestra.run.smithi151.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-29T04:16:32.831 INFO:teuthology.orchestra.run.smithi151.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-29T04:16:32.831 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.831 INFO:teuthology.orchestra.run.smithi151.stdout: # 1m isn't quite enough 2024-03-29T04:16:32.832 INFO:teuthology.orchestra.run.smithi151.stdout: mon_down_mkfs_grace = 2m 2024-03-29T04:16:32.832 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.832 INFO:teuthology.orchestra.run.smithi151.stdout: mon_warn_on_filestore_osds = false 2024-03-29T04:16:32.832 INFO:teuthology.orchestra.run.smithi151.stdout: mon down mkfs grace = 300 2024-03-29T04:16:32.832 INFO:teuthology.orchestra.run.smithi151.stdout: mon op complaint time = 120 2024-03-29T04:16:32.832 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-29T04:16:32.832 INFO:teuthology.orchestra.run.smithi151.stdout:[client] 2024-03-29T04:16:32.832 INFO:teuthology.orchestra.run.smithi151.stdout: rgw cache enabled = true 2024-03-29T04:16:32.832 INFO:teuthology.orchestra.run.smithi151.stdout: rgw enable ops log = true 2024-03-29T04:16:32.832 INFO:teuthology.orchestra.run.smithi151.stdout: rgw enable usage log = true 2024-03-29T04:16:32.832 INFO:teuthology.orchestra.run.smithi151.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-29T04:16:32.832 INFO:teuthology.orchestra.run.smithi151.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-29T04:16:32.832 INFO:teuthology.orchestra.run.smithi151.stdout: client mount timeout = 600 2024-03-29T04:16:32.832 INFO:teuthology.orchestra.run.smithi151.stdout: debug client = 20 2024-03-29T04:16:32.833 INFO:teuthology.orchestra.run.smithi151.stdout: debug ms = 1 2024-03-29T04:16:32.833 INFO:teuthology.orchestra.run.smithi151.stdout: pid file = "" 2024-03-29T04:16:32.833 INFO:teuthology.orchestra.run.smithi151.stdout: rados mon op timeout = 900 2024-03-29T04:16:32.833 INFO:teuthology.orchestra.run.smithi151.stdout: rados osd op timeout = 900 2024-03-29T04:16:32.833 INFO:teuthology.orchestra.run.smithi151.stdout:[mon.a] 2024-03-29T04:16:32.833 INFO:teuthology.orchestra.run.smithi151.stdout:[mon.b] 2024-03-29T04:16:32.833 INFO:teuthology.orchestra.run.smithi151.stdout:[mon.c] 2024-03-29T04:16:32.833 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.a] 2024-03-29T04:16:32.833 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.c] 2024-03-29T04:16:32.833 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.b] 2024-03-29T04:16:32.833 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.d] 2024-03-29T04:16:32.833 INFO:teuthology.orchestra.run.smithi151.stdout:[mds] 2024-03-29T04:16:32.833 INFO:teuthology.orchestra.run.smithi151.stdout: debug mds = 20 2024-03-29T04:16:32.833 INFO:teuthology.orchestra.run.smithi151.stdout: debug mds balancer = 20 2024-03-29T04:16:32.833 INFO:teuthology.orchestra.run.smithi151.stdout: debug ms = 1 2024-03-29T04:16:32.834 INFO:teuthology.orchestra.run.smithi151.stdout: mds debug frag = True 2024-03-29T04:16:32.834 INFO:teuthology.orchestra.run.smithi151.stdout: mds debug scatterstat = True 2024-03-29T04:16:32.834 INFO:teuthology.orchestra.run.smithi151.stdout: mds op complaint time = 180 2024-03-29T04:16:32.834 INFO:teuthology.orchestra.run.smithi151.stdout: mds verify scatter = True 2024-03-29T04:16:32.834 INFO:teuthology.orchestra.run.smithi151.stdout: mds_bal_replicate_threshold = 1 2024-03-29T04:16:32.834 INFO:teuthology.orchestra.run.smithi151.stdout: osd op complaint time = 180 2024-03-29T04:16:32.834 INFO:teuthology.orchestra.run.smithi151.stdout: rados mon op timeout = 900 2024-03-29T04:16:32.834 INFO:teuthology.orchestra.run.smithi151.stdout: rados osd op timeout = 900 2024-03-29T04:16:32.835 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-29T04:16:32.836 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-29T04:16:32.927 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-29T04:16:32.927 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:16:32.927 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-29T04:16:32.933 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:16:32.933 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-29T04:16:32.981 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi022.front.sepia.ceph.com 2024-03-29T04:16:32.981 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:16:32.981 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-29T04:16:32.981 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-29T04:16:33.047 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:16:33.047 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:16:33.097 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi151.front.sepia.ceph.com 2024-03-29T04:16:33.097 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:16:33.097 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-29T04:16:33.097 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-29T04:16:33.120 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:16:33.120 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:16:33.168 INFO:tasks.ceph:Setting up mon nodes... 2024-03-29T04:16:33.168 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-29T04:16:33.169 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-29T04:16:33.213 INFO:teuthology.orchestra.run.smithi022.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-29T04:16:33.216 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-29T04:16:33.260 INFO:teuthology.orchestra.run.smithi151.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-29T04:16:33.275 INFO:tasks.ceph:Setting up mds nodes... 2024-03-29T04:16:33.276 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-29T04:16:33.322 INFO:teuthology.orchestra.run.smithi022.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-29T04:16:33.326 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-29T04:16:33.339 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-29T04:16:33.423 INFO:teuthology.orchestra.run.smithi022.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-29T04:16:33.424 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-29T04:16:33.480 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-29T04:16:33.525 INFO:teuthology.orchestra.run.smithi151.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-29T04:16:33.528 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-29T04:16:33.584 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-29T04:16:33.666 INFO:teuthology.orchestra.run.smithi151.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-29T04:16:33.669 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-29T04:16:33.722 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-29T04:16:33.722 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-29T04:16:33.760 INFO:teuthology.orchestra.run.smithi022.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-29T04:16:33.772 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-29T04:16:33.849 INFO:teuthology.orchestra.run.smithi022.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-29T04:16:33.861 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-29T04:16:33.899 INFO:teuthology.orchestra.run.smithi151.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-29T04:16:33.911 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-29T04:16:33.989 INFO:teuthology.orchestra.run.smithi151.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-29T04:16:34.000 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-29T04:16:34.000 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi022.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi151.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-29T04:16:34.000 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-29T04:16:34.014 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T04:16:34.014 INFO:tasks.ceph:role: osd.0 2024-03-29T04:16:34.014 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi022.front.sepia.ceph.com 2024-03-29T04:16:34.014 DEBUG:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-29T04:16:34.068 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:16:34.069 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:16:34.069 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:16:34.069 INFO:teuthology.orchestra.run.smithi022.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T04:16:34.069 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:16:34.069 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:16:34.069 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:16:34.069 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T04:16:34.069 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:16:34.070 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:16:34.093 INFO:teuthology.orchestra.run.smithi022.stdout:Discarding blocks...Done. 2024-03-29T04:16:34.095 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2024-03-29T04:16:34.095 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-29T04:16:34.156 DEBUG:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-29T04:16:34.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:16:34.210 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: /sbin/restorecon: command not found 2024-03-29T04:16:34.210 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-29T04:16:34.262 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T04:16:34.262 INFO:tasks.ceph:role: osd.1 2024-03-29T04:16:34.262 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi022.front.sepia.ceph.com 2024-03-29T04:16:34.262 DEBUG:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-29T04:16:34.317 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:16:34.317 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:16:34.317 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:16:34.317 INFO:teuthology.orchestra.run.smithi022.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T04:16:34.317 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:16:34.317 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:16:34.317 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:16:34.318 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T04:16:34.318 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:16:34.318 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:16:34.340 INFO:teuthology.orchestra.run.smithi022.stdout:Discarding blocks...Done. 2024-03-29T04:16:34.342 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2024-03-29T04:16:34.342 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-29T04:16:34.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-29T04:16:34.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:16:34.454 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: /sbin/restorecon: command not found 2024-03-29T04:16:34.454 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-29T04:16:34.507 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T04:16:34.507 INFO:tasks.ceph:role: osd.2 2024-03-29T04:16:34.507 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi022.front.sepia.ceph.com 2024-03-29T04:16:34.508 DEBUG:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-29T04:16:34.561 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:16:34.562 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:16:34.562 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:16:34.562 INFO:teuthology.orchestra.run.smithi022.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T04:16:34.562 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:16:34.562 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:16:34.562 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:16:34.562 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T04:16:34.562 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:16:34.562 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:16:34.585 INFO:teuthology.orchestra.run.smithi022.stdout:Discarding blocks...Done. 2024-03-29T04:16:34.591 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2024-03-29T04:16:34.591 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-29T04:16:34.643 DEBUG:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-29T04:16:34.695 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: /sbin/restorecon: command not found 2024-03-29T04:16:34.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:16:34.696 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-29T04:16:34.757 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T04:16:34.757 INFO:tasks.ceph:role: osd.3 2024-03-29T04:16:34.757 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi022.front.sepia.ceph.com 2024-03-29T04:16:34.757 DEBUG:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-29T04:16:34.812 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:16:34.812 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:16:34.813 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:16:34.813 INFO:teuthology.orchestra.run.smithi022.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T04:16:34.813 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:16:34.813 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:16:34.813 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:16:34.813 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T04:16:34.813 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:16:34.813 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:16:34.836 INFO:teuthology.orchestra.run.smithi022.stdout:Discarding blocks...Done. 2024-03-29T04:16:34.838 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2024-03-29T04:16:34.838 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-29T04:16:34.901 DEBUG:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-29T04:16:34.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:16:34.954 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: /sbin/restorecon: command not found 2024-03-29T04:16:34.954 DEBUG:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:16:35.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:35.093+0000 7f65936aa7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-29T04:16:35.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:35.093+0000 7f65936aa7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-29T04:16:35.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:35.093+0000 7f65936aa7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-29T04:16:35.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:35.093+0000 7f65936aa7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-29T04:16:35.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:35.093+0000 7f65936aa7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-29T04:16:35.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:35.093+0000 7f65936aa7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-29T04:16:38.045 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-29T04:16:38.061 DEBUG:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:16:38.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:38.165+0000 7fe9435a07c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-29T04:16:38.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:38.165+0000 7fe9435a07c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-29T04:16:38.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:38.165+0000 7fe9435a07c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-29T04:16:38.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:38.165+0000 7fe9435a07c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-29T04:16:38.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:38.165+0000 7fe9435a07c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-29T04:16:38.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:38.165+0000 7fe9435a07c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-29T04:16:41.112 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-29T04:16:41.174 DEBUG:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:16:41.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:41.272+0000 7ff4064767c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-29T04:16:41.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:41.276+0000 7ff4064767c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-29T04:16:41.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:41.276+0000 7ff4064767c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-29T04:16:41.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:41.276+0000 7ff4064767c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-29T04:16:41.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:41.276+0000 7ff4064767c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-29T04:16:41.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:41.276+0000 7ff4064767c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-29T04:16:44.240 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-29T04:16:44.297 DEBUG:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:16:44.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:44.400+0000 7f5de55677c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-29T04:16:44.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:44.400+0000 7f5de55677c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-29T04:16:44.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:44.400+0000 7f5de55677c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-29T04:16:44.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:44.400+0000 7f5de55677c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-29T04:16:44.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:44.400+0000 7f5de55677c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-29T04:16:44.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:16:44.400+0000 7f5de55677c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-29T04:16:47.355 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-29T04:16:47.409 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-29T04:16:47.423 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T04:16:47.424 INFO:tasks.ceph:role: osd.4 2024-03-29T04:16:47.424 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi151.front.sepia.ceph.com 2024-03-29T04:16:47.424 DEBUG:teuthology.orchestra.run.smithi151:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-29T04:16:47.486 INFO:teuthology.orchestra.run.smithi151.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:16:47.486 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:16:47.486 INFO:teuthology.orchestra.run.smithi151.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:16:47.486 INFO:teuthology.orchestra.run.smithi151.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T04:16:47.487 INFO:teuthology.orchestra.run.smithi151.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:16:47.487 INFO:teuthology.orchestra.run.smithi151.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:16:47.487 INFO:teuthology.orchestra.run.smithi151.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:16:47.487 INFO:teuthology.orchestra.run.smithi151.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T04:16:47.487 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:16:47.487 INFO:teuthology.orchestra.run.smithi151.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:16:47.975 INFO:teuthology.orchestra.run.smithi151.stdout:Discarding blocks...Done. 2024-03-29T04:16:47.977 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi151.front.sepia.ceph.com -o noatime 2024-03-29T04:16:47.977 DEBUG:teuthology.orchestra.run.smithi151:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-29T04:16:48.125 DEBUG:teuthology.orchestra.run.smithi151:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-29T04:16:48.137 INFO:teuthology.orchestra.run.smithi151.stderr:sudo: /sbin/restorecon: command not found 2024-03-29T04:16:48.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:16:48.138 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-29T04:16:48.189 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T04:16:48.189 INFO:tasks.ceph:role: osd.5 2024-03-29T04:16:48.189 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi151.front.sepia.ceph.com 2024-03-29T04:16:48.190 DEBUG:teuthology.orchestra.run.smithi151:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-29T04:16:48.249 INFO:teuthology.orchestra.run.smithi151.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:16:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:16:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:16:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T04:16:48.250 INFO:teuthology.orchestra.run.smithi151.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:16:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:16:48.250 INFO:teuthology.orchestra.run.smithi151.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:16:48.250 INFO:teuthology.orchestra.run.smithi151.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T04:16:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:16:48.250 INFO:teuthology.orchestra.run.smithi151.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:16:48.679 INFO:teuthology.orchestra.run.smithi151.stdout:Discarding blocks...Done. 2024-03-29T04:16:48.681 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi151.front.sepia.ceph.com -o noatime 2024-03-29T04:16:48.681 DEBUG:teuthology.orchestra.run.smithi151:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-29T04:16:48.743 DEBUG:teuthology.orchestra.run.smithi151:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-29T04:16:48.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:16:48.796 INFO:teuthology.orchestra.run.smithi151.stderr:sudo: /sbin/restorecon: command not found 2024-03-29T04:16:48.796 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-29T04:16:48.850 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T04:16:48.850 INFO:tasks.ceph:role: osd.6 2024-03-29T04:16:48.851 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi151.front.sepia.ceph.com 2024-03-29T04:16:48.851 DEBUG:teuthology.orchestra.run.smithi151:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-29T04:16:48.914 INFO:teuthology.orchestra.run.smithi151.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:16:48.915 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:16:48.915 INFO:teuthology.orchestra.run.smithi151.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:16:48.915 INFO:teuthology.orchestra.run.smithi151.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T04:16:48.915 INFO:teuthology.orchestra.run.smithi151.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:16:48.915 INFO:teuthology.orchestra.run.smithi151.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:16:48.915 INFO:teuthology.orchestra.run.smithi151.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:16:48.915 INFO:teuthology.orchestra.run.smithi151.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T04:16:48.915 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:16:48.915 INFO:teuthology.orchestra.run.smithi151.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:16:49.371 INFO:teuthology.orchestra.run.smithi151.stdout:Discarding blocks...Done. 2024-03-29T04:16:49.374 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi151.front.sepia.ceph.com -o noatime 2024-03-29T04:16:49.374 DEBUG:teuthology.orchestra.run.smithi151:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-29T04:16:49.459 DEBUG:teuthology.orchestra.run.smithi151:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-29T04:16:49.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:16:49.469 INFO:teuthology.orchestra.run.smithi151.stderr:sudo: /sbin/restorecon: command not found 2024-03-29T04:16:49.470 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-29T04:16:49.521 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T04:16:49.521 INFO:tasks.ceph:role: osd.7 2024-03-29T04:16:49.521 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi151.front.sepia.ceph.com 2024-03-29T04:16:49.521 DEBUG:teuthology.orchestra.run.smithi151:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-29T04:16:49.582 INFO:teuthology.orchestra.run.smithi151.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:16:49.582 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:16:49.582 INFO:teuthology.orchestra.run.smithi151.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:16:49.582 INFO:teuthology.orchestra.run.smithi151.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T04:16:49.582 INFO:teuthology.orchestra.run.smithi151.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:16:49.582 INFO:teuthology.orchestra.run.smithi151.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:16:49.582 INFO:teuthology.orchestra.run.smithi151.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:16:49.582 INFO:teuthology.orchestra.run.smithi151.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T04:16:49.582 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:16:49.582 INFO:teuthology.orchestra.run.smithi151.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:16:50.042 INFO:teuthology.orchestra.run.smithi151.stdout:Discarding blocks...Done. 2024-03-29T04:16:50.044 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi151.front.sepia.ceph.com -o noatime 2024-03-29T04:16:50.044 DEBUG:teuthology.orchestra.run.smithi151:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-29T04:16:50.178 DEBUG:teuthology.orchestra.run.smithi151:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-29T04:16:50.188 INFO:teuthology.orchestra.run.smithi151.stderr:sudo: /sbin/restorecon: command not found 2024-03-29T04:16:50.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:16:50.189 DEBUG:teuthology.orchestra.run.smithi151:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:16:50.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:50.319+0000 7f66cf9837c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-29T04:16:50.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:50.327+0000 7f66cf9837c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-29T04:16:50.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:50.327+0000 7f66cf9837c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-29T04:16:50.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:50.327+0000 7f66cf9837c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-29T04:16:50.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:50.327+0000 7f66cf9837c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-29T04:16:50.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:50.335+0000 7f66cf9837c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-29T04:16:53.287 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-29T04:16:53.300 DEBUG:teuthology.orchestra.run.smithi151:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:16:53.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:53.403+0000 7f7eda4997c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-29T04:16:53.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:53.403+0000 7f7eda4997c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-29T04:16:53.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:53.403+0000 7f7eda4997c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-29T04:16:53.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:53.403+0000 7f7eda4997c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-29T04:16:53.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:53.403+0000 7f7eda4997c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-29T04:16:53.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:53.403+0000 7f7eda4997c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-29T04:16:56.362 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-29T04:16:56.419 DEBUG:teuthology.orchestra.run.smithi151:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:16:56.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:56.519+0000 7ff2dd60a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-29T04:16:56.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:56.519+0000 7ff2dd60a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-29T04:16:56.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:56.519+0000 7ff2dd60a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-29T04:16:56.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:56.519+0000 7ff2dd60a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-29T04:16:56.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:56.519+0000 7ff2dd60a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-29T04:16:56.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:56.523+0000 7ff2dd60a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-29T04:16:59.471 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-29T04:16:59.527 DEBUG:teuthology.orchestra.run.smithi151:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:16:59.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:59.627+0000 7f8a66d787c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-29T04:16:59.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:59.627+0000 7f8a66d787c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-29T04:16:59.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:59.627+0000 7f8a66d787c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-29T04:16:59.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:59.627+0000 7f8a66d787c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-29T04:16:59.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:59.627+0000 7f8a66d787c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-29T04:16:59.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:16:59.627+0000 7f8a66d787c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-29T04:17:01.041 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:17:01.044 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:17:02.599 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-29T04:17:02.612 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-29T04:17:02.612 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:17:02.613 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-29T04:17:02.627 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:17:02.627 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-29T04:17:02.684 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:17:02.684 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-29T04:17:02.741 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:17:02.741 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-29T04:17:02.797 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:17:02.828 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-29T04:17:02.852 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:17:02.852 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-29T04:17:02.909 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:17:02.910 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-29T04:17:02.966 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:17:02.966 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-29T04:17:02.980 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:17:02.980 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-29T04:17:03.034 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:17:03.035 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-29T04:17:03.091 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:17:03.091 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-29T04:17:03.147 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:17:03.147 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-29T04:17:03.201 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:17:03.201 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-29T04:17:03.255 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:17:03.255 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-29T04:17:03.310 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:17:03.310 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-29T04:17:03.316 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:17:03.317 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-29T04:17:03.365 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:17:03.365 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-29T04:17:03.370 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:17:03.371 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-29T04:17:03.419 INFO:tasks.ceph:Adding keys to all mons... 2024-03-29T04:17:03.420 DEBUG:teuthology.orchestra.run.smithi022:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-29T04:17:03.422 DEBUG:teuthology.orchestra.run.smithi151:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-29T04:17:03.466 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr.y] 2024-03-29T04:17:03.466 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQChQAZm9VsMDhAAyPeg5VhuJCh+ckoeg2oyLA== 2024-03-29T04:17:03.467 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.a] 2024-03-29T04:17:03.467 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQChQAZm8SiSFBAAaJWqaxBx6kDLU7I0kFLktA== 2024-03-29T04:17:03.467 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.c] 2024-03-29T04:17:03.467 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQChQAZmKb50GhAAnYbR1X7mjwGXOIBuUyKbyg== 2024-03-29T04:17:03.467 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.0] 2024-03-29T04:17:03.467 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQCjQAZmPvDTBRAATcIvO00st0mEUJajJ74c3g== 2024-03-29T04:17:03.468 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.1] 2024-03-29T04:17:03.468 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQCmQAZmghcIChAAJtWKve3N64fgrFbwqAu5fA== 2024-03-29T04:17:03.468 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.2] 2024-03-29T04:17:03.468 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQCpQAZm6cmbEBAAX8HSs37nD+h4M5NxnaSCvQ== 2024-03-29T04:17:03.468 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.3] 2024-03-29T04:17:03.468 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQCsQAZmuoEQGBAAHej5NEVOUGzYe7X0uQ16Fg== 2024-03-29T04:17:03.469 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr.x] 2024-03-29T04:17:03.469 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQChQAZmttHXEBAAtEZLIP57GMS6Hl5oD0g7cg== 2024-03-29T04:17:03.469 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.b] 2024-03-29T04:17:03.469 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQChQAZmiqqdIBAAzeqQSv7ISWKkAKFtOvkVAw== 2024-03-29T04:17:03.469 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.d] 2024-03-29T04:17:03.469 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQChQAZmUioOKRAADH572VL0PgMZtPBAkSo5Zg== 2024-03-29T04:17:03.470 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.4] 2024-03-29T04:17:03.470 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQCyQAZmqVNfExAAOqWGiHZ5tauRUw5vYLn0dw== 2024-03-29T04:17:03.470 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.5] 2024-03-29T04:17:03.470 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQC1QAZmMuxLGBAAml0kEA4UJBm6TlDEC2wvjA== 2024-03-29T04:17:03.470 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.6] 2024-03-29T04:17:03.470 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQC4QAZmROE5HxAAE7wEvV6p+bHMWB3tca+PxQ== 2024-03-29T04:17:03.471 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.7] 2024-03-29T04:17:03.471 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQC7QAZmPyGqJRAAX2sQ5dQpmAFwMltdIP5ivg== 2024-03-29T04:17:03.471 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2024-03-29T04:17:03.471 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQChQAZmViasLhAA6Vlzyow8ABjacgHj4nOI/A== 2024-03-29T04:17:03.471 INFO:teuthology.orchestra.run.smithi022.stdout:[client.2] 2024-03-29T04:17:03.471 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQChQAZmyA39MxAAyXqwQAI7pdhqMfjtFNZnQg== 2024-03-29T04:17:03.472 INFO:teuthology.orchestra.run.smithi022.stdout:[client.1] 2024-03-29T04:17:03.472 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQChQAZmPjHrNhAAF2KuoExf7NtGgDgQAORiKA== 2024-03-29T04:17:03.472 INFO:teuthology.orchestra.run.smithi022.stdout:[client.3] 2024-03-29T04:17:03.472 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQCiQAZmx/GrABAAQG0VVWy/qu6lpNKiombqQw== 2024-03-29T04:17:03.474 INFO:teuthology.orchestra.run.smithi151.stdout:[mgr.y] 2024-03-29T04:17:03.474 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQChQAZm9VsMDhAAyPeg5VhuJCh+ckoeg2oyLA== 2024-03-29T04:17:03.474 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.a] 2024-03-29T04:17:03.475 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQChQAZm8SiSFBAAaJWqaxBx6kDLU7I0kFLktA== 2024-03-29T04:17:03.475 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.c] 2024-03-29T04:17:03.475 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQChQAZmKb50GhAAnYbR1X7mjwGXOIBuUyKbyg== 2024-03-29T04:17:03.475 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.0] 2024-03-29T04:17:03.475 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQCjQAZmPvDTBRAATcIvO00st0mEUJajJ74c3g== 2024-03-29T04:17:03.475 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.1] 2024-03-29T04:17:03.476 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQCmQAZmghcIChAAJtWKve3N64fgrFbwqAu5fA== 2024-03-29T04:17:03.476 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.2] 2024-03-29T04:17:03.476 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQCpQAZm6cmbEBAAX8HSs37nD+h4M5NxnaSCvQ== 2024-03-29T04:17:03.476 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.3] 2024-03-29T04:17:03.476 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQCsQAZmuoEQGBAAHej5NEVOUGzYe7X0uQ16Fg== 2024-03-29T04:17:03.476 INFO:teuthology.orchestra.run.smithi151.stdout:[mgr.x] 2024-03-29T04:17:03.477 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQChQAZmttHXEBAAtEZLIP57GMS6Hl5oD0g7cg== 2024-03-29T04:17:03.477 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.b] 2024-03-29T04:17:03.477 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQChQAZmiqqdIBAAzeqQSv7ISWKkAKFtOvkVAw== 2024-03-29T04:17:03.477 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.d] 2024-03-29T04:17:03.477 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQChQAZmUioOKRAADH572VL0PgMZtPBAkSo5Zg== 2024-03-29T04:17:03.477 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.4] 2024-03-29T04:17:03.477 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQCyQAZmqVNfExAAOqWGiHZ5tauRUw5vYLn0dw== 2024-03-29T04:17:03.478 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.5] 2024-03-29T04:17:03.478 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQC1QAZmMuxLGBAAml0kEA4UJBm6TlDEC2wvjA== 2024-03-29T04:17:03.478 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.6] 2024-03-29T04:17:03.478 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQC4QAZmROE5HxAAE7wEvV6p+bHMWB3tca+PxQ== 2024-03-29T04:17:03.478 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.7] 2024-03-29T04:17:03.478 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQC7QAZmPyGqJRAAX2sQ5dQpmAFwMltdIP5ivg== 2024-03-29T04:17:03.479 INFO:teuthology.orchestra.run.smithi151.stdout:[client.0] 2024-03-29T04:17:03.479 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQChQAZmViasLhAA6Vlzyow8ABjacgHj4nOI/A== 2024-03-29T04:17:03.479 INFO:teuthology.orchestra.run.smithi151.stdout:[client.2] 2024-03-29T04:17:03.479 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQChQAZmyA39MxAAyXqwQAI7pdhqMfjtFNZnQg== 2024-03-29T04:17:03.479 INFO:teuthology.orchestra.run.smithi151.stdout:[client.1] 2024-03-29T04:17:03.479 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQChQAZmPjHrNhAAF2KuoExf7NtGgDgQAORiKA== 2024-03-29T04:17:03.480 INFO:teuthology.orchestra.run.smithi151.stdout:[client.3] 2024-03-29T04:17:03.480 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQCiQAZmx/GrABAAQG0VVWy/qu6lpNKiombqQw== 2024-03-29T04:17:03.481 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T04:17:03.519 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T04:17:03.569 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:17:03.609 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:17:03.659 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:17:03.661 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:17:03.746 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:17:03.749 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:17:03.830 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:17:03.832 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:17:03.911 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:17:03.914 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:17:03.999 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:17:04.002 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:17:04.046 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T04:17:04.048 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T04:17:04.121 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:17:04.129 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:17:04.208 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:17:04.211 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:17:04.298 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:17:04.301 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:17:04.378 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:17:04.386 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:17:04.466 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:17:04.469 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:17:04.554 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:17:04.557 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:17:04.649 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:17:04.651 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:17:04.739 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:17:04.741 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:17:04.787 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:17:04.833 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:17:04.876 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:17:04.878 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:17:04.960 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-29T04:17:04.960 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-29T04:17:04.973 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-29T04:17:05.067 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-29T04:17:05.078 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-29T04:17:05.091 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-29T04:17:05.184 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-29T04:17:05.195 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-29T04:17:05.248 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-29T04:17:05.334 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-29T04:17:05.388 DEBUG:teuthology.orchestra.run.smithi022:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:17:05.391 DEBUG:teuthology.orchestra.run.smithi151:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:17:05.435 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-29T04:17:05.436 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-29T04:17:05.436 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-29T04:17:05.438 INFO:tasks.ceph.mon.a:Started 2024-03-29T04:17:05.438 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-29T04:17:05.438 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-29T04:17:05.480 INFO:tasks.ceph.mon.b:Started 2024-03-29T04:17:05.480 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-29T04:17:05.480 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-29T04:17:05.483 INFO:tasks.ceph.mon.c:Started 2024-03-29T04:17:05.483 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-29T04:17:05.483 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-29T04:17:05.483 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-29T04:17:05.486 INFO:tasks.ceph.mgr.y:Started 2024-03-29T04:17:05.486 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-29T04:17:05.486 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-29T04:17:05.488 INFO:tasks.ceph.mgr.x:Started 2024-03-29T04:17:05.489 INFO:tasks.ceph:Setting crush tunables to default 2024-03-29T04:17:05.489 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph osd crush tunables default 2024-03-29T04:17:05.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:05.628+0000 7fea49fdd640 1 Processor -- start 2024-03-29T04:17:05.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:05.628+0000 7fea49fdd640 1 -- start start 2024-03-29T04:17:05.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:05.628+0000 7fea49fdd640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 0x7fea440b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:05.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:05.628+0000 7fea49fdd640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea4406c850 con 0x7fea4406c070 2024-03-29T04:17:05.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:05.628+0000 7fea49fdd640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea4406cb30 con 0x7fea440cf6c0 2024-03-29T04:17:05.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:05.628+0000 7fea49fdd640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fea4415e8a0 con 0x7fea4415e4c0 2024-03-29T04:17:05.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:05.628+0000 7fea3affd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fea4406c070 0x7fea440cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51554/0 (socket says 172.21.15.22:51554) 2024-03-29T04:17:05.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:05.628+0000 7fea3affd640 1 -- 172.21.15.22:0/3129656543 learned_addr learned my addr 172.21.15.22:0/3129656543 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:05.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:05.628+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 0x7fea440b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:05.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:05.628+0000 7fea437fe640 1 -- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 msgr2=0x7fea440b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T04:17:05.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:05.628+0000 7fea437fe640 1 -- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 msgr2=0x7fea440b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:17:05.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:05.628+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 0x7fea440b9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:17:05.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:05.628+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 0x7fea440b9e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:17:05.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:05.832+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 0x7fea440b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:05.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:05.832+0000 7fea437fe640 1 -- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 msgr2=0x7fea440b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T04:17:05.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:05.832+0000 7fea437fe640 1 -- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 msgr2=0x7fea440b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:17:05.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:05.832+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 0x7fea440b9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:17:05.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:05.832+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 0x7fea440b9e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T04:17:06.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:06.232+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 0x7fea440b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:06.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:06.232+0000 7fea437fe640 1 -- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 msgr2=0x7fea440b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T04:17:06.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:06.232+0000 7fea437fe640 1 -- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 msgr2=0x7fea440b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:17:06.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:06.232+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 0x7fea440b9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:17:06.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:06.232+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 0x7fea440b9e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-29T04:17:07.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:07.032+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 0x7fea440b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:07.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:07.032+0000 7fea437fe640 1 -- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 msgr2=0x7fea440b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T04:17:07.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:07.032+0000 7fea437fe640 1 -- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 msgr2=0x7fea440b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:17:07.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:07.032+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 0x7fea440b9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:17:07.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:07.032+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 0x7fea440b9e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-29T04:17:08.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.628+0000 7fea41ffb640 1 -- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 msgr2=0x7fea440b9e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:17:08.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.628+0000 7fea41ffb640 1 --2- 172.21.15.22:0/3129656543 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea4415e4c0 0x7fea440b9e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:08.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.628+0000 7fea41ffb640 1 -- 172.21.15.22:0/3129656543 >> v1:172.21.15.151:6789/0 conn(0x7fea440cf6c0 legacy=0x7fea440d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:08.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.628+0000 7fea41ffb640 1 -- 172.21.15.22:0/3129656543 >> v1:172.21.15.22:6789/0 conn(0x7fea4406c070 legacy=0x7fea440cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:08.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.628+0000 7fea41ffb640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 0x7fea28005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:08.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.628+0000 7fea41ffb640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 0x7fea280086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:08.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.628+0000 7fea41ffb640 1 -- 172.21.15.22:0/3129656543 --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fea4415e8a0 con 0x7fea28006200 2024-03-29T04:17:08.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.628+0000 7fea41ffb640 1 -- 172.21.15.22:0/3129656543 --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fea28008d30 con 0x7fea28003730 2024-03-29T04:17:08.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.628+0000 7fea3affd640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 0x7fea280086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:08.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.628+0000 7fea3affd640 1 -- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 msgr2=0x7fea280086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-29T04:17:08.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.628+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 0x7fea28005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:08.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.628+0000 7fea3affd640 1 -- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 msgr2=0x7fea280086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:17:08.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.628+0000 7fea3affd640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 0x7fea280086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:17:08.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.628+0000 7fea3affd640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 0x7fea280086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:17:08.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.628+0000 7fea437fe640 1 -- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 msgr2=0x7fea28005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T04:17:08.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.628+0000 7fea437fe640 1 -- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 msgr2=0x7fea28005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:17:08.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.628+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 0x7fea28005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:17:08.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.628+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 0x7fea28005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:17:08.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.832+0000 7fea3affd640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 0x7fea280086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:08.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.832+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 0x7fea28005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:08.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.832+0000 7fea3affd640 1 -- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 msgr2=0x7fea280086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T04:17:08.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.832+0000 7fea3affd640 1 -- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 msgr2=0x7fea280086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:17:08.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.832+0000 7fea3affd640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 0x7fea280086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:17:08.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.832+0000 7fea3affd640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 0x7fea280086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T04:17:08.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.832+0000 7fea437fe640 1 -- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 msgr2=0x7fea28005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-29T04:17:08.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.832+0000 7fea437fe640 1 -- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 msgr2=0x7fea28005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:17:08.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.832+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 0x7fea28005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:17:08.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:08.832+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 0x7fea28005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T04:17:09.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:09.236+0000 7fea3affd640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 0x7fea280086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:09.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:09.236+0000 7fea3affd640 1 -- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 msgr2=0x7fea280086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T04:17:09.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:09.236+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 0x7fea28005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:09.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:09.236+0000 7fea3affd640 1 -- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 msgr2=0x7fea280086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:17:09.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:09.236+0000 7fea3affd640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 0x7fea280086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:17:09.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:09.236+0000 7fea3affd640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 0x7fea280086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-29T04:17:09.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:09.236+0000 7fea437fe640 1 -- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 msgr2=0x7fea28005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-29T04:17:09.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:09.236+0000 7fea437fe640 1 -- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 msgr2=0x7fea28005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:17:09.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:09.236+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 0x7fea28005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:17:09.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:09.236+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 0x7fea28005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-29T04:17:10.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:10.040+0000 7fea3affd640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 0x7fea280086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:10.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:10.040+0000 7fea3affd640 1 -- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 msgr2=0x7fea280086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-29T04:17:10.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:10.040+0000 7fea3affd640 1 -- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 msgr2=0x7fea280086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:17:10.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:10.040+0000 7fea3affd640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 0x7fea280086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:17:10.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:10.040+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 0x7fea28005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:10.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:10.040+0000 7fea3affd640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 0x7fea280086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-29T04:17:10.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:10.040+0000 7fea437fe640 1 -- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 msgr2=0x7fea28005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T04:17:10.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:10.040+0000 7fea437fe640 1 -- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 msgr2=0x7fea28005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:17:10.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:10.040+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 0x7fea28005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:17:10.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:10.040+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 0x7fea28005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-29T04:17:11.104 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:11.120+0000 7f9fee2c8180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-29T04:17:11.146 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:11.163+0000 7f559409a180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-29T04:17:11.568 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:11.588+0000 7f9fee2c8180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-29T04:17:11.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.628+0000 7fea41ffb640 1 -- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 msgr2=0x7fea28005bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:17:11.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.628+0000 7fea41ffb640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.151:3300/0 conn(0x7fea28003730 0x7fea28005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:11.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.628+0000 7fea41ffb640 1 -- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 msgr2=0x7fea280086a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:17:11.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.628+0000 7fea41ffb640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28006200 0x7fea280086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:11.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.628+0000 7fea43fff640 1 -- 172.21.15.22:0/3129656543 reap_dead start 2024-03-29T04:17:11.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.628+0000 7fea41ffb640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28009140 0x7fea2800b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:11.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.628+0000 7fea41ffb640 1 -- 172.21.15.22:0/3129656543 --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea4406ce10 con 0x7fea28008720 2024-03-29T04:17:11.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.628+0000 7fea41ffb640 1 -- 172.21.15.22:0/3129656543 --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea28005d00 con 0x7fea440cf6c0 2024-03-29T04:17:11.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.628+0000 7fea41ffb640 1 -- 172.21.15.22:0/3129656543 --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fea4415e8a0 con 0x7fea28009140 2024-03-29T04:17:11.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.628+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> v2:172.21.15.22:3300/0 conn(0x7fea28009140 0x7fea2800b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:11.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.628+0000 7fea437fe640 1 -- 172.21.15.22:0/3129656543 >> v1:172.21.15.151:6789/0 conn(0x7fea440cf6c0 legacy=0x7fea2800bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:11.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.628+0000 7fea437fe640 1 -- 172.21.15.22:0/3129656543 >> v1:172.21.15.22:6789/0 conn(0x7fea28008720 legacy=0x7fea2800d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:11.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.628+0000 7fea42ffd640 1 -- 172.21.15.22:0/3129656543 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4003604416 0 0) 0x7fea4406ce10 con 0x7fea28008720 2024-03-29T04:17:11.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.628+0000 7fea437fe640 1 -- 172.21.15.22:0/3129656543 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea28005f20 con 0x7fea28009140 2024-03-29T04:17:11.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.636+0000 7fea437fe640 1 --2- 172.21.15.22:0/3129656543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea28009140 0x7fea2800b530 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fea34000ee0 tx=0x7fea3401daa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215c633beb7cc737 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:11.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.636+0000 7fea42ffd640 1 -- 172.21.15.22:0/3129656543 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea340b3070 con 0x7fea28009140 2024-03-29T04:17:11.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.636+0000 7fea42ffd640 1 -- 172.21.15.22:0/3129656543 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea340a6aa0 con 0x7fea28009140 2024-03-29T04:17:11.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.636+0000 7fea42ffd640 1 -- 172.21.15.22:0/3129656543 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3400f040 con 0x7fea28009140 2024-03-29T04:17:11.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.636+0000 7fea49fdd640 1 -- 172.21.15.22:0/3129656543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea28009140 msgr2=0x7fea2800b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:11.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.636+0000 7fea49fdd640 1 --2- 172.21.15.22:0/3129656543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea28009140 0x7fea2800b530 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fea34000ee0 tx=0x7fea3401daa0 comp rx=0 tx=0).stop 2024-03-29T04:17:11.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.636+0000 7fea49fdd640 1 -- 172.21.15.22:0/3129656543 shutdown_connections 2024-03-29T04:17:11.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.636+0000 7fea49fdd640 1 --2- 172.21.15.22:0/3129656543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea28009140 0x7fea2800b530 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:11.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.636+0000 7fea49fdd640 1 -- 172.21.15.22:0/3129656543 >> 172.21.15.22:0/3129656543 conn(0x7fea440b6410 msgr2=0x7fea440d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:11.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.636+0000 7fea49fdd640 1 -- 172.21.15.22:0/3129656543 shutdown_connections 2024-03-29T04:17:11.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.636+0000 7fea49fdd640 1 -- 172.21.15.22:0/3129656543 wait complete. 2024-03-29T04:17:11.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.640+0000 7fea49fdd640 1 Processor -- start 2024-03-29T04:17:11.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.640+0000 7fea49fdd640 1 -- start start 2024-03-29T04:17:11.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea49fdd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea28008720 0x7fea44160470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:11.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea49fdd640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea28009140 0x7fea44151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:11.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea49fdd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea440cf6c0 0x7fea4415a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:11.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea49fdd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fea440b9fb0 con 0x7fea440cf6c0 2024-03-29T04:17:11.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea49fdd640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fea440b9e30 con 0x7fea28008720 2024-03-29T04:17:11.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea49fdd640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fea440ba130 con 0x7fea28009140 2024-03-29T04:17:11.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea43fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea440cf6c0 0x7fea4415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:11.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea43fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea440cf6c0 0x7fea4415a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55372/0 (socket says 172.21.15.22:55372) 2024-03-29T04:17:11.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea43fff640 1 -- 172.21.15.22:0/524139785 learned_addr learned my addr 172.21.15.22:0/524139785 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:11.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea3affd640 1 --2- 172.21.15.22:0/524139785 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea28009140 0x7fea44151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:11.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea437fe640 1 --2- 172.21.15.22:0/524139785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea28008720 0x7fea44160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:11.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea43fff640 1 -- 172.21.15.22:0/524139785 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea28009140 msgr2=0x7fea44151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:11.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea43fff640 1 --2- 172.21.15.22:0/524139785 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea28009140 0x7fea44151e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:11.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea43fff640 1 -- 172.21.15.22:0/524139785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea28008720 msgr2=0x7fea44160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:11.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea43fff640 1 --2- 172.21.15.22:0/524139785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea28008720 0x7fea44160470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:11.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea43fff640 1 -- 172.21.15.22:0/524139785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4415eae0 con 0x7fea440cf6c0 2024-03-29T04:17:11.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea43fff640 1 --2- 172.21.15.22:0/524139785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea440cf6c0 0x7fea4415a360 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fea3c001360 tx=0x7fea3c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:11.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea40ff9640 1 -- 172.21.15.22:0/524139785 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c017070 con 0x7fea440cf6c0 2024-03-29T04:17:11.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea49fdd640 1 -- 172.21.15.22:0/524139785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4415ed80 con 0x7fea440cf6c0 2024-03-29T04:17:11.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea40ff9640 1 -- 172.21.15.22:0/524139785 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea3c007070 con 0x7fea440cf6c0 2024-03-29T04:17:11.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea40ff9640 1 -- 172.21.15.22:0/524139785 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c007440 con 0x7fea440cf6c0 2024-03-29T04:17:11.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea49fdd640 1 -- 172.21.15.22:0/524139785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea4415f340 con 0x7fea440cf6c0 2024-03-29T04:17:11.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea40ff9640 1 -- 172.21.15.22:0/524139785 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fea3c01a070 con 0x7fea440cf6c0 2024-03-29T04:17:11.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea40ff9640 1 -- 172.21.15.22:0/524139785 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fea3c012740 con 0x7fea440cf6c0 2024-03-29T04:17:11.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.644+0000 7fea49fdd640 1 -- 172.21.15.22:0/524139785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea44068900 con 0x7fea440cf6c0 2024-03-29T04:17:11.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.652+0000 7fea40ff9640 1 -- 172.21.15.22:0/524139785 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fea3c021020 con 0x7fea440cf6c0 2024-03-29T04:17:11.648 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:11.667+0000 7f559409a180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-29T04:17:11.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.728+0000 7fea49fdd640 1 -- 172.21.15.22:0/524139785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fea440d43c0 con 0x7fea440cf6c0 2024-03-29T04:17:11.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.728+0000 7fea40ff9640 1 -- 172.21.15.22:0/524139785 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fea3c0077d0 con 0x7fea440cf6c0 2024-03-29T04:17:11.711 INFO:teuthology.orchestra.run.smithi022.stderr:adjusted tunables profile to default 2024-03-29T04:17:11.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.732+0000 7fea49fdd640 1 -- 172.21.15.22:0/524139785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea440cf6c0 msgr2=0x7fea4415a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:11.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.732+0000 7fea49fdd640 1 --2- 172.21.15.22:0/524139785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea440cf6c0 0x7fea4415a360 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fea3c001360 tx=0x7fea3c00c960 comp rx=0 tx=0).stop 2024-03-29T04:17:11.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.732+0000 7fea49fdd640 1 -- 172.21.15.22:0/524139785 shutdown_connections 2024-03-29T04:17:11.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.732+0000 7fea49fdd640 1 --2- 172.21.15.22:0/524139785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea440cf6c0 0x7fea4415a360 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:11.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.732+0000 7fea49fdd640 1 --2- 172.21.15.22:0/524139785 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fea28009140 0x7fea44151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:11.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.732+0000 7fea49fdd640 1 --2- 172.21.15.22:0/524139785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea28008720 0x7fea44160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:11.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.732+0000 7fea49fdd640 1 -- 172.21.15.22:0/524139785 >> 172.21.15.22:0/524139785 conn(0x7fea440b6410 msgr2=0x7fea440b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:11.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.732+0000 7fea49fdd640 1 -- 172.21.15.22:0/524139785 shutdown_connections 2024-03-29T04:17:11.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:11.732+0000 7fea49fdd640 1 -- 172.21.15.22:0/524139785 wait complete. 2024-03-29T04:17:11.731 INFO:tasks.ceph:check_enable_crimson: False 2024-03-29T04:17:11.731 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-29T04:17:11.731 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:17:11.731 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-29T04:17:11.743 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:17:11.743 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-29T04:17:11.759 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:11.776+0000 7f9fee2c8180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-29T04:17:11.770 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:17:11.770 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-29T04:17:11.823 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:17:11.823 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-29T04:17:11.861 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:11.879+0000 7f559409a180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-29T04:17:11.876 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:17:11.876 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-29T04:17:11.888 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:17:11.888 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-29T04:17:11.942 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:17:11.942 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-29T04:17:11.997 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:17:11.997 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-29T04:17:12.050 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new 0022de92-e2b8-4c00-ae06-21ca89b626e8 0 2024-03-29T04:17:12.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.227+0000 7fe29eb1b640 1 Processor -- start 2024-03-29T04:17:12.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.227+0000 7fe29eb1b640 1 -- start start 2024-03-29T04:17:12.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe29eb1b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe2980ccc00 0x7fe2980b7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:12.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe29eb1b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe298158f20 con 0x7fe298158b40 2024-03-29T04:17:12.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe29eb1b640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2981620f0 con 0x7fe2980cc750 2024-03-29T04:17:12.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe29eb1b640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fe298162d00 con 0x7fe2980ccc00 2024-03-29T04:17:12.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe297fff640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7fe2980cc750 0x7fe2980b6c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.151:45470/0 (socket says 172.21.15.151:45470) 2024-03-29T04:17:12.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe297fff640 1 -- 172.21.15.151:0/1861497461 learned_addr learned my addr 172.21.15.151:0/1861497461 (peer_addr_for_me v1:172.21.15.151:0/0) 2024-03-29T04:17:12.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe29d0b8640 1 --2- 172.21.15.151:0/1861497461 >> v2:172.21.15.22:3300/0 conn(0x7fe2980ccc00 0x7fe2980b7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:12.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe2977fe640 1 -- 172.21.15.151:0/1861497461 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 516623549 0 0) 0x7fe2981620f0 con 0x7fe2980cc750 2024-03-29T04:17:12.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe2977fe640 1 -- 172.21.15.151:0/1861497461 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe280003530 con 0x7fe2980cc750 2024-03-29T04:17:12.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe2977fe640 1 -- 172.21.15.151:0/1861497461 <== mon.1 v1:172.21.15.151:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 557709582 0 0) 0x7fe280003530 con 0x7fe2980cc750 2024-03-29T04:17:12.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe2977fe640 1 -- 172.21.15.151:0/1861497461 >> v2:172.21.15.22:3300/0 conn(0x7fe2980ccc00 msgr2=0x7fe2980b7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:12.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe2977fe640 1 --2- 172.21.15.151:0/1861497461 >> v2:172.21.15.22:3300/0 conn(0x7fe2980ccc00 0x7fe2980b7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe2977fe640 1 -- 172.21.15.151:0/1861497461 >> v1:172.21.15.22:6789/0 conn(0x7fe298158b40 legacy=0x7fe2980b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe2977fe640 1 -- 172.21.15.151:0/1861497461 --> v1:172.21.15.151:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2981623d0 con 0x7fe2980cc750 2024-03-29T04:17:12.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe2977fe640 1 -- 172.21.15.151:0/1861497461 <== mon.1 v1:172.21.15.151:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7fe28c001680 con 0x7fe2980cc750 2024-03-29T04:17:12.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe2977fe640 1 -- 172.21.15.151:0/1861497461 >> v1:172.21.15.151:6789/0 conn(0x7fe2980cc750 legacy=0x7fe2980b6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe2977fe640 1 --2- 172.21.15.151:0/1861497461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe280003a10 0x7fe280003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:12.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe2977fe640 1 --2- 172.21.15.151:0/1861497461 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe280004810 0x7fe280004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:12.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe2977fe640 1 --2- 172.21.15.151:0/1861497461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe280005220 0x7fe28000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:12.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe2977fe640 1 -- 172.21.15.151:0/1861497461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe298162d00 con 0x7fe280003a10 2024-03-29T04:17:12.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe2977fe640 1 -- 172.21.15.151:0/1861497461 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe28000c170 con 0x7fe280005220 2024-03-29T04:17:12.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe2977fe640 1 -- 172.21.15.151:0/1861497461 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe28000c370 con 0x7fe280004810 2024-03-29T04:17:12.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe29d0b8640 1 --2- 172.21.15.151:0/1861497461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe280005220 0x7fe28000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:12.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe297fff640 1 --2- 172.21.15.151:0/1861497461 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe280004810 0x7fe280004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:12.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe29c8b7640 1 --2- 172.21.15.151:0/1861497461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe280003a10 0x7fe280003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:12.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe29d0b8640 1 -- 172.21.15.151:0/1861497461 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe280004810 msgr2=0x7fe280004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:12.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe29d0b8640 1 --2- 172.21.15.151:0/1861497461 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe280004810 0x7fe280004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe29d0b8640 1 -- 172.21.15.151:0/1861497461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe280003a10 msgr2=0x7fe280003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:12.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe29d0b8640 1 --2- 172.21.15.151:0/1861497461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe280003a10 0x7fe280003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe29d0b8640 1 -- 172.21.15.151:0/1861497461 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe280003530 con 0x7fe280005220 2024-03-29T04:17:12.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe29d0b8640 1 -- 172.21.15.151:0/1861497461 reap_dead start 2024-03-29T04:17:12.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe29d0b8640 1 --2- 172.21.15.151:0/1861497461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe280005220 0x7fe28000b9d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe290009f70 tx=0x7fe29001cc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b86012bb755e2703 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:12.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe2977fe640 1 -- 172.21.15.151:0/1861497461 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2900b2070 con 0x7fe280005220 2024-03-29T04:17:12.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe2977fe640 1 -- 172.21.15.151:0/1861497461 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2900a5cc0 con 0x7fe280005220 2024-03-29T04:17:12.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.231+0000 7fe2977fe640 1 -- 172.21.15.151:0/1861497461 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe290009bf0 con 0x7fe280005220 2024-03-29T04:17:12.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29eb1b640 1 -- 172.21.15.151:0/1861497461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe280005220 msgr2=0x7fe28000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29eb1b640 1 --2- 172.21.15.151:0/1861497461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe280005220 0x7fe28000b9d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe290009f70 tx=0x7fe29001cc10 comp rx=0 tx=0).stop 2024-03-29T04:17:12.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29eb1b640 1 -- 172.21.15.151:0/1861497461 shutdown_connections 2024-03-29T04:17:12.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29eb1b640 1 --2- 172.21.15.151:0/1861497461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe280005220 0x7fe28000b9d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29eb1b640 1 -- 172.21.15.151:0/1861497461 >> 172.21.15.151:0/1861497461 conn(0x7fe2980b3cf0 msgr2=0x7fe2980b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:12.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29eb1b640 1 -- 172.21.15.151:0/1861497461 shutdown_connections 2024-03-29T04:17:12.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29eb1b640 1 -- 172.21.15.151:0/1861497461 wait complete. 2024-03-29T04:17:12.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29eb1b640 1 Processor -- start 2024-03-29T04:17:12.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29eb1b640 1 -- start start 2024-03-29T04:17:12.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29eb1b640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe280005220 0x7fe2980c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:12.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29eb1b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe29814a380 0x7fe298155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:12.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29eb1b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe29814b870 0x7fe298148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:12.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29eb1b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe2980d1eb0 con 0x7fe29814a380 2024-03-29T04:17:12.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29eb1b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe2980d1d30 con 0x7fe29814b870 2024-03-29T04:17:12.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29eb1b640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe2980b4ff0 con 0x7fe280005220 2024-03-29T04:17:12.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29c8b7640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe280005220 0x7fe2980c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:12.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29c8b7640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe280005220 0x7fe2980c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.151:53164/0 (socket says 172.21.15.151:53164) 2024-03-29T04:17:12.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29d0b8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe29814b870 0x7fe298148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:12.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29c8b7640 1 -- 172.21.15.151:0/3210282003 learned_addr learned my addr 172.21.15.151:0/3210282003 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-29T04:17:12.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe297fff640 1 --2- 172.21.15.151:0/3210282003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe29814a380 0x7fe298155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:12.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29c8b7640 1 -- 172.21.15.151:0/3210282003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe29814b870 msgr2=0x7fe298148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29c8b7640 1 --2- 172.21.15.151:0/3210282003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe29814b870 0x7fe298148d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29c8b7640 1 -- 172.21.15.151:0/3210282003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe29814a380 msgr2=0x7fe298155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29c8b7640 1 --2- 172.21.15.151:0/3210282003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe29814a380 0x7fe298155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29c8b7640 1 -- 172.21.15.151:0/3210282003 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2980c9fa0 con 0x7fe280005220 2024-03-29T04:17:12.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29d0b8640 1 --2- 172.21.15.151:0/3210282003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe29814b870 0x7fe298148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:17:12.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29c8b7640 1 --2- 172.21.15.151:0/3210282003 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe280005220 0x7fe2980c91d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe28800e480 tx=0x7fe28800e940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:12.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe2957fa640 1 -- 172.21.15.151:0/3210282003 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2880052e0 con 0x7fe280005220 2024-03-29T04:17:12.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe2957fa640 1 -- 172.21.15.151:0/3210282003 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe288018070 con 0x7fe280005220 2024-03-29T04:17:12.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29eb1b640 1 -- 172.21.15.151:0/3210282003 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe29814eb30 con 0x7fe280005220 2024-03-29T04:17:12.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe2957fa640 1 -- 172.21.15.151:0/3210282003 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe288003800 con 0x7fe280005220 2024-03-29T04:17:12.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.235+0000 7fe29eb1b640 1 -- 172.21.15.151:0/3210282003 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2980c9bc0 con 0x7fe280005220 2024-03-29T04:17:12.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.239+0000 7fe2957fa640 1 -- 172.21.15.151:0/3210282003 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe2880039a0 con 0x7fe280005220 2024-03-29T04:17:12.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.239+0000 7fe2957fa640 1 -- 172.21.15.151:0/3210282003 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fe28801dd20 con 0x7fe280005220 2024-03-29T04:17:12.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.239+0000 7fe29eb1b640 1 -- 172.21.15.151:0/3210282003 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe29814efe0 con 0x7fe280005220 2024-03-29T04:17:12.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.251+0000 7fe2957fa640 1 -- 172.21.15.151:0/3210282003 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe288003e20 con 0x7fe280005220 2024-03-29T04:17:12.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.319+0000 7fe29eb1b640 1 -- 172.21.15.151:0/3210282003 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0022de92-e2b8-4c00-ae06-21ca89b626e8", "id": 0} v 0) -- 0x7fe298068900 con 0x7fe280005220 2024-03-29T04:17:12.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.327+0000 7fe2957fa640 1 -- 172.21.15.151:0/3210282003 <== mon.2 v2:172.21.15.151:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0022de92-e2b8-4c00-ae06-21ca89b626e8", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fe288005480 con 0x7fe280005220 2024-03-29T04:17:12.307 INFO:teuthology.orchestra.run.smithi151.stdout:0 2024-03-29T04:17:12.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.327+0000 7fe29eb1b640 1 -- 172.21.15.151:0/3210282003 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe280005220 msgr2=0x7fe2980c91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.327+0000 7fe29eb1b640 1 --2- 172.21.15.151:0/3210282003 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe280005220 0x7fe2980c91d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe28800e480 tx=0x7fe28800e940 comp rx=0 tx=0).stop 2024-03-29T04:17:12.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.327+0000 7fe29eb1b640 1 -- 172.21.15.151:0/3210282003 shutdown_connections 2024-03-29T04:17:12.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.327+0000 7fe29eb1b640 1 --2- 172.21.15.151:0/3210282003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe29814b870 0x7fe298148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.327+0000 7fe29eb1b640 1 --2- 172.21.15.151:0/3210282003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe29814a380 0x7fe298155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.327+0000 7fe29eb1b640 1 --2- 172.21.15.151:0/3210282003 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe280005220 0x7fe2980c91d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.327+0000 7fe29eb1b640 1 -- 172.21.15.151:0/3210282003 >> 172.21.15.151:0/3210282003 conn(0x7fe2980b3cf0 msgr2=0x7fe2980cdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:12.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.327+0000 7fe29eb1b640 1 -- 172.21.15.151:0/3210282003 shutdown_connections 2024-03-29T04:17:12.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.327+0000 7fe29eb1b640 1 -- 172.21.15.151:0/3210282003 wait complete. 2024-03-29T04:17:12.324 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new 75b2b30d-31a8-447c-adba-05194124a6a9 1 2024-03-29T04:17:12.338 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:12.356+0000 7f9fee2c8180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-29T04:17:12.394 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:12.412+0000 7f9fee2c8180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-29T04:17:12.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.459+0000 7f85d82aa640 1 Processor -- start 2024-03-29T04:17:12.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.459+0000 7f85d82aa640 1 -- start start 2024-03-29T04:17:12.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.463+0000 7f85d82aa640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f85d015afc0 0x7f85d00b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:12.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.463+0000 7f85d82aa640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f85d00700f0 0x7f85d00ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:12.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.463+0000 7f85d82aa640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85d00b4ba0 con 0x7f85d00b4520 2024-03-29T04:17:12.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.463+0000 7f85d82aa640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f85d0070c30 con 0x7f85d015afc0 2024-03-29T04:17:12.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.463+0000 7f85d82aa640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f85d00b4960 con 0x7f85d00700f0 2024-03-29T04:17:12.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.463+0000 7f85d6847640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f85d00700f0 0x7f85d00ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:12.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.463+0000 7f85d6847640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f85d00700f0 0x7f85d00ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49118/0 (socket says 172.21.15.151:49118) 2024-03-29T04:17:12.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.463+0000 7f85d6847640 1 -- 172.21.15.151:0/1669600381 learned_addr learned my addr 172.21.15.151:0/1669600381 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-29T04:17:12.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.463+0000 7f85d6046640 1 --2- 172.21.15.151:0/1669600381 >> v2:172.21.15.22:3300/0 conn(0x7f85d015afc0 0x7f85d00b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:12.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.463+0000 7f85d6847640 1 -- 172.21.15.151:0/1669600381 >> v2:172.21.15.22:3300/0 conn(0x7f85d015afc0 msgr2=0x7f85d00b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:12.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.463+0000 7f85d6847640 1 --2- 172.21.15.151:0/1669600381 >> v2:172.21.15.22:3300/0 conn(0x7f85d015afc0 0x7f85d00b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.463+0000 7f85d6847640 1 -- 172.21.15.151:0/1669600381 >> v1:172.21.15.22:6789/0 conn(0x7f85d00b4520 legacy=0x7f85d006f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.463+0000 7f85d6847640 1 -- 172.21.15.151:0/1669600381 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d015ec60 con 0x7f85d00700f0 2024-03-29T04:17:12.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d6847640 1 --2- 172.21.15.151:0/1669600381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85d00700f0 0x7f85d00ce7b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f85d006e500 tx=0x7f85cc01c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fccb1468fe2976ca server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:12.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d5044640 1 -- 172.21.15.151:0/1669600381 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc0a5880 con 0x7f85d00700f0 2024-03-29T04:17:12.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d5044640 1 -- 172.21.15.151:0/1669600381 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85cc0a5a20 con 0x7f85d00700f0 2024-03-29T04:17:12.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d5044640 1 -- 172.21.15.151:0/1669600381 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc0a5df0 con 0x7f85d00700f0 2024-03-29T04:17:12.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d82aa640 1 -- 172.21.15.151:0/1669600381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85d00700f0 msgr2=0x7f85d00ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d82aa640 1 --2- 172.21.15.151:0/1669600381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85d00700f0 0x7f85d00ce7b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f85d006e500 tx=0x7f85cc01c9c0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d82aa640 1 -- 172.21.15.151:0/1669600381 shutdown_connections 2024-03-29T04:17:12.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d82aa640 1 --2- 172.21.15.151:0/1669600381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85d00700f0 0x7f85d00ce7b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d82aa640 1 --2- 172.21.15.151:0/1669600381 >> v2:172.21.15.22:3300/0 conn(0x7f85d015afc0 0x7f85d00b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d82aa640 1 -- 172.21.15.151:0/1669600381 >> 172.21.15.151:0/1669600381 conn(0x7f85d00d1070 msgr2=0x7f85d00d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:12.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d82aa640 1 -- 172.21.15.151:0/1669600381 shutdown_connections 2024-03-29T04:17:12.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d82aa640 1 -- 172.21.15.151:0/1669600381 wait complete. 2024-03-29T04:17:12.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d82aa640 1 Processor -- start 2024-03-29T04:17:12.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d82aa640 1 -- start start 2024-03-29T04:17:12.449 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:12.468+0000 7f9fee2c8180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-29T04:17:12.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d82aa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85d00700f0 0x7f85d0159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:12.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d82aa640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85d00b4520 0x7f85d01508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:12.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d82aa640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85d015afc0 0x7f85d01f9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:12.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d82aa640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f85d015fa20 con 0x7f85d00700f0 2024-03-29T04:17:12.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d82aa640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f85d015b3a0 con 0x7f85d015afc0 2024-03-29T04:17:12.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d82aa640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f85d015a400 con 0x7f85d00b4520 2024-03-29T04:17:12.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d5845640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85d00b4520 0x7f85d01508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:12.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d6847640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85d015afc0 0x7f85d01f9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:12.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d5845640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85d00b4520 0x7f85d01508e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.151:53176/0 (socket says 172.21.15.151:53176) 2024-03-29T04:17:12.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d5845640 1 -- 172.21.15.151:0/1096927541 learned_addr learned my addr 172.21.15.151:0/1096927541 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-29T04:17:12.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d6046640 1 --2- 172.21.15.151:0/1096927541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85d00700f0 0x7f85d0159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:12.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d6847640 1 -- 172.21.15.151:0/1096927541 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85d00b4520 msgr2=0x7f85d01508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d6847640 1 --2- 172.21.15.151:0/1096927541 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85d00b4520 0x7f85d01508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d6847640 1 -- 172.21.15.151:0/1096927541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85d00700f0 msgr2=0x7f85d0159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d6847640 1 --2- 172.21.15.151:0/1096927541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85d00700f0 0x7f85d0159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d6847640 1 -- 172.21.15.151:0/1096927541 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d0164af0 con 0x7f85d015afc0 2024-03-29T04:17:12.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85d6847640 1 --2- 172.21.15.151:0/1096927541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85d015afc0 0x7f85d01f9e80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f85cc0ab000 tx=0x7f85cc019760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:12.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.467+0000 7f85c6ffd640 1 -- 172.21.15.151:0/1096927541 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc0bc030 con 0x7f85d015afc0 2024-03-29T04:17:12.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.471+0000 7f85c6ffd640 1 -- 172.21.15.151:0/1096927541 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85cc0b8470 con 0x7f85d015afc0 2024-03-29T04:17:12.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.471+0000 7f85d82aa640 1 -- 172.21.15.151:0/1096927541 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d0164dd0 con 0x7f85d015afc0 2024-03-29T04:17:12.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.471+0000 7f85c6ffd640 1 -- 172.21.15.151:0/1096927541 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc0b8880 con 0x7f85d015afc0 2024-03-29T04:17:12.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.471+0000 7f85d82aa640 1 -- 172.21.15.151:0/1096927541 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d0165390 con 0x7f85d015afc0 2024-03-29T04:17:12.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.471+0000 7f85c6ffd640 1 -- 172.21.15.151:0/1096927541 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f85cc0bf070 con 0x7f85d015afc0 2024-03-29T04:17:12.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.471+0000 7f85c6ffd640 1 -- 172.21.15.151:0/1096927541 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f85cc0c1ca0 con 0x7f85d015afc0 2024-03-29T04:17:12.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.471+0000 7f85d82aa640 1 -- 172.21.15.151:0/1096927541 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d0068900 con 0x7f85d015afc0 2024-03-29T04:17:12.455 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:12.471+0000 7f559409a180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-29T04:17:12.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.471+0000 7f85c6ffd640 1 -- 172.21.15.151:0/1096927541 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f85cc0165e0 con 0x7f85d015afc0 2024-03-29T04:17:12.507 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:12.527+0000 7f559409a180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-29T04:17:12.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.539+0000 7f85d82aa640 1 -- 172.21.15.151:0/1096927541 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd new", "uuid": "75b2b30d-31a8-447c-adba-05194124a6a9", "id": 1} v 0) -- 0x7f85d00706c0 con 0x7f85d015afc0 2024-03-29T04:17:12.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.547+0000 7f85c6ffd640 1 -- 172.21.15.151:0/1096927541 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "75b2b30d-31a8-447c-adba-05194124a6a9", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f85cc0b8a20 con 0x7f85d015afc0 2024-03-29T04:17:12.528 INFO:teuthology.orchestra.run.smithi151.stdout:1 2024-03-29T04:17:12.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.547+0000 7f85d82aa640 1 -- 172.21.15.151:0/1096927541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85d015afc0 msgr2=0x7f85d01f9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.547+0000 7f85d82aa640 1 --2- 172.21.15.151:0/1096927541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85d015afc0 0x7f85d01f9e80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f85cc0ab000 tx=0x7f85cc019760 comp rx=0 tx=0).stop 2024-03-29T04:17:12.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.547+0000 7f85d82aa640 1 -- 172.21.15.151:0/1096927541 shutdown_connections 2024-03-29T04:17:12.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.547+0000 7f85d82aa640 1 --2- 172.21.15.151:0/1096927541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85d015afc0 0x7f85d01f9e80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.547+0000 7f85d82aa640 1 --2- 172.21.15.151:0/1096927541 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85d00b4520 0x7f85d01508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.547+0000 7f85d82aa640 1 --2- 172.21.15.151:0/1096927541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85d00700f0 0x7f85d0159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.547+0000 7f85d82aa640 1 -- 172.21.15.151:0/1096927541 >> 172.21.15.151:0/1096927541 conn(0x7f85d00d1070 msgr2=0x7f85d006f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:12.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.547+0000 7f85d82aa640 1 -- 172.21.15.151:0/1096927541 shutdown_connections 2024-03-29T04:17:12.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.547+0000 7f85d82aa640 1 -- 172.21.15.151:0/1096927541 wait complete. 2024-03-29T04:17:12.530 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:12.548+0000 7f9fee2c8180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-29T04:17:12.544 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new 0de67084-451e-438c-ad00-e722cee6d793 2 2024-03-29T04:17:12.562 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:12.583+0000 7f559409a180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-29T04:17:12.585 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:12.604+0000 7f9fee2c8180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-29T04:17:12.642 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:12.659+0000 7f559409a180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-29T04:17:12.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.679+0000 7f496b4b3640 1 Processor -- start 2024-03-29T04:17:12.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.679+0000 7f496b4b3640 1 -- start start 2024-03-29T04:17:12.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f496b4b3640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f496415c160 0x7f496406b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:12.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f496b4b3640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f496415fe20 0x7f4964160290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:12.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f496b4b3640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4964160c80 con 0x7f49640ccaa0 2024-03-29T04:17:12.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f496b4b3640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f49641607d0 con 0x7f496415c160 2024-03-29T04:17:12.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f496b4b3640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f49640cd680 con 0x7f496415fe20 2024-03-29T04:17:12.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f4968a4e640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f496415fe20 0x7f4964160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:12.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f4969a50640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f49640ccaa0 0x7f49640cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.151:45480/0 (socket says 172.21.15.151:45480) 2024-03-29T04:17:12.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f4968a4e640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f496415fe20 0x7f4964160290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49142/0 (socket says 172.21.15.151:49142) 2024-03-29T04:17:12.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f4969a50640 1 -- 172.21.15.151:0/283928717 learned_addr learned my addr 172.21.15.151:0/283928717 (peer_addr_for_me v1:172.21.15.151:0/0) 2024-03-29T04:17:12.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f496924f640 1 --2- 172.21.15.151:0/283928717 >> v2:172.21.15.22:3300/0 conn(0x7f496415c160 0x7f496406b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:12.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f4968a4e640 1 -- 172.21.15.151:0/283928717 >> v2:172.21.15.22:3300/0 conn(0x7f496415c160 msgr2=0x7f496406b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:12.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f4968a4e640 1 --2- 172.21.15.151:0/283928717 >> v2:172.21.15.22:3300/0 conn(0x7f496415c160 0x7f496406b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f4968a4e640 1 -- 172.21.15.151:0/283928717 >> v1:172.21.15.151:6789/0 conn(0x7f49640ccaa0 legacy=0x7f49640cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f4968a4e640 1 -- 172.21.15.151:0/283928717 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49640cd960 con 0x7f496415fe20 2024-03-29T04:17:12.665 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:12.684+0000 7f9fee2c8180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-29T04:17:12.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f4968a4e640 1 --2- 172.21.15.151:0/283928717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f496415fe20 0x7f4964160290 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4964142bf0 tx=0x7f495401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=640d10ed9b721c06 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:12.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f495bfff640 1 -- 172.21.15.151:0/283928717 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49540ab650 con 0x7f496415fe20 2024-03-29T04:17:12.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f495bfff640 1 -- 172.21.15.151:0/283928717 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49540b0070 con 0x7f496415fe20 2024-03-29T04:17:12.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f495bfff640 1 -- 172.21.15.151:0/283928717 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49540aba10 con 0x7f496415fe20 2024-03-29T04:17:12.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f496b4b3640 1 -- 172.21.15.151:0/283928717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f496415fe20 msgr2=0x7f4964160290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f496b4b3640 1 --2- 172.21.15.151:0/283928717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f496415fe20 0x7f4964160290 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4964142bf0 tx=0x7f495401c450 comp rx=0 tx=0).stop 2024-03-29T04:17:12.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f496b4b3640 1 -- 172.21.15.151:0/283928717 shutdown_connections 2024-03-29T04:17:12.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f496b4b3640 1 --2- 172.21.15.151:0/283928717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f496415fe20 0x7f4964160290 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f496b4b3640 1 --2- 172.21.15.151:0/283928717 >> v2:172.21.15.22:3300/0 conn(0x7f496415c160 0x7f496406b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f496b4b3640 1 -- 172.21.15.151:0/283928717 >> 172.21.15.151:0/283928717 conn(0x7f49640d0c90 msgr2=0x7f49640d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:12.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f496b4b3640 1 -- 172.21.15.151:0/283928717 shutdown_connections 2024-03-29T04:17:12.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.683+0000 7f496b4b3640 1 -- 172.21.15.151:0/283928717 wait complete. 2024-03-29T04:17:12.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f496b4b3640 1 Processor -- start 2024-03-29T04:17:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f496b4b3640 1 -- start start 2024-03-29T04:17:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f496b4b3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49640ccaa0 0x7f496415a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f496b4b3640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f496415c160 0x7f496413ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f496b4b3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496415fe20 0x7f496413a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f496b4b3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f49640cf420 con 0x7f496415fe20 2024-03-29T04:17:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f496b4b3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f49640cf2a0 con 0x7f49640ccaa0 2024-03-29T04:17:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f496b4b3640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f496415d550 con 0x7f496415c160 2024-03-29T04:17:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f496924f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49640ccaa0 0x7f496415a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f4968a4e640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f496415c160 0x7f496413ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f4968a4e640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f496415c160 0x7f496413ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.151:53182/0 (socket says 172.21.15.151:53182) 2024-03-29T04:17:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f4968a4e640 1 -- 172.21.15.151:0/1826218460 learned_addr learned my addr 172.21.15.151:0/1826218460 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-29T04:17:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f496924f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49640ccaa0 0x7f496415a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49146/0 (socket says 172.21.15.151:49146) 2024-03-29T04:17:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f4969a50640 1 --2- 172.21.15.151:0/1826218460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496415fe20 0x7f496413a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:12.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f4968a4e640 1 -- 172.21.15.151:0/1826218460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49640ccaa0 msgr2=0x7f496415a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f4968a4e640 1 --2- 172.21.15.151:0/1826218460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49640ccaa0 0x7f496415a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f4968a4e640 1 -- 172.21.15.151:0/1826218460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496415fe20 msgr2=0x7f496413a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f4968a4e640 1 --2- 172.21.15.151:0/1826218460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496415fe20 0x7f496413a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f4968a4e640 1 -- 172.21.15.151:0/1826218460 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496415af90 con 0x7f496415c160 2024-03-29T04:17:12.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f496924f640 1 --2- 172.21.15.151:0/1826218460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49640ccaa0 0x7f496415a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:17:12.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f4968a4e640 1 --2- 172.21.15.151:0/1826218460 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f496415c160 0x7f496413ca70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f495401a080 tx=0x7f495401ac90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:12.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f4959ffb640 1 -- 172.21.15.151:0/1826218460 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49540bf610 con 0x7f496415c160 2024-03-29T04:17:12.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f4959ffb640 1 -- 172.21.15.151:0/1826218460 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49540bf7d0 con 0x7f496415c160 2024-03-29T04:17:12.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f496b4b3640 1 -- 172.21.15.151:0/1826218460 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496415b270 con 0x7f496415c160 2024-03-29T04:17:12.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f4959ffb640 1 -- 172.21.15.151:0/1826218460 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49540bfbc0 con 0x7f496415c160 2024-03-29T04:17:12.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f496b4b3640 1 -- 172.21.15.151:0/1826218460 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f496415b830 con 0x7f496415c160 2024-03-29T04:17:12.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f4959ffb640 1 -- 172.21.15.151:0/1826218460 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f49540bd020 con 0x7f496415c160 2024-03-29T04:17:12.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f4959ffb640 1 -- 172.21.15.151:0/1826218460 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f49540c8a40 con 0x7f496415c160 2024-03-29T04:17:12.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.687+0000 7f496b4b3640 1 -- 172.21.15.151:0/1826218460 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4964068900 con 0x7f496415c160 2024-03-29T04:17:12.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.691+0000 7f4959ffb640 1 -- 172.21.15.151:0/1826218460 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4954016070 con 0x7f496415c160 2024-03-29T04:17:12.698 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:12.715+0000 7f559409a180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-29T04:17:12.720 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:12.740+0000 7f9fee2c8180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-29T04:17:12.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.759+0000 7f496b4b3640 1 -- 172.21.15.151:0/1826218460 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0de67084-451e-438c-ad00-e722cee6d793", "id": 2} v 0) -- 0x7f496406e040 con 0x7f496415c160 2024-03-29T04:17:12.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.763+0000 7f4959ffb640 1 -- 172.21.15.151:0/1826218460 <== mon.2 v2:172.21.15.151:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0de67084-451e-438c-ad00-e722cee6d793", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f49540c8ce0 con 0x7f496415c160 2024-03-29T04:17:12.746 INFO:teuthology.orchestra.run.smithi151.stdout:2 2024-03-29T04:17:12.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.767+0000 7f496b4b3640 1 -- 172.21.15.151:0/1826218460 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f496415c160 msgr2=0x7f496413ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.767+0000 7f496b4b3640 1 --2- 172.21.15.151:0/1826218460 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f496415c160 0x7f496413ca70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f495401a080 tx=0x7f495401ac90 comp rx=0 tx=0).stop 2024-03-29T04:17:12.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.767+0000 7f496b4b3640 1 -- 172.21.15.151:0/1826218460 shutdown_connections 2024-03-29T04:17:12.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.767+0000 7f496b4b3640 1 --2- 172.21.15.151:0/1826218460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496415fe20 0x7f496413a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.767+0000 7f496b4b3640 1 --2- 172.21.15.151:0/1826218460 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f496415c160 0x7f496413ca70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.767+0000 7f496b4b3640 1 --2- 172.21.15.151:0/1826218460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49640ccaa0 0x7f496415a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.767+0000 7f496b4b3640 1 -- 172.21.15.151:0/1826218460 >> 172.21.15.151:0/1826218460 conn(0x7f49640d0c90 msgr2=0x7f496415f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:12.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.767+0000 7f496b4b3640 1 -- 172.21.15.151:0/1826218460 shutdown_connections 2024-03-29T04:17:12.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.767+0000 7f496b4b3640 1 -- 172.21.15.151:0/1826218460 wait complete. 2024-03-29T04:17:12.763 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new 5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c 3 2024-03-29T04:17:12.779 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:12.799+0000 7f559409a180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-29T04:17:12.834 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:12.851+0000 7f559409a180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-29T04:17:12.879 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:12.896+0000 7f9fee2c8180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-29T04:17:12.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.911+0000 7ff269791640 1 Processor -- start 2024-03-29T04:17:12.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.911+0000 7ff269791640 1 -- start start 2024-03-29T04:17:12.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.911+0000 7ff269791640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff26406f4b0 0x7ff2641654d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:12.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.911+0000 7ff269791640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff26415ace0 con 0x7ff26406c070 2024-03-29T04:17:12.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.911+0000 7ff269791640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff26406c570 con 0x7ff26415a900 2024-03-29T04:17:12.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.911+0000 7ff269791640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7ff26406c850 con 0x7ff26406f4b0 2024-03-29T04:17:12.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.915+0000 7ff2637fe640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff26406f4b0 0x7ff2641654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:12.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.915+0000 7ff262ffd640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7ff26415a900 0x7ff2640afad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.151:45494/0 (socket says 172.21.15.151:45494) 2024-03-29T04:17:12.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.915+0000 7ff262ffd640 1 -- 172.21.15.151:0/3612304814 learned_addr learned my addr 172.21.15.151:0/3612304814 (peer_addr_for_me v1:172.21.15.151:0/0) 2024-03-29T04:17:12.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.915+0000 7ff2637fe640 1 -- 172.21.15.151:0/3612304814 >> v1:172.21.15.151:6789/0 conn(0x7ff26415a900 legacy=0x7ff2640afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.915+0000 7ff2637fe640 1 -- 172.21.15.151:0/3612304814 >> v1:172.21.15.22:6789/0 conn(0x7ff26406c070 legacy=0x7ff26406eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.915+0000 7ff2637fe640 1 -- 172.21.15.151:0/3612304814 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26406cb30 con 0x7ff26406f4b0 2024-03-29T04:17:12.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.915+0000 7ff2637fe640 1 --2- 172.21.15.151:0/3612304814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff26406f4b0 0x7ff2641654d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00aea0 tx=0x7ff25c01da70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=220f475da9f01fc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:12.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.915+0000 7ff261ffb640 1 -- 172.21.15.151:0/3612304814 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c0a67e0 con 0x7ff26406f4b0 2024-03-29T04:17:12.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.915+0000 7ff261ffb640 1 -- 172.21.15.151:0/3612304814 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff25c0a6980 con 0x7ff26406f4b0 2024-03-29T04:17:12.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.915+0000 7ff261ffb640 1 -- 172.21.15.151:0/3612304814 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c0a6d50 con 0x7ff26406f4b0 2024-03-29T04:17:12.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.915+0000 7ff269791640 1 -- 172.21.15.151:0/3612304814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff26406f4b0 msgr2=0x7ff2641654d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.915+0000 7ff269791640 1 --2- 172.21.15.151:0/3612304814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff26406f4b0 0x7ff2641654d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00aea0 tx=0x7ff25c01da70 comp rx=0 tx=0).stop 2024-03-29T04:17:12.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.915+0000 7ff269791640 1 -- 172.21.15.151:0/3612304814 shutdown_connections 2024-03-29T04:17:12.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.915+0000 7ff269791640 1 --2- 172.21.15.151:0/3612304814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff26406f4b0 0x7ff2641654d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.915+0000 7ff269791640 1 -- 172.21.15.151:0/3612304814 >> 172.21.15.151:0/3612304814 conn(0x7ff2640d0c90 msgr2=0x7ff2640d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:12.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.915+0000 7ff269791640 1 -- 172.21.15.151:0/3612304814 shutdown_connections 2024-03-29T04:17:12.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.915+0000 7ff269791640 1 -- 172.21.15.151:0/3612304814 wait complete. 2024-03-29T04:17:12.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff269791640 1 Processor -- start 2024-03-29T04:17:12.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff269791640 1 -- start start 2024-03-29T04:17:12.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff269791640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff26406c070 0x7ff2641554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:12.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff269791640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff26406f4b0 0x7ff264141320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:12.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff269791640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26415a900 0x7ff264150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:12.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff269791640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff264167db0 con 0x7ff26415a900 2024-03-29T04:17:12.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff269791640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff264167c30 con 0x7ff26406f4b0 2024-03-29T04:17:12.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff269791640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff264167f30 con 0x7ff26406c070 2024-03-29T04:17:12.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff262ffd640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff26406c070 0x7ff2641554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:12.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff2627fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff26406f4b0 0x7ff264141320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:12.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff262ffd640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff26406c070 0x7ff2641554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.151:53188/0 (socket says 172.21.15.151:53188) 2024-03-29T04:17:12.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff262ffd640 1 -- 172.21.15.151:0/1202381082 learned_addr learned my addr 172.21.15.151:0/1202381082 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-29T04:17:12.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff2627fc640 1 --2- 172.21.15.151:0/1202381082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff26406f4b0 0x7ff264141320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49178/0 (socket says 172.21.15.151:49178) 2024-03-29T04:17:12.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff2637fe640 1 --2- 172.21.15.151:0/1202381082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26415a900 0x7ff264150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:12.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff262ffd640 1 -- 172.21.15.151:0/1202381082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff26406f4b0 msgr2=0x7ff264141320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff262ffd640 1 --2- 172.21.15.151:0/1202381082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff26406f4b0 0x7ff264141320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff262ffd640 1 -- 172.21.15.151:0/1202381082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26415a900 msgr2=0x7ff264150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff262ffd640 1 --2- 172.21.15.151:0/1202381082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26415a900 0x7ff264150d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff262ffd640 1 -- 172.21.15.151:0/1202381082 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff264154430 con 0x7ff26406c070 2024-03-29T04:17:12.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff2627fc640 1 --2- 172.21.15.151:0/1202381082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff26406f4b0 0x7ff264141320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:17:12.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff262ffd640 1 --2- 172.21.15.151:0/1202381082 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff26406c070 0x7ff2641554d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff25400c400 tx=0x7ff25400c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:12.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff24b7fe640 1 -- 172.21.15.151:0/1202381082 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254017070 con 0x7ff26406c070 2024-03-29T04:17:12.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff24b7fe640 1 -- 172.21.15.151:0/1202381082 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff254005cf0 con 0x7ff26406c070 2024-03-29T04:17:12.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff269791640 1 -- 172.21.15.151:0/1202381082 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff264154710 con 0x7ff26406c070 2024-03-29T04:17:12.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff24b7fe640 1 -- 172.21.15.151:0/1202381082 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2540072d0 con 0x7ff26406c070 2024-03-29T04:17:12.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff269791640 1 -- 172.21.15.151:0/1202381082 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff264154cd0 con 0x7ff26406c070 2024-03-29T04:17:12.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff24b7fe640 1 -- 172.21.15.151:0/1202381082 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff25401a070 con 0x7ff26406c070 2024-03-29T04:17:12.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff24b7fe640 1 -- 172.21.15.151:0/1202381082 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7ff25401cc10 con 0x7ff26406c070 2024-03-29T04:17:12.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.919+0000 7ff269791640 1 -- 172.21.15.151:0/1202381082 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff264068900 con 0x7ff26406c070 2024-03-29T04:17:12.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.923+0000 7ff24b7fe640 1 -- 172.21.15.151:0/1202381082 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff254007470 con 0x7ff26406c070 2024-03-29T04:17:12.943 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:12.960+0000 7f9fee2c8180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-29T04:17:12.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.991+0000 7ff269791640 1 -- 172.21.15.151:0/1202381082 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c", "id": 3} v 0) -- 0x7ff2640b5a30 con 0x7ff26406c070 2024-03-29T04:17:12.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.995+0000 7ff24b7fe640 1 -- 172.21.15.151:0/1202381082 <== mon.2 v2:172.21.15.151:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7ff254007710 con 0x7ff26406c070 2024-03-29T04:17:12.976 INFO:teuthology.orchestra.run.smithi151.stdout:3 2024-03-29T04:17:12.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.995+0000 7ff269791640 1 -- 172.21.15.151:0/1202381082 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff26406c070 msgr2=0x7ff2641554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:12.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.995+0000 7ff269791640 1 --2- 172.21.15.151:0/1202381082 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff26406c070 0x7ff2641554d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff25400c400 tx=0x7ff25400c8c0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.995+0000 7ff269791640 1 -- 172.21.15.151:0/1202381082 shutdown_connections 2024-03-29T04:17:12.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.995+0000 7ff269791640 1 --2- 172.21.15.151:0/1202381082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26415a900 0x7ff264150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.995+0000 7ff269791640 1 --2- 172.21.15.151:0/1202381082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff26406f4b0 0x7ff264141320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.995+0000 7ff269791640 1 --2- 172.21.15.151:0/1202381082 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff26406c070 0x7ff2641554d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:12.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.995+0000 7ff269791640 1 -- 172.21.15.151:0/1202381082 >> 172.21.15.151:0/1202381082 conn(0x7ff2640d0c90 msgr2=0x7ff2640cf3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:12.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.995+0000 7ff269791640 1 -- 172.21.15.151:0/1202381082 shutdown_connections 2024-03-29T04:17:12.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:12.995+0000 7ff269791640 1 -- 172.21.15.151:0/1202381082 wait complete. 2024-03-29T04:17:12.992 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:13.011+0000 7f559409a180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-29T04:17:12.993 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new 085f3376-4fe9-4509-9c38-ac04f9bbcb1c 4 2024-03-29T04:17:13.055 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:13.075+0000 7f559409a180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-29T04:17:13.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.127+0000 7f9eac873640 1 Processor -- start 2024-03-29T04:17:13.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.127+0000 7f9eac873640 1 -- start start 2024-03-29T04:17:13.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.131+0000 7f9eac873640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9ea806f090 0x7f9ea80b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:13.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.131+0000 7f9eac873640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ea80ce4d0 con 0x7f9ea8070640 2024-03-29T04:17:13.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.131+0000 7f9eac873640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ea8070a50 con 0x7f9ea815f880 2024-03-29T04:17:13.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.131+0000 7f9eac873640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f9ea80cc7f0 con 0x7f9ea806f090 2024-03-29T04:17:13.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.131+0000 7f9e9ffff640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9ea806f090 0x7f9ea80b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:13.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.131+0000 7f9e9ffff640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9ea806f090 0x7f9ea80b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49190/0 (socket says 172.21.15.151:49190) 2024-03-29T04:17:13.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.131+0000 7f9e9ffff640 1 -- 172.21.15.151:0/1495846945 learned_addr learned my addr 172.21.15.151:0/1495846945 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-29T04:17:13.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.131+0000 7f9e9ffff640 1 -- 172.21.15.151:0/1495846945 >> v1:172.21.15.151:6789/0 conn(0x7f9ea815f880 legacy=0x7f9ea806e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:13.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.131+0000 7f9e9ffff640 1 -- 172.21.15.151:0/1495846945 >> v1:172.21.15.22:6789/0 conn(0x7f9ea8070640 legacy=0x7f9ea80ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:13.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.131+0000 7f9e9ffff640 1 -- 172.21.15.151:0/1495846945 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea80ccad0 con 0x7f9ea806f090 2024-03-29T04:17:13.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.131+0000 7f9e9ffff640 1 --2- 172.21.15.151:0/1495846945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ea806f090 0x7f9ea80b4ff0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9ea80b4860 tx=0x7f9e9001c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d77bbf0e9c9589a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:13.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.131+0000 7f9ea5d9b640 1 -- 172.21.15.151:0/1495846945 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e900b0070 con 0x7f9ea806f090 2024-03-29T04:17:13.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.131+0000 7f9ea5d9b640 1 -- 172.21.15.151:0/1495846945 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e900ab600 con 0x7f9ea806f090 2024-03-29T04:17:13.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.131+0000 7f9ea5d9b640 1 -- 172.21.15.151:0/1495846945 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e900ab9d0 con 0x7f9ea806f090 2024-03-29T04:17:13.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9eac873640 1 -- 172.21.15.151:0/1495846945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ea806f090 msgr2=0x7f9ea80b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:13.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9eac873640 1 --2- 172.21.15.151:0/1495846945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ea806f090 0x7f9ea80b4ff0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9ea80b4860 tx=0x7f9e9001c3e0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9eac873640 1 -- 172.21.15.151:0/1495846945 shutdown_connections 2024-03-29T04:17:13.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9eac873640 1 --2- 172.21.15.151:0/1495846945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ea806f090 0x7f9ea80b4ff0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9eac873640 1 -- 172.21.15.151:0/1495846945 >> 172.21.15.151:0/1495846945 conn(0x7f9ea80b3cf0 msgr2=0x7f9ea80b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:13.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9eac873640 1 -- 172.21.15.151:0/1495846945 shutdown_connections 2024-03-29T04:17:13.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9eac873640 1 -- 172.21.15.151:0/1495846945 wait complete. 2024-03-29T04:17:13.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9eac873640 1 Processor -- start 2024-03-29T04:17:13.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9eac873640 1 -- start start 2024-03-29T04:17:13.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9eac873640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ea806f090 0x7f9ea80c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:13.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9eac873640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ea8070640 0x7f9ea814e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:13.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9eac873640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ea815f880 0x7f9ea81550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:13.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9eac873640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea80cf650 con 0x7f9ea806f090 2024-03-29T04:17:13.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9eac873640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea80cf4d0 con 0x7f9ea8070640 2024-03-29T04:17:13.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9eac873640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9ea8160850 con 0x7f9ea815f880 2024-03-29T04:17:13.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9e9ffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ea8070640 0x7f9ea814e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:13.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9ea6d9d640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ea815f880 0x7f9ea81550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:13.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9e9ffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ea8070640 0x7f9ea814e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49198/0 (socket says 172.21.15.151:49198) 2024-03-29T04:17:13.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9ea6d9d640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ea815f880 0x7f9ea81550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.151:53190/0 (socket says 172.21.15.151:53190) 2024-03-29T04:17:13.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9e9ffff640 1 -- 172.21.15.151:0/2786465729 learned_addr learned my addr 172.21.15.151:0/2786465729 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-29T04:17:13.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9ea659c640 1 --2- 172.21.15.151:0/2786465729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ea806f090 0x7f9ea80c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:13.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9e9ffff640 1 -- 172.21.15.151:0/2786465729 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ea815f880 msgr2=0x7f9ea81550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:13.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9e9ffff640 1 --2- 172.21.15.151:0/2786465729 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ea815f880 0x7f9ea81550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9e9ffff640 1 -- 172.21.15.151:0/2786465729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ea806f090 msgr2=0x7f9ea80c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:13.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9e9ffff640 1 --2- 172.21.15.151:0/2786465729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ea806f090 0x7f9ea80c6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9e9ffff640 1 -- 172.21.15.151:0/2786465729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea80c7bb0 con 0x7f9ea8070640 2024-03-29T04:17:13.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9ea659c640 1 --2- 172.21.15.151:0/2786465729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ea806f090 0x7f9ea80c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:13.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9e9ffff640 1 --2- 172.21.15.151:0/2786465729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ea8070640 0x7f9ea814e870 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9e9001a010 tx=0x7f9e90019180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:13.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9e9effd640 1 -- 172.21.15.151:0/2786465729 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e900b0040 con 0x7f9ea8070640 2024-03-29T04:17:13.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9e9effd640 1 -- 172.21.15.151:0/2786465729 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e900b4640 con 0x7f9ea8070640 2024-03-29T04:17:13.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9e9effd640 1 -- 172.21.15.151:0/2786465729 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e900b4a30 con 0x7f9ea8070640 2024-03-29T04:17:13.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9eac873640 1 -- 172.21.15.151:0/2786465729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea815ed90 con 0x7f9ea8070640 2024-03-29T04:17:13.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9eac873640 1 -- 172.21.15.151:0/2786465729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea80c8130 con 0x7f9ea8070640 2024-03-29T04:17:13.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9e9effd640 1 -- 172.21.15.151:0/2786465729 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9e900ab4c0 con 0x7f9ea8070640 2024-03-29T04:17:13.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9e9effd640 1 -- 172.21.15.151:0/2786465729 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f9e900b33c0 con 0x7f9ea8070640 2024-03-29T04:17:13.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.135+0000 7f9eac873640 1 -- 172.21.15.151:0/2786465729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea8068900 con 0x7f9ea8070640 2024-03-29T04:17:13.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.139+0000 7f9e9effd640 1 -- 172.21.15.151:0/2786465729 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9e900160f0 con 0x7f9ea8070640 2024-03-29T04:17:13.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.207+0000 7f9eac873640 1 -- 172.21.15.151:0/2786465729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd new", "uuid": "085f3376-4fe9-4509-9c38-ac04f9bbcb1c", "id": 4} v 0) -- 0x7f9ea80d1790 con 0x7f9ea8070640 2024-03-29T04:17:13.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.211+0000 7f9e9effd640 1 -- 172.21.15.151:0/2786465729 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "085f3376-4fe9-4509-9c38-ac04f9bbcb1c", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f9e900b4bd0 con 0x7f9ea8070640 2024-03-29T04:17:13.194 INFO:teuthology.orchestra.run.smithi151.stdout:4 2024-03-29T04:17:13.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.215+0000 7f9eac873640 1 -- 172.21.15.151:0/2786465729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ea8070640 msgr2=0x7f9ea814e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:13.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.215+0000 7f9eac873640 1 --2- 172.21.15.151:0/2786465729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ea8070640 0x7f9ea814e870 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9e9001a010 tx=0x7f9e90019180 comp rx=0 tx=0).stop 2024-03-29T04:17:13.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.215+0000 7f9eac873640 1 -- 172.21.15.151:0/2786465729 shutdown_connections 2024-03-29T04:17:13.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.215+0000 7f9eac873640 1 --2- 172.21.15.151:0/2786465729 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ea815f880 0x7f9ea81550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.215+0000 7f9eac873640 1 --2- 172.21.15.151:0/2786465729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ea8070640 0x7f9ea814e870 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.215+0000 7f9eac873640 1 --2- 172.21.15.151:0/2786465729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ea806f090 0x7f9ea80c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.215+0000 7f9eac873640 1 -- 172.21.15.151:0/2786465729 >> 172.21.15.151:0/2786465729 conn(0x7f9ea80b3cf0 msgr2=0x7f9ea80b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:13.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.215+0000 7f9eac873640 1 -- 172.21.15.151:0/2786465729 shutdown_connections 2024-03-29T04:17:13.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.215+0000 7f9eac873640 1 -- 172.21.15.151:0/2786465729 wait complete. 2024-03-29T04:17:13.210 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new 1da43d7c-2d62-49a6-b02e-e67bbaa9dc35 5 2024-03-29T04:17:13.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.339+0000 7fb226170640 1 Processor -- start 2024-03-29T04:17:13.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.339+0000 7fb226170640 1 -- start start 2024-03-29T04:17:13.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.339+0000 7fb226170640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fb22015b280 0x7fb2200cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:13.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.339+0000 7fb226170640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb2200afc00 0x7fb2200cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:13.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.339+0000 7fb226170640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2200cbcb0 con 0x7fb22015aea0 2024-03-29T04:17:13.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.339+0000 7fb226170640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fb2200ce9d0 con 0x7fb22015b280 2024-03-29T04:17:13.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.339+0000 7fb226170640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb2200cbf90 con 0x7fb2200afc00 2024-03-29T04:17:13.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.343+0000 7fb21ffff640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb2200afc00 0x7fb2200cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:13.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.343+0000 7fb21effd640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fb22015b280 0x7fb2200cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:13.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.343+0000 7fb21ffff640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb2200afc00 0x7fb2200cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.151:53198/0 (socket says 172.21.15.151:53198) 2024-03-29T04:17:13.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.343+0000 7fb21ffff640 1 -- 172.21.15.151:0/1587966845 learned_addr learned my addr 172.21.15.151:0/1587966845 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-29T04:17:13.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.343+0000 7fb21ffff640 1 -- 172.21.15.151:0/1587966845 >> v2:172.21.15.151:3300/0 conn(0x7fb22015b280 msgr2=0x7fb2200cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:13.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.343+0000 7fb21ffff640 1 --2- 172.21.15.151:0/1587966845 >> v2:172.21.15.151:3300/0 conn(0x7fb22015b280 0x7fb2200cc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.343+0000 7fb21ffff640 1 -- 172.21.15.151:0/1587966845 >> v1:172.21.15.151:6789/0 conn(0x7fb22015aea0 legacy=0x7fb2200ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:13.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.343+0000 7fb21ffff640 1 -- 172.21.15.151:0/1587966845 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22006f3e0 con 0x7fb2200afc00 2024-03-29T04:17:13.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.343+0000 7fb21ffff640 1 --2- 172.21.15.151:0/1587966845 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb2200afc00 0x7fb2200cc7d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb218009e50 tx=0x7fb21801ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=675f2309106f6de1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:13.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.343+0000 7fb21e7fc640 1 -- 172.21.15.151:0/1587966845 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2180a7910 con 0x7fb2200afc00 2024-03-29T04:17:13.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.343+0000 7fb21e7fc640 1 -- 172.21.15.151:0/1587966845 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2180a7ab0 con 0x7fb2200afc00 2024-03-29T04:17:13.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.343+0000 7fb21e7fc640 1 -- 172.21.15.151:0/1587966845 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2180af3f0 con 0x7fb2200afc00 2024-03-29T04:17:13.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.343+0000 7fb226170640 1 -- 172.21.15.151:0/1587966845 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb2200afc00 msgr2=0x7fb2200cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:13.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.343+0000 7fb226170640 1 --2- 172.21.15.151:0/1587966845 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb2200afc00 0x7fb2200cc7d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb218009e50 tx=0x7fb21801ec40 comp rx=0 tx=0).stop 2024-03-29T04:17:13.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.343+0000 7fb226170640 1 -- 172.21.15.151:0/1587966845 shutdown_connections 2024-03-29T04:17:13.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.343+0000 7fb226170640 1 --2- 172.21.15.151:0/1587966845 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb2200afc00 0x7fb2200cc7d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.343+0000 7fb226170640 1 --2- 172.21.15.151:0/1587966845 >> v2:172.21.15.151:3300/0 conn(0x7fb22015b280 0x7fb2200cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.343+0000 7fb226170640 1 -- 172.21.15.151:0/1587966845 >> 172.21.15.151:0/1587966845 conn(0x7fb2200cd170 msgr2=0x7fb2200cd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:13.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.343+0000 7fb226170640 1 -- 172.21.15.151:0/1587966845 shutdown_connections 2024-03-29T04:17:13.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.343+0000 7fb226170640 1 -- 172.21.15.151:0/1587966845 wait complete. 2024-03-29T04:17:13.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb226170640 1 Processor -- start 2024-03-29T04:17:13.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb226170640 1 -- start start 2024-03-29T04:17:13.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb226170640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2200afc00 0x7fb2200b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:13.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb226170640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb22015aea0 0x7fb220155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:13.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb226170640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb22015b280 0x7fb2200b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:13.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb226170640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb22006b660 con 0x7fb2200afc00 2024-03-29T04:17:13.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb226170640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb220070620 con 0x7fb22015b280 2024-03-29T04:17:13.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb226170640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb22006b7e0 con 0x7fb22015aea0 2024-03-29T04:17:13.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb21effd640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb22015aea0 0x7fb220155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:13.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb21ffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb22015b280 0x7fb2200b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:13.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb21effd640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb22015aea0 0x7fb220155080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.151:53214/0 (socket says 172.21.15.151:53214) 2024-03-29T04:17:13.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb21effd640 1 -- 172.21.15.151:0/2233479091 learned_addr learned my addr 172.21.15.151:0/2233479091 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-29T04:17:13.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb21ffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb22015b280 0x7fb2200b51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49224/0 (socket says 172.21.15.151:49224) 2024-03-29T04:17:13.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb21f7fe640 1 --2- 172.21.15.151:0/2233479091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2200afc00 0x7fb2200b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:13.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb21ffff640 1 -- 172.21.15.151:0/2233479091 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb22015aea0 msgr2=0x7fb220155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:13.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb21ffff640 1 --2- 172.21.15.151:0/2233479091 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb22015aea0 0x7fb220155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb21ffff640 1 -- 172.21.15.151:0/2233479091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2200afc00 msgr2=0x7fb2200b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:13.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb21ffff640 1 --2- 172.21.15.151:0/2233479091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2200afc00 0x7fb2200b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb21ffff640 1 -- 172.21.15.151:0/2233479091 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb220150120 con 0x7fb22015b280 2024-03-29T04:17:13.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb21effd640 1 --2- 172.21.15.151:0/2233479091 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb22015aea0 0x7fb220155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:17:13.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb21f7fe640 1 --2- 172.21.15.151:0/2233479091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2200afc00 0x7fb2200b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:13.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb21ffff640 1 --2- 172.21.15.151:0/2233479091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb22015b280 0x7fb2200b51d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb2180ad000 tx=0x7fb21801b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:13.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb207fff640 1 -- 172.21.15.151:0/2233479091 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2180bf020 con 0x7fb22015b280 2024-03-29T04:17:13.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb207fff640 1 -- 172.21.15.151:0/2233479091 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb218007e50 con 0x7fb22015b280 2024-03-29T04:17:13.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb207fff640 1 -- 172.21.15.151:0/2233479091 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2180a7910 con 0x7fb22015b280 2024-03-29T04:17:13.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb226170640 1 -- 172.21.15.151:0/2233479091 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2200b4130 con 0x7fb22015b280 2024-03-29T04:17:13.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb226170640 1 -- 172.21.15.151:0/2233479091 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2200b46f0 con 0x7fb22015b280 2024-03-29T04:17:13.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb207fff640 1 -- 172.21.15.151:0/2233479091 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb2180c2070 con 0x7fb22015b280 2024-03-29T04:17:13.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb207fff640 1 -- 172.21.15.151:0/2233479091 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fb2180bade0 con 0x7fb22015b280 2024-03-29T04:17:13.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.347+0000 7fb226170640 1 -- 172.21.15.151:0/2233479091 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb220068900 con 0x7fb22015b280 2024-03-29T04:17:13.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.351+0000 7fb207fff640 1 -- 172.21.15.151:0/2233479091 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb218018830 con 0x7fb22015b280 2024-03-29T04:17:13.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.463+0000 7fb226170640 1 -- 172.21.15.151:0/2233479091 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1da43d7c-2d62-49a6-b02e-e67bbaa9dc35", "id": 5} v 0) -- 0x7fb2200ceb50 con 0x7fb22015b280 2024-03-29T04:17:13.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.467+0000 7fb207fff640 1 -- 172.21.15.151:0/2233479091 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1da43d7c-2d62-49a6-b02e-e67bbaa9dc35", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fb2180ba6b0 con 0x7fb22015b280 2024-03-29T04:17:13.450 INFO:teuthology.orchestra.run.smithi151.stdout:5 2024-03-29T04:17:13.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.471+0000 7fb226170640 1 -- 172.21.15.151:0/2233479091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb22015b280 msgr2=0x7fb2200b51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:13.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.471+0000 7fb226170640 1 --2- 172.21.15.151:0/2233479091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb22015b280 0x7fb2200b51d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb2180ad000 tx=0x7fb21801b9e0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.471+0000 7fb226170640 1 -- 172.21.15.151:0/2233479091 shutdown_connections 2024-03-29T04:17:13.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.471+0000 7fb226170640 1 --2- 172.21.15.151:0/2233479091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb22015b280 0x7fb2200b51d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.471+0000 7fb226170640 1 --2- 172.21.15.151:0/2233479091 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb22015aea0 0x7fb220155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.471+0000 7fb226170640 1 --2- 172.21.15.151:0/2233479091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2200afc00 0x7fb2200b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.471+0000 7fb226170640 1 -- 172.21.15.151:0/2233479091 >> 172.21.15.151:0/2233479091 conn(0x7fb2200cd170 msgr2=0x7fb2200b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:13.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.471+0000 7fb226170640 1 -- 172.21.15.151:0/2233479091 shutdown_connections 2024-03-29T04:17:13.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.471+0000 7fb226170640 1 -- 172.21.15.151:0/2233479091 wait complete. 2024-03-29T04:17:13.466 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new 3a524e69-571d-4115-b9e6-e130c1f34085 6 2024-03-29T04:17:13.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.591+0000 7fcfc6446640 1 Processor -- start 2024-03-29T04:17:13.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.591+0000 7fcfc6446640 1 -- start start 2024-03-29T04:17:13.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.591+0000 7fcfc6446640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcfc015afc0 0x7fcfc00b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:13.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.591+0000 7fcfc6446640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcfc00b4520 0x7fcfc006f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:13.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.591+0000 7fcfc6446640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fcfc006ff20 0x7fcfc006c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:13.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.591+0000 7fcfc6446640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fcfc0070c30 con 0x7fcfc015afc0 2024-03-29T04:17:13.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.591+0000 7fcfc6446640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fcfc00b4960 con 0x7fcfc006ff20 2024-03-29T04:17:13.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.591+0000 7fcfc6446640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc00b4ba0 con 0x7fcfc00b4520 2024-03-29T04:17:13.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.591+0000 7fcfbf7fe640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcfc00b4520 0x7fcfc006f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:13.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.591+0000 7fcfbf7fe640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcfc00b4520 0x7fcfc006f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.151:50344/0 (socket says 172.21.15.151:50344) 2024-03-29T04:17:13.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.591+0000 7fcfbf7fe640 1 -- 172.21.15.151:0/2491887812 learned_addr learned my addr 172.21.15.151:0/2491887812 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-29T04:17:13.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.591+0000 7fcfc49e3640 1 --2- 172.21.15.151:0/2491887812 >> v2:172.21.15.151:3300/0 conn(0x7fcfc006ff20 0x7fcfc006c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:13.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.591+0000 7fcfbffff640 1 --2- 172.21.15.151:0/2491887812 >> v2:172.21.15.22:3300/0 conn(0x7fcfc015afc0 0x7fcfc00b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:13.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.595+0000 7fcfbf7fe640 1 -- 172.21.15.151:0/2491887812 >> v2:172.21.15.151:3300/0 conn(0x7fcfc006ff20 msgr2=0x7fcfc006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:13.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.595+0000 7fcfbf7fe640 1 --2- 172.21.15.151:0/2491887812 >> v2:172.21.15.151:3300/0 conn(0x7fcfc006ff20 0x7fcfc006c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.595+0000 7fcfbf7fe640 1 -- 172.21.15.151:0/2491887812 >> v2:172.21.15.22:3300/0 conn(0x7fcfc015afc0 msgr2=0x7fcfc00b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:13.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.595+0000 7fcfbf7fe640 1 --2- 172.21.15.151:0/2491887812 >> v2:172.21.15.22:3300/0 conn(0x7fcfc015afc0 0x7fcfc00b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.595+0000 7fcfbf7fe640 1 -- 172.21.15.151:0/2491887812 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc015ec60 con 0x7fcfc00b4520 2024-03-29T04:17:13.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.595+0000 7fcfbffff640 1 --2- 172.21.15.151:0/2491887812 >> v2:172.21.15.22:3300/0 conn(0x7fcfc015afc0 0x7fcfc00b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:13.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.595+0000 7fcfbf7fe640 1 --2- 172.21.15.151:0/2491887812 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcfc00b4520 0x7fcfc006f9e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00b5da0 tx=0x7fcfb001e930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92f477ae13eb0494 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:13.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.595+0000 7fcfbeffd640 1 -- 172.21.15.151:0/2491887812 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb00b4070 con 0x7fcfc00b4520 2024-03-29T04:17:13.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.595+0000 7fcfbeffd640 1 -- 172.21.15.151:0/2491887812 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb00a78b0 con 0x7fcfc00b4520 2024-03-29T04:17:13.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.595+0000 7fcfbeffd640 1 -- 172.21.15.151:0/2491887812 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb00a7c80 con 0x7fcfc00b4520 2024-03-29T04:17:13.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.595+0000 7fcfc6446640 1 -- 172.21.15.151:0/2491887812 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcfc00b4520 msgr2=0x7fcfc006f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:13.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.595+0000 7fcfc6446640 1 --2- 172.21.15.151:0/2491887812 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcfc00b4520 0x7fcfc006f9e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00b5da0 tx=0x7fcfb001e930 comp rx=0 tx=0).stop 2024-03-29T04:17:13.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.595+0000 7fcfc6446640 1 -- 172.21.15.151:0/2491887812 shutdown_connections 2024-03-29T04:17:13.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.595+0000 7fcfc6446640 1 --2- 172.21.15.151:0/2491887812 >> v2:172.21.15.151:3300/0 conn(0x7fcfc006ff20 0x7fcfc006c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.595+0000 7fcfc6446640 1 --2- 172.21.15.151:0/2491887812 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcfc00b4520 0x7fcfc006f9e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.595+0000 7fcfc6446640 1 --2- 172.21.15.151:0/2491887812 >> v2:172.21.15.22:3300/0 conn(0x7fcfc015afc0 0x7fcfc00b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.595+0000 7fcfc6446640 1 -- 172.21.15.151:0/2491887812 >> 172.21.15.151:0/2491887812 conn(0x7fcfc00d1070 msgr2=0x7fcfc00d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:13.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.595+0000 7fcfc6446640 1 -- 172.21.15.151:0/2491887812 shutdown_connections 2024-03-29T04:17:13.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.595+0000 7fcfc6446640 1 -- 172.21.15.151:0/2491887812 wait complete. 2024-03-29T04:17:13.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfc6446640 1 Processor -- start 2024-03-29T04:17:13.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfc6446640 1 -- start start 2024-03-29T04:17:13.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfc6446640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfc006ff20 0x7fcfc0147010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:13.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfc6446640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfc00b4520 0x7fcfc014ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:13.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfc6446640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcfc015afc0 0x7fcfc0157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:13.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfc6446640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc015fa20 con 0x7fcfc00b4520 2024-03-29T04:17:13.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfc6446640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc015b3a0 con 0x7fcfc006ff20 2024-03-29T04:17:13.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfc6446640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc014e7c0 con 0x7fcfc015afc0 2024-03-29T04:17:13.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfbffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfc006ff20 0x7fcfc0147010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:13.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfc49e3640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcfc015afc0 0x7fcfc0157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:13.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfc49e3640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcfc015afc0 0x7fcfc0157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.151:50360/0 (socket says 172.21.15.151:50360) 2024-03-29T04:17:13.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfc49e3640 1 -- 172.21.15.151:0/1607343936 learned_addr learned my addr 172.21.15.151:0/1607343936 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-29T04:17:13.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfbf7fe640 1 --2- 172.21.15.151:0/1607343936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfc00b4520 0x7fcfc014ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:13.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfbffff640 1 -- 172.21.15.151:0/1607343936 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcfc015afc0 msgr2=0x7fcfc0157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:13.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfbffff640 1 --2- 172.21.15.151:0/1607343936 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcfc015afc0 0x7fcfc0157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfbffff640 1 -- 172.21.15.151:0/1607343936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfc00b4520 msgr2=0x7fcfc014ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:13.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfbffff640 1 --2- 172.21.15.151:0/1607343936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfc00b4520 0x7fcfc014ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfbffff640 1 -- 172.21.15.151:0/1607343936 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc0158060 con 0x7fcfc006ff20 2024-03-29T04:17:13.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfbffff640 1 --2- 172.21.15.151:0/1607343936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfc006ff20 0x7fcfc0147010 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400c7f0 tx=0x7fcfb400ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:13.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfbcff9640 1 -- 172.21.15.151:0/1607343936 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4015070 con 0x7fcfc006ff20 2024-03-29T04:17:13.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfbcff9640 1 -- 172.21.15.151:0/1607343936 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb4004220 con 0x7fcfc006ff20 2024-03-29T04:17:13.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfc6446640 1 -- 172.21.15.151:0/1607343936 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc0157d80 con 0x7fcfc006ff20 2024-03-29T04:17:13.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfbcff9640 1 -- 172.21.15.151:0/1607343936 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4003620 con 0x7fcfc006ff20 2024-03-29T04:17:13.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfc6446640 1 -- 172.21.15.151:0/1607343936 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc014e450 con 0x7fcfc006ff20 2024-03-29T04:17:13.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfbcff9640 1 -- 172.21.15.151:0/1607343936 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcfb40039d0 con 0x7fcfc006ff20 2024-03-29T04:17:13.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfbcff9640 1 -- 172.21.15.151:0/1607343936 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fcfb4019a90 con 0x7fcfc006ff20 2024-03-29T04:17:13.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.599+0000 7fcfc6446640 1 -- 172.21.15.151:0/1607343936 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfc0068900 con 0x7fcfc006ff20 2024-03-29T04:17:13.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.603+0000 7fcfbcff9640 1 -- 172.21.15.151:0/1607343936 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fcfb40194a0 con 0x7fcfc006ff20 2024-03-29T04:17:13.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.663+0000 7fcfc6446640 1 -- 172.21.15.151:0/1607343936 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3a524e69-571d-4115-b9e6-e130c1f34085", "id": 6} v 0) -- 0x7fcfc0158380 con 0x7fcfc006ff20 2024-03-29T04:17:13.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.671+0000 7fcfbcff9640 1 -- 172.21.15.151:0/1607343936 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3a524e69-571d-4115-b9e6-e130c1f34085", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fcfb4019680 con 0x7fcfc006ff20 2024-03-29T04:17:13.652 INFO:teuthology.orchestra.run.smithi151.stdout:6 2024-03-29T04:17:13.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.671+0000 7fcfc6446640 1 -- 172.21.15.151:0/1607343936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfc006ff20 msgr2=0x7fcfc0147010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:13.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.671+0000 7fcfc6446640 1 --2- 172.21.15.151:0/1607343936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfc006ff20 0x7fcfc0147010 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400c7f0 tx=0x7fcfb400ccb0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.671+0000 7fcfc6446640 1 -- 172.21.15.151:0/1607343936 shutdown_connections 2024-03-29T04:17:13.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.671+0000 7fcfc6446640 1 --2- 172.21.15.151:0/1607343936 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcfc015afc0 0x7fcfc0157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.671+0000 7fcfc6446640 1 --2- 172.21.15.151:0/1607343936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfc00b4520 0x7fcfc014ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.671+0000 7fcfc6446640 1 --2- 172.21.15.151:0/1607343936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfc006ff20 0x7fcfc0147010 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.671+0000 7fcfc6446640 1 -- 172.21.15.151:0/1607343936 >> 172.21.15.151:0/1607343936 conn(0x7fcfc00d1070 msgr2=0x7fcfc006b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:13.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.671+0000 7fcfc6446640 1 -- 172.21.15.151:0/1607343936 shutdown_connections 2024-03-29T04:17:13.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.671+0000 7fcfc6446640 1 -- 172.21.15.151:0/1607343936 wait complete. 2024-03-29T04:17:13.668 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd new bdf4fc46-ba73-47af-b980-11f11f191d77 7 2024-03-29T04:17:13.705 INFO:tasks.ceph.mgr.y.smithi022.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-29T04:17:13.705 INFO:tasks.ceph.mgr.y.smithi022.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-29T04:17:13.705 INFO:tasks.ceph.mgr.y.smithi022.stderr: from numpy import show_config as show_numpy_config 2024-03-29T04:17:13.705 INFO:tasks.ceph.mgr.x.smithi151.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-29T04:17:13.705 INFO:tasks.ceph.mgr.x.smithi151.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-29T04:17:13.705 INFO:tasks.ceph.mgr.x.smithi151.stderr: from numpy import show_config as show_numpy_config 2024-03-29T04:17:13.739 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:13.756+0000 7f9fee2c8180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-29T04:17:13.754 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:13.771+0000 7f559409a180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-29T04:17:13.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.799+0000 7f5ca0883640 1 Processor -- start 2024-03-29T04:17:13.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.799+0000 7f5ca0883640 1 -- start start 2024-03-29T04:17:13.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.799+0000 7f5ca0883640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5c9c147e00 0x7f5c9c0b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:13.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.803+0000 7f5ca0883640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5c9c0b4f30 0x7f5c9c06c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:13.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.803+0000 7f5ca0883640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c9c0b5470 con 0x7f5c9c06c5b0 2024-03-29T04:17:13.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.803+0000 7f5ca0883640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f5c9c0b5750 con 0x7f5c9c147e00 2024-03-29T04:17:13.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.803+0000 7f5ca0883640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5c9c0b5a30 con 0x7f5c9c0b4f30 2024-03-29T04:17:13.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.803+0000 7f5c9ad9d640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f5c9c06c5b0 0x7f5c9c06f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.151:38120/0 (socket says 172.21.15.151:38120) 2024-03-29T04:17:13.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.803+0000 7f5c9ad9d640 1 -- 172.21.15.151:0/1060172455 learned_addr learned my addr 172.21.15.151:0/1060172455 (peer_addr_for_me v1:172.21.15.151:0/0) 2024-03-29T04:17:13.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.803+0000 7f5c99d9b640 1 --2- 172.21.15.151:0/1060172455 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5c9c0b4f30 0x7f5c9c06c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:13.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.803+0000 7f5c9a59c640 1 --2- 172.21.15.151:0/1060172455 >> v2:172.21.15.22:3300/0 conn(0x7f5c9c147e00 0x7f5c9c0b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:13.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.803+0000 7f5c9959a640 1 -- 172.21.15.151:0/1060172455 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3773000480 0 0) 0x7f5c9c0b5470 con 0x7f5c9c06c5b0 2024-03-29T04:17:13.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.803+0000 7f5c9959a640 1 -- 172.21.15.151:0/1060172455 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c84003530 con 0x7f5c9c06c5b0 2024-03-29T04:17:13.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.803+0000 7f5c99d9b640 1 -- 172.21.15.151:0/1060172455 >> v2:172.21.15.22:3300/0 conn(0x7f5c9c147e00 msgr2=0x7f5c9c0b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:13.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.803+0000 7f5c99d9b640 1 --2- 172.21.15.151:0/1060172455 >> v2:172.21.15.22:3300/0 conn(0x7f5c9c147e00 0x7f5c9c0b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.803+0000 7f5c99d9b640 1 -- 172.21.15.151:0/1060172455 >> v1:172.21.15.151:6789/0 conn(0x7f5c9c06c5b0 legacy=0x7f5c9c06f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:13.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.803+0000 7f5c99d9b640 1 -- 172.21.15.151:0/1060172455 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9c0b5d10 con 0x7f5c9c0b4f30 2024-03-29T04:17:13.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.807+0000 7f5c99d9b640 1 --2- 172.21.15.151:0/1060172455 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5c9c0b4f30 0x7f5c9c06c070 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5c900097f0 tx=0x7f5c9001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=184a4f850c2c6603 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:13.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.807+0000 7f5c9959a640 1 -- 172.21.15.151:0/1060172455 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c900b0070 con 0x7f5c9c0b4f30 2024-03-29T04:17:13.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.807+0000 7f5c9959a640 1 -- 172.21.15.151:0/1060172455 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c900ab600 con 0x7f5c9c0b4f30 2024-03-29T04:17:13.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.807+0000 7f5c9959a640 1 -- 172.21.15.151:0/1060172455 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c900ab9d0 con 0x7f5c9c0b4f30 2024-03-29T04:17:13.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.807+0000 7f5ca0883640 1 -- 172.21.15.151:0/1060172455 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5c9c0b4f30 msgr2=0x7f5c9c06c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:13.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.807+0000 7f5ca0883640 1 --2- 172.21.15.151:0/1060172455 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5c9c0b4f30 0x7f5c9c06c070 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5c900097f0 tx=0x7f5c9001c4a0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.807+0000 7f5ca0883640 1 -- 172.21.15.151:0/1060172455 shutdown_connections 2024-03-29T04:17:13.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.807+0000 7f5ca0883640 1 --2- 172.21.15.151:0/1060172455 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5c9c0b4f30 0x7f5c9c06c070 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.807+0000 7f5ca0883640 1 --2- 172.21.15.151:0/1060172455 >> v2:172.21.15.22:3300/0 conn(0x7f5c9c147e00 0x7f5c9c0b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.807+0000 7f5ca0883640 1 -- 172.21.15.151:0/1060172455 >> 172.21.15.151:0/1060172455 conn(0x7f5c9c0d1070 msgr2=0x7f5c9c0d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:13.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.807+0000 7f5ca0883640 1 -- 172.21.15.151:0/1060172455 shutdown_connections 2024-03-29T04:17:13.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.807+0000 7f5ca0883640 1 -- 172.21.15.151:0/1060172455 wait complete. 2024-03-29T04:17:13.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.807+0000 7f5ca0883640 1 Processor -- start 2024-03-29T04:17:13.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.807+0000 7f5ca0883640 1 -- start start 2024-03-29T04:17:13.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5ca0883640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c9c06c5b0 0x7f5c9c168010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:13.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5ca0883640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c9c0b4f30 0x7f5c9c146c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:13.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5ca0883640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5c9c147e00 0x7f5c9c160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:13.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5ca0883640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c16b330 con 0x7f5c9c0b4f30 2024-03-29T04:17:13.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5ca0883640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c06c9c0 con 0x7f5c9c06c5b0 2024-03-29T04:17:13.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5ca0883640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5c9c16b4b0 con 0x7f5c9c147e00 2024-03-29T04:17:13.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5c9a59c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c9c06c5b0 0x7f5c9c168010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:13.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5c9a59c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c9c06c5b0 0x7f5c9c168010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39084/0 (socket says 172.21.15.151:39084) 2024-03-29T04:17:13.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5c9a59c640 1 -- 172.21.15.151:0/4220434197 learned_addr learned my addr 172.21.15.151:0/4220434197 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-29T04:17:13.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5c9ad9d640 1 --2- 172.21.15.151:0/4220434197 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5c9c147e00 0x7f5c9c160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:13.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5c99d9b640 1 --2- 172.21.15.151:0/4220434197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c9c0b4f30 0x7f5c9c146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:13.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5c9a59c640 1 -- 172.21.15.151:0/4220434197 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5c9c147e00 msgr2=0x7f5c9c160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:13.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5c9a59c640 1 --2- 172.21.15.151:0/4220434197 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5c9c147e00 0x7f5c9c160a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5c9a59c640 1 -- 172.21.15.151:0/4220434197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c9c0b4f30 msgr2=0x7f5c9c146c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:13.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5c9a59c640 1 --2- 172.21.15.151:0/4220434197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c9c0b4f30 0x7f5c9c146c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5c9a59c640 1 -- 172.21.15.151:0/4220434197 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9c167c10 con 0x7f5c9c06c5b0 2024-03-29T04:17:13.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5c99d9b640 1 --2- 172.21.15.151:0/4220434197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c9c0b4f30 0x7f5c9c146c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:13.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5c9ad9d640 1 --2- 172.21.15.151:0/4220434197 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5c9c147e00 0x7f5c9c160a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:17:13.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5c9a59c640 1 --2- 172.21.15.151:0/4220434197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c9c06c5b0 0x7f5c9c168010 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5c94007e40 tx=0x7f5c9400e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:13.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5c8affd640 1 -- 172.21.15.151:0/4220434197 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94014620 con 0x7f5c9c06c5b0 2024-03-29T04:17:13.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5c8affd640 1 -- 172.21.15.151:0/4220434197 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c94019070 con 0x7f5c9c06c5b0 2024-03-29T04:17:13.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5ca0883640 1 -- 172.21.15.151:0/4220434197 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9c166740 con 0x7f5c9c06c5b0 2024-03-29T04:17:13.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5c8affd640 1 -- 172.21.15.151:0/4220434197 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94014a00 con 0x7f5c9c06c5b0 2024-03-29T04:17:13.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5ca0883640 1 -- 172.21.15.151:0/4220434197 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c9c166d00 con 0x7f5c9c06c5b0 2024-03-29T04:17:13.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5c8affd640 1 -- 172.21.15.151:0/4220434197 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5c94014ba0 con 0x7f5c9c06c5b0 2024-03-29T04:17:13.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5c8affd640 1 -- 172.21.15.151:0/4220434197 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f5c9401d3c0 con 0x7f5c9c06c5b0 2024-03-29T04:17:13.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5ca0883640 1 -- 172.21.15.151:0/4220434197 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c9c1f75b0 con 0x7f5c9c06c5b0 2024-03-29T04:17:13.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.811+0000 7f5c8affd640 1 -- 172.21.15.151:0/4220434197 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5c9401e460 con 0x7f5c9c06c5b0 2024-03-29T04:17:13.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.875+0000 7f5ca0883640 1 -- 172.21.15.151:0/4220434197 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bdf4fc46-ba73-47af-b980-11f11f191d77", "id": 7} v 0) -- 0x7f5c9c075930 con 0x7f5c9c06c5b0 2024-03-29T04:17:13.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.883+0000 7f5c8affd640 1 -- 172.21.15.151:0/4220434197 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bdf4fc46-ba73-47af-b980-11f11f191d77", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f5c9401e640 con 0x7f5c9c06c5b0 2024-03-29T04:17:13.865 INFO:teuthology.orchestra.run.smithi151.stdout:7 2024-03-29T04:17:13.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.887+0000 7f5ca0883640 1 -- 172.21.15.151:0/4220434197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c9c06c5b0 msgr2=0x7f5c9c168010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:13.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.887+0000 7f5ca0883640 1 --2- 172.21.15.151:0/4220434197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c9c06c5b0 0x7f5c9c168010 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5c94007e40 tx=0x7f5c9400e4b0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.887+0000 7f5ca0883640 1 -- 172.21.15.151:0/4220434197 shutdown_connections 2024-03-29T04:17:13.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.887+0000 7f5ca0883640 1 --2- 172.21.15.151:0/4220434197 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5c9c147e00 0x7f5c9c160a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.887+0000 7f5ca0883640 1 --2- 172.21.15.151:0/4220434197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c9c0b4f30 0x7f5c9c146c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.887+0000 7f5ca0883640 1 --2- 172.21.15.151:0/4220434197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c9c06c5b0 0x7f5c9c168010 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:13.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.887+0000 7f5ca0883640 1 -- 172.21.15.151:0/4220434197 >> 172.21.15.151:0/4220434197 conn(0x7f5c9c0d1070 msgr2=0x7f5c9c0d1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:13.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.887+0000 7f5ca0883640 1 -- 172.21.15.151:0/4220434197 shutdown_connections 2024-03-29T04:17:13.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:17:13.887+0000 7f5ca0883640 1 -- 172.21.15.151:0/4220434197 wait complete. 2024-03-29T04:17:13.883 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-29T04:17:13.884 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-29T04:17:13.886 INFO:tasks.ceph.osd.0:Started 2024-03-29T04:17:13.886 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-29T04:17:13.886 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-29T04:17:13.891 INFO:tasks.ceph.osd.1:Started 2024-03-29T04:17:13.891 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-29T04:17:13.891 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-29T04:17:13.892 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:13.908+0000 7f9fee2c8180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-29T04:17:13.894 INFO:tasks.ceph.osd.2:Started 2024-03-29T04:17:13.894 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-29T04:17:13.894 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-29T04:17:13.898 INFO:tasks.ceph.osd.3:Started 2024-03-29T04:17:13.899 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-29T04:17:13.899 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-29T04:17:13.911 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:13.931+0000 7f559409a180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-29T04:17:13.912 INFO:tasks.ceph.osd.4:Started 2024-03-29T04:17:13.912 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-29T04:17:13.913 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-29T04:17:13.914 INFO:tasks.ceph.osd.5:Started 2024-03-29T04:17:13.915 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-29T04:17:13.915 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-29T04:17:13.917 INFO:tasks.ceph.osd.6:Started 2024-03-29T04:17:13.917 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-29T04:17:13.917 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-29T04:17:13.919 INFO:tasks.ceph.osd.7:Started 2024-03-29T04:17:13.919 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:17:14.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.056+0000 7f4e35632640 1 Processor -- start 2024-03-29T04:17:14.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.056+0000 7f4e35632640 1 -- start start 2024-03-29T04:17:14.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e35632640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4e3006b660 0x7f4e30161fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:14.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e35632640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e3006bc40 con 0x7f4e30147e80 2024-03-29T04:17:14.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e35632640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e30070600 con 0x7f4e30162500 2024-03-29T04:17:14.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e35632640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4e30070c50 con 0x7f4e3006b660 2024-03-29T04:17:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e2effd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f4e30147e80 0x7f4e3006c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51594/0 (socket says 172.21.15.22:51594) 2024-03-29T04:17:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e2effd640 1 -- 172.21.15.22:0/1910877108 learned_addr learned my addr 172.21.15.22:0/1910877108 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e267fc640 1 --2- 172.21.15.22:0/1910877108 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4e3006b660 0x7f4e30161fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e2e7fc640 1 -- 172.21.15.22:0/1910877108 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2349745441 0 0) 0x7f4e3006bc40 con 0x7f4e30147e80 2024-03-29T04:17:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e2e7fc640 1 -- 172.21.15.22:0/1910877108 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e1c003530 con 0x7f4e30147e80 2024-03-29T04:17:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e2e7fc640 1 -- 172.21.15.22:0/1910877108 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1392328564 0 0) 0x7f4e1c003530 con 0x7f4e30147e80 2024-03-29T04:17:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e2e7fc640 1 -- 172.21.15.22:0/1910877108 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4e3006b660 msgr2=0x7f4e30161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e2e7fc640 1 --2- 172.21.15.22:0/1910877108 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4e3006b660 0x7f4e30161fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e2e7fc640 1 -- 172.21.15.22:0/1910877108 >> v1:172.21.15.151:6789/0 conn(0x7f4e30162500 legacy=0x7f4e3015fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e2e7fc640 1 -- 172.21.15.22:0/1910877108 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e300708e0 con 0x7f4e30147e80 2024-03-29T04:17:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e2e7fc640 1 -- 172.21.15.22:0/1910877108 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f4e20001680 con 0x7f4e30147e80 2024-03-29T04:17:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e2e7fc640 1 -- 172.21.15.22:0/1910877108 >> v1:172.21.15.22:6789/0 conn(0x7f4e30147e80 legacy=0x7f4e3006c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e2e7fc640 1 --2- 172.21.15.22:0/1910877108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e1c0039f0 0x7f4e1c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e2e7fc640 1 --2- 172.21.15.22:0/1910877108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c0043a0 0x7f4e1c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e2e7fc640 1 --2- 172.21.15.22:0/1910877108 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4e3006b660 0x7f4e1c0050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e2e7fc640 1 -- 172.21.15.22:0/1910877108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4e30070c50 con 0x7f4e1c0043a0 2024-03-29T04:17:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e2e7fc640 1 -- 172.21.15.22:0/1910877108 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c00b9d0 con 0x7f4e1c0039f0 2024-03-29T04:17:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e2e7fc640 1 -- 172.21.15.22:0/1910877108 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4e1c00bbd0 con 0x7f4e3006b660 2024-03-29T04:17:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e267fc640 1 --2- 172.21.15.22:0/1910877108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c0043a0 0x7f4e1c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e2effd640 1 --2- 172.21.15.22:0/1910877108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e1c0039f0 0x7f4e1c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e2f7fe640 1 --2- 172.21.15.22:0/1910877108 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4e3006b660 0x7f4e1c0050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e267fc640 1 -- 172.21.15.22:0/1910877108 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4e3006b660 msgr2=0x7f4e1c0050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e267fc640 1 --2- 172.21.15.22:0/1910877108 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4e3006b660 0x7f4e1c0050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e267fc640 1 -- 172.21.15.22:0/1910877108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e1c0039f0 msgr2=0x7f4e1c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e267fc640 1 --2- 172.21.15.22:0/1910877108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e1c0039f0 0x7f4e1c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e267fc640 1 -- 172.21.15.22:0/1910877108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1c003530 con 0x7f4e1c0043a0 2024-03-29T04:17:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.060+0000 7f4e2f7fe640 1 --2- 172.21.15.22:0/1910877108 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4e3006b660 0x7f4e1c0050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:17:14.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e267fc640 1 --2- 172.21.15.22:0/1910877108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c0043a0 0x7f4e1c004870 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009ae0 tx=0x7f4e1801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b269965f9a7d693 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:14.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e2e7fc640 1 -- 172.21.15.22:0/1910877108 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e180b2070 con 0x7f4e1c0043a0 2024-03-29T04:17:14.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e2e7fc640 1 -- 172.21.15.22:0/1910877108 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e180a5660 con 0x7f4e1c0043a0 2024-03-29T04:17:14.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e2e7fc640 1 -- 172.21.15.22:0/1910877108 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e180096b0 con 0x7f4e1c0043a0 2024-03-29T04:17:14.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e35632640 1 -- 172.21.15.22:0/1910877108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c0043a0 msgr2=0x7f4e1c004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:14.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e35632640 1 --2- 172.21.15.22:0/1910877108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c0043a0 0x7f4e1c004870 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009ae0 tx=0x7f4e1801ca70 comp rx=0 tx=0).stop 2024-03-29T04:17:14.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e2f7fe640 1 -- 172.21.15.22:0/1910877108 reap_dead start 2024-03-29T04:17:14.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e35632640 1 -- 172.21.15.22:0/1910877108 shutdown_connections 2024-03-29T04:17:14.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e35632640 1 -- 172.21.15.22:0/1910877108 >> 172.21.15.22:0/1910877108 conn(0x7f4e3006e410 msgr2=0x7f4e3006e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:14.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e35632640 1 -- 172.21.15.22:0/1910877108 shutdown_connections 2024-03-29T04:17:14.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e35632640 1 -- 172.21.15.22:0/1910877108 wait complete. 2024-03-29T04:17:14.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e35632640 1 Processor -- start 2024-03-29T04:17:14.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e35632640 1 -- start start 2024-03-29T04:17:14.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e35632640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e1c0043a0 0x7f4e3015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:14.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e35632640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4e300ce0f0 0x7f4e301f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:14.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e35632640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e300afe80 0x7f4e300ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:14.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e35632640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4e30162350 con 0x7f4e300afe80 2024-03-29T04:17:14.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e35632640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4e301621d0 con 0x7f4e1c0043a0 2024-03-29T04:17:14.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e35632640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4e30160230 con 0x7f4e300ce0f0 2024-03-29T04:17:14.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e2f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e300afe80 0x7f4e300ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:14.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e2f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e300afe80 0x7f4e300ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55446/0 (socket says 172.21.15.22:55446) 2024-03-29T04:17:14.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e2f7fe640 1 -- 172.21.15.22:0/1189265199 learned_addr learned my addr 172.21.15.22:0/1189265199 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:14.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e267fc640 1 --2- 172.21.15.22:0/1189265199 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4e300ce0f0 0x7f4e301f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:14.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e2effd640 1 --2- 172.21.15.22:0/1189265199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e1c0043a0 0x7f4e3015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:14.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e2f7fe640 1 -- 172.21.15.22:0/1189265199 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4e300ce0f0 msgr2=0x7f4e301f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:14.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e2f7fe640 1 --2- 172.21.15.22:0/1189265199 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4e300ce0f0 0x7f4e301f51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:14.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e2f7fe640 1 -- 172.21.15.22:0/1189265199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e1c0043a0 msgr2=0x7f4e3015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:14.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e2f7fe640 1 --2- 172.21.15.22:0/1189265199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e1c0043a0 0x7f4e3015dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:14.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e2f7fe640 1 -- 172.21.15.22:0/1189265199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e300adaf0 con 0x7f4e300afe80 2024-03-29T04:17:14.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e2effd640 1 --2- 172.21.15.22:0/1189265199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e1c0043a0 0x7f4e3015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:14.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e2f7fe640 1 --2- 172.21.15.22:0/1189265199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e300afe80 0x7f4e300ceec0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4e3006f4c0 tx=0x7f4e28010e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:14.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e27fff640 1 -- 172.21.15.22:0/1189265199 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28003d90 con 0x7f4e300afe80 2024-03-29T04:17:14.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e27fff640 1 -- 172.21.15.22:0/1189265199 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e28019070 con 0x7f4e300afe80 2024-03-29T04:17:14.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e35632640 1 -- 172.21.15.22:0/1189265199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e300addd0 con 0x7f4e300afe80 2024-03-29T04:17:14.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e27fff640 1 -- 172.21.15.22:0/1189265199 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2800ea10 con 0x7f4e300afe80 2024-03-29T04:17:14.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e35632640 1 -- 172.21.15.22:0/1189265199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e301f56f0 con 0x7f4e300afe80 2024-03-29T04:17:14.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e27fff640 1 -- 172.21.15.22:0/1189265199 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4e28013070 con 0x7f4e300afe80 2024-03-29T04:17:14.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.064+0000 7f4e27fff640 1 -- 172.21.15.22:0/1189265199 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f4e2801eb30 con 0x7f4e300afe80 2024-03-29T04:17:14.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.068+0000 7f4e35632640 1 -- 172.21.15.22:0/1189265199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e30068890 con 0x7f4e300afe80 2024-03-29T04:17:14.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.068+0000 7f4e27fff640 1 -- 172.21.15.22:0/1189265199 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4e2800ebb0 con 0x7f4e300afe80 2024-03-29T04:17:14.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.124+0000 7f4e35632640 1 -- 172.21.15.22:0/1189265199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e30142d00 con 0x7f4e300afe80 2024-03-29T04:17:14.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.128+0000 7f4e27fff640 1 -- 172.21.15.22:0/1189265199 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f4e2800ed90 con 0x7f4e300afe80 2024-03-29T04:17:14.108 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:14.108 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":10,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:17:13.882337+0000","last_up_change":"0.000000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:17:14.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.128+0000 7f4e35632640 1 -- 172.21.15.22:0/1189265199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e300afe80 msgr2=0x7f4e300ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:14.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.128+0000 7f4e35632640 1 --2- 172.21.15.22:0/1189265199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e300afe80 0x7f4e300ceec0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4e3006f4c0 tx=0x7f4e28010e70 comp rx=0 tx=0).stop 2024-03-29T04:17:14.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.128+0000 7f4e35632640 1 -- 172.21.15.22:0/1189265199 shutdown_connections 2024-03-29T04:17:14.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.128+0000 7f4e35632640 1 --2- 172.21.15.22:0/1189265199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e300afe80 0x7f4e300ceec0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:14.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.128+0000 7f4e35632640 1 --2- 172.21.15.22:0/1189265199 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4e300ce0f0 0x7f4e301f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:14.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.128+0000 7f4e35632640 1 --2- 172.21.15.22:0/1189265199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e1c0043a0 0x7f4e3015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:14.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.128+0000 7f4e35632640 1 -- 172.21.15.22:0/1189265199 >> 172.21.15.22:0/1189265199 conn(0x7f4e3006e410 msgr2=0x7f4e3006f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:14.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.128+0000 7f4e35632640 1 -- 172.21.15.22:0/1189265199 shutdown_connections 2024-03-29T04:17:14.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:14.128+0000 7f4e35632640 1 -- 172.21.15.22:0/1189265199 wait complete. 2024-03-29T04:17:14.115 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:14.135+0000 7f559409a180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-29T04:17:14.125 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-29T04:17:14.125 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-29T04:17:14.163 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:14.180+0000 7f9fee2c8180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-29T04:17:14.185 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:14.203+0000 7f559409a180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-29T04:17:14.230 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:14.248+0000 7f9fee2c8180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-29T04:17:14.332 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:14.351+0000 7f559409a180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-29T04:17:14.371 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:14.388+0000 7f9fee2c8180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-29T04:17:14.528 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-03-29T04:17:14.547+0000 7f93fed077c0 -1 Falling back to public interface 2024-03-29T04:17:14.528 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-03-29T04:17:14.547+0000 7f981d9747c0 -1 Falling back to public interface 2024-03-29T04:17:14.529 INFO:tasks.ceph.osd.1.smithi022.stderr:2024-03-29T04:17:14.548+0000 7f315f7407c0 -1 Falling back to public interface 2024-03-29T04:17:14.532 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-03-29T04:17:14.551+0000 7f94189977c0 -1 Falling back to public interface 2024-03-29T04:17:14.537 INFO:tasks.ceph.osd.2.smithi022.stderr:2024-03-29T04:17:14.556+0000 7ff1f1fe77c0 -1 Falling back to public interface 2024-03-29T04:17:14.537 INFO:tasks.ceph.osd.0.smithi022.stderr:2024-03-29T04:17:14.556+0000 7ff559bd67c0 -1 Falling back to public interface 2024-03-29T04:17:14.538 INFO:tasks.ceph.osd.3.smithi022.stderr:2024-03-29T04:17:14.556+0000 7fec1024f7c0 -1 Falling back to public interface 2024-03-29T04:17:14.539 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-03-29T04:17:14.559+0000 7f062e4427c0 -1 Falling back to public interface 2024-03-29T04:17:14.930 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-29T04:17:14.955 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:14.975+0000 7f559409a180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-29T04:17:15.023 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:15.040+0000 7f9fee2c8180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-29T04:17:15.029 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:15.047+0000 7f559409a180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-29T04:17:15.044 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16ec7be640 1 Processor -- start 2024-03-29T04:17:15.045 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16ec7be640 1 -- start start 2024-03-29T04:17:15.047 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16ec7be640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16e40b5f80 0x7f16e40b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:15.047 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16ec7be640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16e40cfc30 con 0x7f16e41454e0 2024-03-29T04:17:15.047 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16ec7be640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16e40cff10 con 0x7f16e4144b60 2024-03-29T04:17:15.047 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16ec7be640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f16e40b6900 con 0x7f16e40b5f80 2024-03-29T04:17:15.047 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16ead5b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f16e41454e0 0x7f16e4145970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:46962/0 (socket says 172.21.15.22:46962) 2024-03-29T04:17:15.047 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16ead5b640 1 -- 172.21.15.22:0/1177602828 learned_addr learned my addr 172.21.15.22:0/1177602828 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:15.047 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16e9d59640 1 --2- 172.21.15.22:0/1177602828 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16e40b5f80 0x7f16e40b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:15.048 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16e9558640 1 -- 172.21.15.22:0/1177602828 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3133676159 0 0) 0x7f16e40cfc30 con 0x7f16e41454e0 2024-03-29T04:17:15.048 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16e9558640 1 -- 172.21.15.22:0/1177602828 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16d0003530 con 0x7f16e41454e0 2024-03-29T04:17:15.048 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16e9558640 1 -- 172.21.15.22:0/1177602828 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3625223487 0 0) 0x7f16d0003530 con 0x7f16e41454e0 2024-03-29T04:17:15.048 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16e9558640 1 -- 172.21.15.22:0/1177602828 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16e40b5f80 msgr2=0x7f16e40b63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:15.048 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16e9558640 1 --2- 172.21.15.22:0/1177602828 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16e40b5f80 0x7f16e40b63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:15.048 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16e9558640 1 -- 172.21.15.22:0/1177602828 >> v1:172.21.15.151:6789/0 conn(0x7f16e4144b60 legacy=0x7f16e40b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:15.048 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16e9558640 1 -- 172.21.15.22:0/1177602828 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e40d01f0 con 0x7f16e41454e0 2024-03-29T04:17:15.048 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16e9558640 1 -- 172.21.15.22:0/1177602828 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f16dc001a00 con 0x7f16e41454e0 2024-03-29T04:17:15.048 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16e9558640 1 -- 172.21.15.22:0/1177602828 >> v1:172.21.15.22:6789/0 conn(0x7f16e41454e0 legacy=0x7f16e4145970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:15.048 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16e9558640 1 --2- 172.21.15.22:0/1177602828 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16e40b5f80 0x7f16d00039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:15.048 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16e9558640 1 --2- 172.21.15.22:0/1177602828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16d0004020 0x7f16d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:15.048 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16e9558640 1 --2- 172.21.15.22:0/1177602828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16d0004d50 0x7f16d00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:15.048 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16e9558640 1 -- 172.21.15.22:0/1177602828 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f16e40b6900 con 0x7f16d0004d50 2024-03-29T04:17:15.048 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16e9558640 1 -- 172.21.15.22:0/1177602828 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f16d000bc20 con 0x7f16d0004020 2024-03-29T04:17:15.049 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16e9558640 1 -- 172.21.15.22:0/1177602828 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f16d000be20 con 0x7f16e40b5f80 2024-03-29T04:17:15.049 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16ead5b640 1 --2- 172.21.15.22:0/1177602828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16d0004d50 0x7f16d00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:15.049 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16ea55a640 1 --2- 172.21.15.22:0/1177602828 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16e40b5f80 0x7f16d00039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:15.049 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16e9d59640 1 --2- 172.21.15.22:0/1177602828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16d0004020 0x7f16d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:15.049 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16ead5b640 1 -- 172.21.15.22:0/1177602828 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16e40b5f80 msgr2=0x7f16d00039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:15.049 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16ead5b640 1 --2- 172.21.15.22:0/1177602828 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16e40b5f80 0x7f16d00039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:15.049 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16ead5b640 1 -- 172.21.15.22:0/1177602828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16d0004020 msgr2=0x7f16d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:15.049 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16ead5b640 1 --2- 172.21.15.22:0/1177602828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16d0004020 0x7f16d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:15.049 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16ead5b640 1 -- 172.21.15.22:0/1177602828 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d0003530 con 0x7f16d0004d50 2024-03-29T04:17:15.049 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16ea55a640 1 --2- 172.21.15.22:0/1177602828 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16e40b5f80 0x7f16d00039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:15.049 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.064+0000 7f16e9d59640 1 --2- 172.21.15.22:0/1177602828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16d0004020 0x7f16d0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:15.050 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16ead5b640 1 --2- 172.21.15.22:0/1177602828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16d0004d50 0x7f16d00051d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00a840 tx=0x7f16dc01d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80857305f0715f76 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:15.050 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16e9558640 1 -- 172.21.15.22:0/1177602828 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc00a460 con 0x7f16d0004d50 2024-03-29T04:17:15.051 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16e9558640 1 -- 172.21.15.22:0/1177602828 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16dc0ad650 con 0x7f16d0004d50 2024-03-29T04:17:15.051 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16e9558640 1 -- 172.21.15.22:0/1177602828 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc0ada80 con 0x7f16d0004d50 2024-03-29T04:17:15.051 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16ec7be640 1 -- 172.21.15.22:0/1177602828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16d0004d50 msgr2=0x7f16d00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:15.051 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16ec7be640 1 --2- 172.21.15.22:0/1177602828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16d0004d50 0x7f16d00051d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00a840 tx=0x7f16dc01d360 comp rx=0 tx=0).stop 2024-03-29T04:17:15.051 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16ead5b640 1 -- 172.21.15.22:0/1177602828 reap_dead start 2024-03-29T04:17:15.051 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16ec7be640 1 -- 172.21.15.22:0/1177602828 shutdown_connections 2024-03-29T04:17:15.051 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16ec7be640 1 -- 172.21.15.22:0/1177602828 >> 172.21.15.22:0/1177602828 conn(0x7f16e40cda60 msgr2=0x7f16e40cde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:15.051 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16ec7be640 1 -- 172.21.15.22:0/1177602828 shutdown_connections 2024-03-29T04:17:15.051 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16ec7be640 1 -- 172.21.15.22:0/1177602828 wait complete. 2024-03-29T04:17:15.051 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16ec7be640 1 Processor -- start 2024-03-29T04:17:15.051 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16ec7be640 1 -- start start 2024-03-29T04:17:15.053 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16ec7be640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16d0004d50 0x7f16e413d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:15.053 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16ec7be640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16e413dd70 0x7f16e413fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:15.053 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16ec7be640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16e413ffd0 0x7f16e41f4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:15.053 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16ec7be640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f16e40b58c0 con 0x7f16e413ffd0 2024-03-29T04:17:15.054 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16ec7be640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f16e4144f40 con 0x7f16d0004d50 2024-03-29T04:17:15.054 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16ec7be640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f16e40cf890 con 0x7f16e413dd70 2024-03-29T04:17:15.054 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16ead5b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16e413ffd0 0x7f16e41f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:15.054 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16ead5b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16e413ffd0 0x7f16e41f4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60858/0 (socket says 172.21.15.22:60858) 2024-03-29T04:17:15.054 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16ead5b640 1 -- 172.21.15.22:0/1626743894 learned_addr learned my addr 172.21.15.22:0/1626743894 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:15.054 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16e9d59640 1 --2- 172.21.15.22:0/1626743894 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16e413dd70 0x7f16e413fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:15.054 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.068+0000 7f16ea55a640 1 --2- 172.21.15.22:0/1626743894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16d0004d50 0x7f16e413d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:15.054 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.072+0000 7f16ead5b640 1 -- 172.21.15.22:0/1626743894 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16e413dd70 msgr2=0x7f16e413fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:15.054 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.072+0000 7f16ead5b640 1 --2- 172.21.15.22:0/1626743894 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16e413dd70 0x7f16e413fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:15.054 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.072+0000 7f16ead5b640 1 -- 172.21.15.22:0/1626743894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16d0004d50 msgr2=0x7f16e413d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:15.054 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.072+0000 7f16ead5b640 1 --2- 172.21.15.22:0/1626743894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16d0004d50 0x7f16e413d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:15.054 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.072+0000 7f16ead5b640 1 -- 172.21.15.22:0/1626743894 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e4140550 con 0x7f16e413ffd0 2024-03-29T04:17:15.054 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.072+0000 7f16ea55a640 1 --2- 172.21.15.22:0/1626743894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16d0004d50 0x7f16e413d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:15.054 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.072+0000 7f16ead5b640 1 --2- 172.21.15.22:0/1626743894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16e413ffd0 0x7f16e41f4400 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f16dc01d190 tx=0x7f16d0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:15.055 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.072+0000 7f16daffd640 1 -- 172.21.15.22:0/1626743894 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc0bc070 con 0x7f16e413ffd0 2024-03-29T04:17:15.055 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.072+0000 7f16daffd640 1 -- 172.21.15.22:0/1626743894 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16dc0ad430 con 0x7f16e413ffd0 2024-03-29T04:17:15.055 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.072+0000 7f16ec7be640 1 -- 172.21.15.22:0/1626743894 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e4140830 con 0x7f16e413ffd0 2024-03-29T04:17:15.055 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.072+0000 7f16daffd640 1 -- 172.21.15.22:0/1626743894 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc001fc0 con 0x7f16e413ffd0 2024-03-29T04:17:15.055 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.072+0000 7f16ec7be640 1 -- 172.21.15.22:0/1626743894 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e4140df0 con 0x7f16e413ffd0 2024-03-29T04:17:15.055 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.072+0000 7f16daffd640 1 -- 172.21.15.22:0/1626743894 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f16dc0bf070 con 0x7f16e413ffd0 2024-03-29T04:17:15.055 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.072+0000 7f16daffd640 1 -- 172.21.15.22:0/1626743894 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f16dc0b7b30 con 0x7f16e413ffd0 2024-03-29T04:17:15.055 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.072+0000 7f16ec7be640 1 -- 172.21.15.22:0/1626743894 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e4068fa0 con 0x7f16e413ffd0 2024-03-29T04:17:15.055 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.072+0000 7f16daffd640 1 -- 172.21.15.22:0/1626743894 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f16dc0c6070 con 0x7f16e413ffd0 2024-03-29T04:17:15.104 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:15.124+0000 7f9fee2c8180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-29T04:17:15.113 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.132+0000 7f16ec7be640 1 -- 172.21.15.22:0/1626743894 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16e4161c10 con 0x7f16e413ffd0 2024-03-29T04:17:15.114 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.132+0000 7f16daffd640 1 -- 172.21.15.22:0/1626743894 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f16dc017070 con 0x7f16e413ffd0 2024-03-29T04:17:15.114 INFO:teuthology.misc.health.smithi022.stdout: 2024-03-29T04:17:15.114 INFO:teuthology.misc.health.smithi022.stdout:{"epoch":10,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:17:13.882337+0000","last_up_change":"0.000000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:17:15.115 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.132+0000 7f16ec7be640 1 -- 172.21.15.22:0/1626743894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16e413ffd0 msgr2=0x7f16e41f4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:15.116 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.132+0000 7f16ec7be640 1 --2- 172.21.15.22:0/1626743894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16e413ffd0 0x7f16e41f4400 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f16dc01d190 tx=0x7f16d0005840 comp rx=0 tx=0).stop 2024-03-29T04:17:15.116 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.132+0000 7f16ec7be640 1 -- 172.21.15.22:0/1626743894 shutdown_connections 2024-03-29T04:17:15.116 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.132+0000 7f16ec7be640 1 --2- 172.21.15.22:0/1626743894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16e413ffd0 0x7f16e41f4400 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:15.116 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.132+0000 7f16ec7be640 1 --2- 172.21.15.22:0/1626743894 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f16e413dd70 0x7f16e413fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:15.116 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.132+0000 7f16ec7be640 1 --2- 172.21.15.22:0/1626743894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16d0004d50 0x7f16e413d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:15.116 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.132+0000 7f16ec7be640 1 -- 172.21.15.22:0/1626743894 >> 172.21.15.22:0/1626743894 conn(0x7f16e40cda60 msgr2=0x7f16e4146090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:15.116 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.132+0000 7f16ec7be640 1 -- 172.21.15.22:0/1626743894 shutdown_connections 2024-03-29T04:17:15.116 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:15.132+0000 7f16ec7be640 1 -- 172.21.15.22:0/1626743894 wait complete. 2024-03-29T04:17:15.130 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-29T04:17:15.211 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:15.231+0000 7f559409a180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-29T04:17:15.284 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:15.303+0000 7f559409a180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-29T04:17:15.297 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:15.316+0000 7f9fee2c8180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-29T04:17:15.373 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:15.392+0000 7f9fee2c8180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-29T04:17:15.573 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:15.591+0000 7f559409a180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-29T04:17:15.633 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:15.651+0000 7f559409a180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-29T04:17:15.680 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:15.700+0000 7f9fee2c8180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-29T04:17:15.746 INFO:tasks.ceph.mgr.y.smithi022.stderr:2024-03-29T04:17:15.764+0000 7f9fee2c8180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-29T04:17:16.676 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:16.695+0000 7f553b4e1640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-29T04:17:16.676 INFO:tasks.ceph.mgr.x.smithi151.stderr:2024-03-29T04:17:16.695+0000 7f553b4e1640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-29T04:17:18.183 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-03-29T04:17:18.203+0000 7f94189977c0 -1 osd.4 0 log_to_monitors true 2024-03-29T04:17:18.310 INFO:tasks.ceph.osd.3.smithi022.stderr:2024-03-29T04:17:18.328+0000 7fec1024f7c0 -1 osd.3 0 log_to_monitors true 2024-03-29T04:17:18.325 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-03-29T04:17:18.343+0000 7f062e4427c0 -1 osd.6 0 log_to_monitors true 2024-03-29T04:17:18.406 INFO:tasks.ceph.osd.0.smithi022.stderr:2024-03-29T04:17:18.424+0000 7ff559bd67c0 -1 osd.0 0 log_to_monitors true 2024-03-29T04:17:18.421 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-03-29T04:17:18.439+0000 7f93fed077c0 -1 osd.5 0 log_to_monitors true 2024-03-29T04:17:18.434 INFO:tasks.ceph.osd.1.smithi022.stderr:2024-03-29T04:17:18.452+0000 7f315f7407c0 -1 osd.1 0 log_to_monitors true 2024-03-29T04:17:18.446 INFO:tasks.ceph.osd.2.smithi022.stderr:2024-03-29T04:17:18.464+0000 7ff1f1fe77c0 -1 osd.2 0 log_to_monitors true 2024-03-29T04:17:18.493 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-03-29T04:17:18.511+0000 7f981d9747c0 -1 osd.7 0 log_to_monitors true 2024-03-29T04:17:21.292 INFO:tasks.ceph.osd.1.smithi022.stderr:2024-03-29T04:17:21.308+0000 7f315b6e9640 -1 osd.1 0 waiting for initial osdmap 2024-03-29T04:17:21.304 INFO:tasks.ceph.osd.0.smithi022.stderr:2024-03-29T04:17:21.324+0000 7ff555b7f640 -1 osd.0 0 waiting for initial osdmap 2024-03-29T04:17:21.312 INFO:tasks.ceph.osd.1.smithi022.stderr:2024-03-29T04:17:21.332+0000 7f315351d640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:17:21.332 INFO:tasks.ceph.osd.0.smithi022.stderr:2024-03-29T04:17:21.352+0000 7ff54d9b3640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:17:21.341 INFO:tasks.ceph.osd.2.smithi022.stderr:2024-03-29T04:17:21.360+0000 7ff1edf90640 -1 osd.2 0 waiting for initial osdmap 2024-03-29T04:17:21.346 INFO:tasks.ceph.osd.3.smithi022.stderr:2024-03-29T04:17:21.364+0000 7fec0c1f8640 -1 osd.3 0 waiting for initial osdmap 2024-03-29T04:17:21.372 INFO:tasks.ceph.osd.2.smithi022.stderr:2024-03-29T04:17:21.392+0000 7ff1e5dc4640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:17:21.380 INFO:tasks.ceph.osd.3.smithi022.stderr:2024-03-29T04:17:21.400+0000 7fec0402c640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:17:21.910 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-03-29T04:17:21.927+0000 7f062a3eb640 -1 osd.6 0 waiting for initial osdmap 2024-03-29T04:17:21.935 INFO:tasks.ceph.osd.6.smithi151.stderr:2024-03-29T04:17:21.955+0000 7f062221f640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:17:21.937 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-29T04:17:21.959 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-03-29T04:17:21.979+0000 7f9414940640 -1 osd.4 0 waiting for initial osdmap 2024-03-29T04:17:21.991 INFO:tasks.ceph.osd.4.smithi151.stderr:2024-03-29T04:17:22.011+0000 7f940c774640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:17:22.006 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-03-29T04:17:22.027+0000 7f93fb4c2640 -1 osd.5 0 waiting for initial osdmap 2024-03-29T04:17:22.029 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-03-29T04:17:22.047+0000 7f981991d640 -1 osd.7 0 waiting for initial osdmap 2024-03-29T04:17:22.039 INFO:tasks.ceph.osd.5.smithi151.stderr:2024-03-29T04:17:22.059+0000 7f93f2ae4640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:17:22.051 INFO:tasks.ceph.osd.7.smithi151.stderr:2024-03-29T04:17:22.071+0000 7f9811751640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:17:22.064 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.084+0000 7fbc20fca640 1 Processor -- start 2024-03-29T04:17:22.064 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.084+0000 7fbc20fca640 1 -- start start 2024-03-29T04:17:22.066 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.084+0000 7fbc20fca640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fbc1c1583d0 0x7fbc1c06c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:22.066 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.084+0000 7fbc20fca640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fbc1c1587b0 0x7fbc1c0ad740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:22.066 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.084+0000 7fbc20fca640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc1c15bc40 con 0x7fbc1c0adc80 2024-03-29T04:17:22.066 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.084+0000 7fbc20fca640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fbc1c0cff00 con 0x7fbc1c1583d0 2024-03-29T04:17:22.066 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.084+0000 7fbc20fca640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fbc1c15bf20 con 0x7fbc1c1587b0 2024-03-29T04:17:22.066 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.084+0000 7fbc1a59c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fbc1c1583d0 0x7fbc1c06c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:22.066 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.084+0000 7fbc1a59c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fbc1c1583d0 0x7fbc1c06c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60912/0 (socket says 172.21.15.22:60912) 2024-03-29T04:17:22.067 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.084+0000 7fbc1a59c640 1 -- 172.21.15.22:0/3833953013 learned_addr learned my addr 172.21.15.22:0/3833953013 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:22.067 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.084+0000 7fbc19d9b640 1 --2- 172.21.15.22:0/3833953013 >> v2:172.21.15.151:3300/0 conn(0x7fbc1c1587b0 0x7fbc1c0ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:22.067 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.084+0000 7fbc1a59c640 1 -- 172.21.15.22:0/3833953013 >> v2:172.21.15.151:3300/0 conn(0x7fbc1c1587b0 msgr2=0x7fbc1c0ad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:22.067 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.084+0000 7fbc1a59c640 1 --2- 172.21.15.22:0/3833953013 >> v2:172.21.15.151:3300/0 conn(0x7fbc1c1587b0 0x7fbc1c0ad740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:22.067 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.084+0000 7fbc1a59c640 1 -- 172.21.15.22:0/3833953013 >> v1:172.21.15.151:6789/0 conn(0x7fbc1c0adc80 legacy=0x7fbc1c15b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:22.067 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.084+0000 7fbc1a59c640 1 -- 172.21.15.22:0/3833953013 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1c06b6b0 con 0x7fbc1c1583d0 2024-03-29T04:17:22.067 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.084+0000 7fbc19d9b640 1 --2- 172.21.15.22:0/3833953013 >> v2:172.21.15.151:3300/0 conn(0x7fbc1c1587b0 0x7fbc1c0ad740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:17:22.068 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc1a59c640 1 --2- 172.21.15.22:0/3833953013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc1c1583d0 0x7fbc1c06c190 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0ae470 tx=0x7fbc0c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705dadb66afa55c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:22.068 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc1959a640 1 -- 172.21.15.22:0/3833953013 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c0ab610 con 0x7fbc1c1583d0 2024-03-29T04:17:22.068 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc1959a640 1 -- 172.21.15.22:0/3833953013 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc0c0b0070 con 0x7fbc1c1583d0 2024-03-29T04:17:22.068 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc1959a640 1 -- 172.21.15.22:0/3833953013 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c0ab9d0 con 0x7fbc1c1583d0 2024-03-29T04:17:22.070 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc20fca640 1 -- 172.21.15.22:0/3833953013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc1c1583d0 msgr2=0x7fbc1c06c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:22.070 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc20fca640 1 --2- 172.21.15.22:0/3833953013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc1c1583d0 0x7fbc1c06c190 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0ae470 tx=0x7fbc0c01c520 comp rx=0 tx=0).stop 2024-03-29T04:17:22.070 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc20fca640 1 -- 172.21.15.22:0/3833953013 shutdown_connections 2024-03-29T04:17:22.070 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc20fca640 1 --2- 172.21.15.22:0/3833953013 >> v2:172.21.15.151:3300/0 conn(0x7fbc1c1587b0 0x7fbc1c0ad740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:22.070 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc20fca640 1 --2- 172.21.15.22:0/3833953013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc1c1583d0 0x7fbc1c06c190 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:22.070 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc20fca640 1 -- 172.21.15.22:0/3833953013 >> 172.21.15.22:0/3833953013 conn(0x7fbc1c0d0100 msgr2=0x7fbc1c0d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:22.070 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc20fca640 1 -- 172.21.15.22:0/3833953013 shutdown_connections 2024-03-29T04:17:22.070 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc20fca640 1 -- 172.21.15.22:0/3833953013 wait complete. 2024-03-29T04:17:22.070 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc20fca640 1 Processor -- start 2024-03-29T04:17:22.070 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc20fca640 1 -- start start 2024-03-29T04:17:22.072 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc20fca640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc1c0adc80 0x7fbc1c0c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:22.073 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc20fca640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc1c1583d0 0x7fbc1c0c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:22.073 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc20fca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc1c1587b0 0x7fbc1c0c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:22.073 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc20fca640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c0cb0e0 con 0x7fbc1c1587b0 2024-03-29T04:17:22.073 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc20fca640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c0caf60 con 0x7fbc1c0adc80 2024-03-29T04:17:22.073 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc20fca640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1c0cb260 con 0x7fbc1c1583d0 2024-03-29T04:17:22.073 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc1ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc1c1587b0 0x7fbc1c0c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:22.073 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc1ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc1c1587b0 0x7fbc1c0c7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60928/0 (socket says 172.21.15.22:60928) 2024-03-29T04:17:22.073 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc1ad9d640 1 -- 172.21.15.22:0/814424935 learned_addr learned my addr 172.21.15.22:0/814424935 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:22.073 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc1a59c640 1 --2- 172.21.15.22:0/814424935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc1c0adc80 0x7fbc1c0c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:22.073 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc19d9b640 1 --2- 172.21.15.22:0/814424935 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc1c1583d0 0x7fbc1c0c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:22.073 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc1ad9d640 1 -- 172.21.15.22:0/814424935 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc1c1583d0 msgr2=0x7fbc1c0c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:22.073 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc1ad9d640 1 --2- 172.21.15.22:0/814424935 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc1c1583d0 0x7fbc1c0c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:22.073 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc1ad9d640 1 -- 172.21.15.22:0/814424935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc1c0adc80 msgr2=0x7fbc1c0c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:22.074 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc1ad9d640 1 --2- 172.21.15.22:0/814424935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc1c0adc80 0x7fbc1c0c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:22.074 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc1ad9d640 1 -- 172.21.15.22:0/814424935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1c0c3d60 con 0x7fbc1c1587b0 2024-03-29T04:17:22.074 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.088+0000 7fbc1ad9d640 1 --2- 172.21.15.22:0/814424935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc1c1587b0 0x7fbc1c0c7310 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400c730 tx=0x7fbc1400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:22.074 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.092+0000 7fbc0affd640 1 -- 172.21.15.22:0/814424935 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14003b80 con 0x7fbc1c1587b0 2024-03-29T04:17:22.074 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.092+0000 7fbc0affd640 1 -- 172.21.15.22:0/814424935 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc14003d20 con 0x7fbc1c1587b0 2024-03-29T04:17:22.074 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.092+0000 7fbc20fca640 1 -- 172.21.15.22:0/814424935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc1c0c4040 con 0x7fbc1c1587b0 2024-03-29T04:17:22.074 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.092+0000 7fbc0affd640 1 -- 172.21.15.22:0/814424935 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1400aa30 con 0x7fbc1c1587b0 2024-03-29T04:17:22.074 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.092+0000 7fbc20fca640 1 -- 172.21.15.22:0/814424935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc1c0c4600 con 0x7fbc1c1587b0 2024-03-29T04:17:22.074 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.092+0000 7fbc20fca640 1 -- 172.21.15.22:0/814424935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc1c068fa0 con 0x7fbc1c1587b0 2024-03-29T04:17:22.075 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.092+0000 7fbc0affd640 1 -- 172.21.15.22:0/814424935 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) ==== 87747+0+0 (secure 0 0 0) 0x7fbc1400abd0 con 0x7fbc1c1587b0 2024-03-29T04:17:22.075 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.092+0000 7fbc0affd640 1 --2- 172.21.15.22:0/814424935 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbbfc067f80 0x7fbbfc06a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:22.075 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.092+0000 7fbc0affd640 1 -- 172.21.15.22:0/814424935 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7fbc14085370 con 0x7fbc1c1587b0 2024-03-29T04:17:22.075 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.092+0000 7fbc1a59c640 1 --2- 172.21.15.22:0/814424935 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbbfc067f80 0x7fbbfc06a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:22.075 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.092+0000 7fbc1a59c640 1 --2- 172.21.15.22:0/814424935 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbbfc067f80 0x7fbbfc06a440 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0af000 tx=0x7fbc0c0a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:22.078 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.096+0000 7fbc0affd640 1 -- 172.21.15.22:0/814424935 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fbc140563b0 con 0x7fbc1c1587b0 2024-03-29T04:17:22.236 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.256+0000 7fbc20fca640 1 -- 172.21.15.22:0/814424935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc1c0cdd30 con 0x7fbc1c1587b0 2024-03-29T04:17:22.237 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.256+0000 7fbc0affd640 1 -- 172.21.15.22:0/814424935 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7fbc1405a6b0 con 0x7fbc1c1587b0 2024-03-29T04:17:22.237 INFO:teuthology.misc.health.smithi022.stdout: 2024-03-29T04:17:22.237 INFO:teuthology.misc.health.smithi022.stdout:{"epoch":12,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:17:19.699617+0000","last_up_change":"0.000000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:17:22.239 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.256+0000 7fbc20fca640 1 -- 172.21.15.22:0/814424935 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbbfc067f80 msgr2=0x7fbbfc06a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:22.239 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.256+0000 7fbc20fca640 1 --2- 172.21.15.22:0/814424935 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbbfc067f80 0x7fbbfc06a440 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0af000 tx=0x7fbc0c0a7000 comp rx=0 tx=0).stop 2024-03-29T04:17:22.239 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.256+0000 7fbc20fca640 1 -- 172.21.15.22:0/814424935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc1c1587b0 msgr2=0x7fbc1c0c7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:22.239 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.256+0000 7fbc20fca640 1 --2- 172.21.15.22:0/814424935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc1c1587b0 0x7fbc1c0c7310 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400c730 tx=0x7fbc1400cbf0 comp rx=0 tx=0).stop 2024-03-29T04:17:22.239 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.256+0000 7fbc20fca640 1 -- 172.21.15.22:0/814424935 shutdown_connections 2024-03-29T04:17:22.240 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.256+0000 7fbc20fca640 1 --2- 172.21.15.22:0/814424935 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbbfc067f80 0x7fbbfc06a440 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:22.240 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.256+0000 7fbc20fca640 1 --2- 172.21.15.22:0/814424935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc1c1587b0 0x7fbc1c0c7310 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:22.240 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.256+0000 7fbc20fca640 1 --2- 172.21.15.22:0/814424935 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc1c1583d0 0x7fbc1c0c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:22.240 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.256+0000 7fbc20fca640 1 --2- 172.21.15.22:0/814424935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc1c0adc80 0x7fbc1c0c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:22.240 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.256+0000 7fbc20fca640 1 -- 172.21.15.22:0/814424935 >> 172.21.15.22:0/814424935 conn(0x7fbc1c0d0100 msgr2=0x7fbc1c0d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:22.240 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.256+0000 7fbc20fca640 1 -- 172.21.15.22:0/814424935 shutdown_connections 2024-03-29T04:17:22.240 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:22.256+0000 7fbc20fca640 1 -- 172.21.15.22:0/814424935 wait complete. 2024-03-29T04:17:22.254 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-29T04:17:29.061 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-29T04:17:29.187 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.204+0000 7fbb6342b640 1 Processor -- start 2024-03-29T04:17:29.187 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.204+0000 7fbb6342b640 1 -- start start 2024-03-29T04:17:29.188 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb6342b640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb5c06f210 0x7fbb5c0ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:29.188 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb6342b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb5c15b920 con 0x7fbb5c15b540 2024-03-29T04:17:29.188 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb6342b640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb5c0ce4b0 con 0x7fbb5c06c720 2024-03-29T04:17:29.188 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb6342b640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fbb5c06e490 con 0x7fbb5c06f210 2024-03-29T04:17:29.189 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb611c7640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fbb5c15b540 0x7fbb5c0cdcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54486/0 (socket says 172.21.15.22:54486) 2024-03-29T04:17:29.189 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb611c7640 1 -- 172.21.15.22:0/2784059702 learned_addr learned my addr 172.21.15.22:0/2784059702 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:29.191 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb609c6640 1 --2- 172.21.15.22:0/2784059702 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb5c06f210 0x7fbb5c0ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:29.191 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb53fff640 1 -- 172.21.15.22:0/2784059702 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1190538923 0 0) 0x7fbb5c15b920 con 0x7fbb5c15b540 2024-03-29T04:17:29.191 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb53fff640 1 -- 172.21.15.22:0/2784059702 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb48003530 con 0x7fbb5c15b540 2024-03-29T04:17:29.191 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb53fff640 1 -- 172.21.15.22:0/2784059702 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2014451275 0 0) 0x7fbb48003530 con 0x7fbb5c15b540 2024-03-29T04:17:29.191 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb53fff640 1 -- 172.21.15.22:0/2784059702 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb5c06f210 msgr2=0x7fbb5c0ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:29.191 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb53fff640 1 --2- 172.21.15.22:0/2784059702 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb5c06f210 0x7fbb5c0ce9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.191 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb53fff640 1 -- 172.21.15.22:0/2784059702 >> v1:172.21.15.151:6789/0 conn(0x7fbb5c06c720 legacy=0x7fbb5c0caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:29.192 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb53fff640 1 -- 172.21.15.22:0/2784059702 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5c06e730 con 0x7fbb5c15b540 2024-03-29T04:17:29.192 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb53fff640 1 -- 172.21.15.22:0/2784059702 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7fbb58001680 con 0x7fbb5c15b540 2024-03-29T04:17:29.192 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb53fff640 1 -- 172.21.15.22:0/2784059702 >> v1:172.21.15.22:6789/0 conn(0x7fbb5c15b540 legacy=0x7fbb5c0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:29.192 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb53fff640 1 --2- 172.21.15.22:0/2784059702 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb5c06f210 0x7fbb480039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:29.192 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb53fff640 1 --2- 172.21.15.22:0/2784059702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb48003f30 0x7fbb48004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:29.192 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb53fff640 1 --2- 172.21.15.22:0/2784059702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb48004d50 0x7fbb480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:29.192 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb53fff640 1 -- 172.21.15.22:0/2784059702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c06e490 con 0x7fbb48003f30 2024-03-29T04:17:29.192 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb53fff640 1 -- 172.21.15.22:0/2784059702 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4800bb10 con 0x7fbb48004d50 2024-03-29T04:17:29.192 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb53fff640 1 -- 172.21.15.22:0/2784059702 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fbb4800df20 con 0x7fbb5c06f210 2024-03-29T04:17:29.192 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb609c6640 1 --2- 172.21.15.22:0/2784059702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb48003f30 0x7fbb48004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:29.192 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb611c7640 1 --2- 172.21.15.22:0/2784059702 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb5c06f210 0x7fbb480039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:29.193 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb619c8640 1 --2- 172.21.15.22:0/2784059702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb48004d50 0x7fbb480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:29.193 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb609c6640 1 -- 172.21.15.22:0/2784059702 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb5c06f210 msgr2=0x7fbb480039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:29.193 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb609c6640 1 --2- 172.21.15.22:0/2784059702 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb5c06f210 0x7fbb480039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.193 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb609c6640 1 -- 172.21.15.22:0/2784059702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb48004d50 msgr2=0x7fbb480051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:29.193 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb609c6640 1 --2- 172.21.15.22:0/2784059702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb48004d50 0x7fbb480051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.193 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.208+0000 7fbb609c6640 1 -- 172.21.15.22:0/2784059702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb48003530 con 0x7fbb48003f30 2024-03-29T04:17:29.194 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb609c6640 1 --2- 172.21.15.22:0/2784059702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb48003f30 0x7fbb48004810 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009ae0 tx=0x7fbb4c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba998cc4836e0ae5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:29.195 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb53fff640 1 -- 172.21.15.22:0/2784059702 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c0b2070 con 0x7fbb48003f30 2024-03-29T04:17:29.195 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb53fff640 1 -- 172.21.15.22:0/2784059702 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb4c0a5660 con 0x7fbb48003f30 2024-03-29T04:17:29.195 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb53fff640 1 -- 172.21.15.22:0/2784059702 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c0096b0 con 0x7fbb48003f30 2024-03-29T04:17:29.195 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb6342b640 1 -- 172.21.15.22:0/2784059702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb48003f30 msgr2=0x7fbb48004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:29.195 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb6342b640 1 --2- 172.21.15.22:0/2784059702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb48003f30 0x7fbb48004810 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009ae0 tx=0x7fbb4c01ca70 comp rx=0 tx=0).stop 2024-03-29T04:17:29.195 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb619c8640 1 -- 172.21.15.22:0/2784059702 reap_dead start 2024-03-29T04:17:29.195 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb6342b640 1 -- 172.21.15.22:0/2784059702 shutdown_connections 2024-03-29T04:17:29.195 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb6342b640 1 -- 172.21.15.22:0/2784059702 >> 172.21.15.22:0/2784059702 conn(0x7fbb5c0d0100 msgr2=0x7fbb5c0d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:29.195 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb6342b640 1 -- 172.21.15.22:0/2784059702 shutdown_connections 2024-03-29T04:17:29.195 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb6342b640 1 -- 172.21.15.22:0/2784059702 wait complete. 2024-03-29T04:17:29.195 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb6342b640 1 Processor -- start 2024-03-29T04:17:29.195 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb6342b640 1 -- start start 2024-03-29T04:17:29.197 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb6342b640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb48003f30 0x7fbb5c09cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:29.198 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb6342b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb5c09d2e0 0x7fbb5c09aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:29.198 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb6342b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb5c09c660 0x7fbb5c09af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:29.198 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb6342b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c0ce9f0 con 0x7fbb5c09d2e0 2024-03-29T04:17:29.198 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb6342b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c06ecf0 con 0x7fbb5c09c660 2024-03-29T04:17:29.198 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb6342b640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fbb5c0ceb70 con 0x7fbb48003f30 2024-03-29T04:17:29.198 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb609c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb5c09d2e0 0x7fbb5c09aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:29.198 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb609c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb5c09d2e0 0x7fbb5c09aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53508/0 (socket says 172.21.15.22:53508) 2024-03-29T04:17:29.198 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.212+0000 7fbb609c6640 1 -- 172.21.15.22:0/4022456407 learned_addr learned my addr 172.21.15.22:0/4022456407 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:29.198 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.216+0000 7fbb611c7640 1 --2- 172.21.15.22:0/4022456407 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb48003f30 0x7fbb5c09cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:29.198 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.216+0000 7fbb609c6640 1 -- 172.21.15.22:0/4022456407 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb48003f30 msgr2=0x7fbb5c09cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:29.199 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.216+0000 7fbb609c6640 1 --2- 172.21.15.22:0/4022456407 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb48003f30 0x7fbb5c09cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.199 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.216+0000 7fbb609c6640 1 -- 172.21.15.22:0/4022456407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb5c09c660 msgr2=0x7fbb5c09af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:29.199 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.216+0000 7fbb609c6640 1 --2- 172.21.15.22:0/4022456407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb5c09c660 0x7fbb5c09af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.199 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.216+0000 7fbb609c6640 1 -- 172.21.15.22:0/4022456407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5c1f35b0 con 0x7fbb5c09d2e0 2024-03-29T04:17:29.199 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.216+0000 7fbb609c6640 1 --2- 172.21.15.22:0/4022456407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb5c09d2e0 0x7fbb5c09aa30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c01c8a0 tx=0x7fbb48005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:29.199 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.216+0000 7fbb51ffb640 1 -- 172.21.15.22:0/4022456407 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c0b2040 con 0x7fbb5c09d2e0 2024-03-29T04:17:29.199 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.216+0000 7fbb51ffb640 1 -- 172.21.15.22:0/4022456407 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb4c004020 con 0x7fbb5c09d2e0 2024-03-29T04:17:29.199 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.216+0000 7fbb6342b640 1 -- 172.21.15.22:0/4022456407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb5c09d7f0 con 0x7fbb5c09d2e0 2024-03-29T04:17:29.199 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.216+0000 7fbb51ffb640 1 -- 172.21.15.22:0/4022456407 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c0043d0 con 0x7fbb5c09d2e0 2024-03-29T04:17:29.199 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.216+0000 7fbb6342b640 1 -- 172.21.15.22:0/4022456407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb5c1f3b50 con 0x7fbb5c09d2e0 2024-03-29T04:17:29.199 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.216+0000 7fbb6342b640 1 -- 172.21.15.22:0/4022456407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb5c068fa0 con 0x7fbb5c09d2e0 2024-03-29T04:17:29.200 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.216+0000 7fbb51ffb640 1 -- 172.21.15.22:0/4022456407 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fbb4c003620 con 0x7fbb5c09d2e0 2024-03-29T04:17:29.200 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.216+0000 7fbb51ffb640 1 --2- 172.21.15.22:0/4022456407 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbb40067ff0 0x7fbb4006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:29.201 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.216+0000 7fbb51ffb640 1 -- 172.21.15.22:0/4022456407 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fbb4c120280 con 0x7fbb5c09d2e0 2024-03-29T04:17:29.201 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.216+0000 7fbb611c7640 1 --2- 172.21.15.22:0/4022456407 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbb40067ff0 0x7fbb4006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:29.201 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.216+0000 7fbb611c7640 1 --2- 172.21.15.22:0/4022456407 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbb40067ff0 0x7fbb4006a4b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbb58000c10 tx=0x7fbb58009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:29.203 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.220+0000 7fbb51ffb640 1 -- 172.21.15.22:0/4022456407 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fbb4c016780 con 0x7fbb5c09d2e0 2024-03-29T04:17:29.364 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.384+0000 7fbb6342b640 1 -- 172.21.15.22:0/4022456407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb5c073210 con 0x7fbb5c09d2e0 2024-03-29T04:17:29.365 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.384+0000 7fbb51ffb640 1 -- 172.21.15.22:0/4022456407 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12884 (secure 0 0 0) 0x7fbb4c0f0690 con 0x7fbb5c09d2e0 2024-03-29T04:17:29.366 INFO:teuthology.misc.health.smithi022.stdout: 2024-03-29T04:17:29.366 INFO:teuthology.misc.health.smithi022.stdout:{"epoch":17,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:17:26.357154+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:17:29.368 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.388+0000 7fbb6342b640 1 -- 172.21.15.22:0/4022456407 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbb40067ff0 msgr2=0x7fbb4006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:29.368 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.388+0000 7fbb6342b640 1 --2- 172.21.15.22:0/4022456407 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbb40067ff0 0x7fbb4006a4b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbb58000c10 tx=0x7fbb58009040 comp rx=0 tx=0).stop 2024-03-29T04:17:29.368 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.388+0000 7fbb6342b640 1 -- 172.21.15.22:0/4022456407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb5c09d2e0 msgr2=0x7fbb5c09aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:29.368 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.388+0000 7fbb6342b640 1 --2- 172.21.15.22:0/4022456407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb5c09d2e0 0x7fbb5c09aa30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c01c8a0 tx=0x7fbb48005840 comp rx=0 tx=0).stop 2024-03-29T04:17:29.368 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.388+0000 7fbb6342b640 1 -- 172.21.15.22:0/4022456407 shutdown_connections 2024-03-29T04:17:29.368 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.388+0000 7fbb6342b640 1 --2- 172.21.15.22:0/4022456407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb5c09c660 0x7fbb5c09af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.368 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.388+0000 7fbb6342b640 1 --2- 172.21.15.22:0/4022456407 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbb40067ff0 0x7fbb4006a4b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.369 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.388+0000 7fbb6342b640 1 --2- 172.21.15.22:0/4022456407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb5c09d2e0 0x7fbb5c09aa30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.369 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.388+0000 7fbb6342b640 1 --2- 172.21.15.22:0/4022456407 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb48003f30 0x7fbb5c09cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.369 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.388+0000 7fbb6342b640 1 -- 172.21.15.22:0/4022456407 >> 172.21.15.22:0/4022456407 conn(0x7fbb5c0d0100 msgr2=0x7fbb5c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:29.369 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.388+0000 7fbb6342b640 1 -- 172.21.15.22:0/4022456407 shutdown_connections 2024-03-29T04:17:29.369 INFO:teuthology.misc.health.smithi022.stderr:2024-03-29T04:17:29.388+0000 7fbb6342b640 1 -- 172.21.15.22:0/4022456407 wait complete. 2024-03-29T04:17:29.385 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-29T04:17:29.385 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-29T04:17:29.385 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-29T04:17:29.385 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-29T04:17:29.388 INFO:tasks.ceph.mds.a:Started 2024-03-29T04:17:29.388 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-29T04:17:29.388 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-29T04:17:29.390 INFO:tasks.ceph.mds.c:Started 2024-03-29T04:17:29.390 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-29T04:17:29.390 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-29T04:17:29.393 INFO:tasks.ceph.mds.b:Started 2024-03-29T04:17:29.393 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-29T04:17:29.393 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-29T04:17:29.395 INFO:tasks.ceph.mds.d:Started 2024-03-29T04:17:29.395 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-29T04:17:29.396 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:17:29.460 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2024-03-29T04:17:29.461 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2024-03-29T04:17:29.475 INFO:tasks.ceph.mds.d.smithi151.stdout:starting mds.d at 2024-03-29T04:17:29.476 INFO:tasks.ceph.mds.b.smithi151.stdout:starting mds.b at 2024-03-29T04:17:29.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.532+0000 7f9eb92de640 1 Processor -- start 2024-03-29T04:17:29.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.532+0000 7f9eb92de640 1 -- start start 2024-03-29T04:17:29.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.532+0000 7f9eb92de640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9eb415bd00 0x7f9eb40d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:29.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.532+0000 7f9eb92de640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9eb40cfc40 0x7f9eb40d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:29.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.532+0000 7f9eb92de640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9eb40cc6c0 con 0x7f9eb415c0e0 2024-03-29T04:17:29.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.532+0000 7f9eb92de640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f9eb40c7190 con 0x7f9eb40cfc40 2024-03-29T04:17:29.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.532+0000 7f9eb92de640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f9eb40cc980 con 0x7f9eb415bd00 2024-03-29T04:17:29.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.532+0000 7f9eb37fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9eb40cfc40 0x7f9eb40d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:29.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.532+0000 7f9eb37fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9eb40cfc40 0x7f9eb40d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53542/0 (socket says 172.21.15.22:53542) 2024-03-29T04:17:29.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.532+0000 7f9eb37fe640 1 -- 172.21.15.22:0/24080619 learned_addr learned my addr 172.21.15.22:0/24080619 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:29.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.532+0000 7f9eb2ffd640 1 --2- 172.21.15.22:0/24080619 >> v2:172.21.15.151:3300/0 conn(0x7f9eb415bd00 0x7f9eb40d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:29.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.532+0000 7f9eb37fe640 1 -- 172.21.15.22:0/24080619 >> v2:172.21.15.151:3300/0 conn(0x7f9eb415bd00 msgr2=0x7f9eb40d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:29.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.532+0000 7f9eb37fe640 1 --2- 172.21.15.22:0/24080619 >> v2:172.21.15.151:3300/0 conn(0x7f9eb415bd00 0x7f9eb40d1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.532+0000 7f9eb37fe640 1 -- 172.21.15.22:0/24080619 >> v1:172.21.15.151:6789/0 conn(0x7f9eb415c0e0 legacy=0x7f9eb40cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:29.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.532+0000 7f9eb37fe640 1 -- 172.21.15.22:0/24080619 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb40ccc60 con 0x7f9eb40cfc40 2024-03-29T04:17:29.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.536+0000 7f9eb37fe640 1 --2- 172.21.15.22:0/24080619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb40cfc40 0x7f9eb40d00d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4073000 tx=0x7f9ea401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46df0d6ea097002 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:29.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.536+0000 7f9eb1ffb640 1 -- 172.21.15.22:0/24080619 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea40b0070 con 0x7f9eb40cfc40 2024-03-29T04:17:29.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.536+0000 7f9eb1ffb640 1 -- 172.21.15.22:0/24080619 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea40ab600 con 0x7f9eb40cfc40 2024-03-29T04:17:29.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.536+0000 7f9eb1ffb640 1 -- 172.21.15.22:0/24080619 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea40ab9d0 con 0x7f9eb40cfc40 2024-03-29T04:17:29.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.536+0000 7f9eb92de640 1 -- 172.21.15.22:0/24080619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb40cfc40 msgr2=0x7f9eb40d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:29.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.536+0000 7f9eb92de640 1 --2- 172.21.15.22:0/24080619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb40cfc40 0x7f9eb40d00d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4073000 tx=0x7f9ea401c4e0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.536+0000 7f9eb92de640 1 -- 172.21.15.22:0/24080619 shutdown_connections 2024-03-29T04:17:29.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.536+0000 7f9eb92de640 1 --2- 172.21.15.22:0/24080619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb40cfc40 0x7f9eb40d00d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.536+0000 7f9eb92de640 1 --2- 172.21.15.22:0/24080619 >> v2:172.21.15.151:3300/0 conn(0x7f9eb415bd00 0x7f9eb40d1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.536+0000 7f9eb92de640 1 -- 172.21.15.22:0/24080619 >> 172.21.15.22:0/24080619 conn(0x7f9eb40b3c80 msgr2=0x7f9eb40b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:29.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.536+0000 7f9eb92de640 1 -- 172.21.15.22:0/24080619 shutdown_connections 2024-03-29T04:17:29.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.536+0000 7f9eb92de640 1 -- 172.21.15.22:0/24080619 wait complete. 2024-03-29T04:17:29.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.536+0000 7f9eb92de640 1 Processor -- start 2024-03-29T04:17:29.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.536+0000 7f9eb92de640 1 -- start start 2024-03-29T04:17:29.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb92de640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9eb40cfc40 0x7f9eb4152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:29.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb92de640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9eb415bd00 0x7f9eb4154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:29.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb92de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb415c0e0 0x7f9eb4154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:29.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb92de640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb4162320 con 0x7f9eb415c0e0 2024-03-29T04:17:29.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb92de640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb41621a0 con 0x7f9eb415bd00 2024-03-29T04:17:29.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb92de640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb41624a0 con 0x7f9eb40cfc40 2024-03-29T04:17:29.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb415c0e0 0x7f9eb4154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:29.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb415c0e0 0x7f9eb4154cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53554/0 (socket says 172.21.15.22:53554) 2024-03-29T04:17:29.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb37fe640 1 -- 172.21.15.22:0/118716454 learned_addr learned my addr 172.21.15.22:0/118716454 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:29.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb27fc640 1 --2- 172.21.15.22:0/118716454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9eb415bd00 0x7f9eb4154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:29.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb2ffd640 1 --2- 172.21.15.22:0/118716454 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9eb40cfc40 0x7f9eb4152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:29.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb37fe640 1 -- 172.21.15.22:0/118716454 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9eb40cfc40 msgr2=0x7f9eb4152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:29.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb37fe640 1 --2- 172.21.15.22:0/118716454 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9eb40cfc40 0x7f9eb4152a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb37fe640 1 -- 172.21.15.22:0/118716454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9eb415bd00 msgr2=0x7f9eb4154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:29.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb37fe640 1 --2- 172.21.15.22:0/118716454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9eb415bd00 0x7f9eb4154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb37fe640 1 -- 172.21.15.22:0/118716454 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb4159a50 con 0x7f9eb415c0e0 2024-03-29T04:17:29.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb37fe640 1 --2- 172.21.15.22:0/118716454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb415c0e0 0x7f9eb4154cd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40ae040 tx=0x7f9ea401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:29.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eab7fe640 1 -- 172.21.15.22:0/118716454 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea40b0040 con 0x7f9eb415c0e0 2024-03-29T04:17:29.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eab7fe640 1 -- 172.21.15.22:0/118716454 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea40bc070 con 0x7f9eb415c0e0 2024-03-29T04:17:29.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eab7fe640 1 -- 172.21.15.22:0/118716454 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea40aa970 con 0x7f9eb415c0e0 2024-03-29T04:17:29.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb92de640 1 -- 172.21.15.22:0/118716454 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb4159d30 con 0x7f9eb415c0e0 2024-03-29T04:17:29.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb92de640 1 -- 172.21.15.22:0/118716454 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb415a2f0 con 0x7f9eb415c0e0 2024-03-29T04:17:29.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb92de640 1 -- 172.21.15.22:0/118716454 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eb4068890 con 0x7f9eb415c0e0 2024-03-29T04:17:29.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eab7fe640 1 -- 172.21.15.22:0/118716454 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9ea40aab10 con 0x7f9eb415c0e0 2024-03-29T04:17:29.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eab7fe640 1 --2- 172.21.15.22:0/118716454 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9e9c067fc0 0x7f9e9c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:29.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eab7fe640 1 -- 172.21.15.22:0/118716454 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f9ea41208e0 con 0x7f9eb415c0e0 2024-03-29T04:17:29.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb2ffd640 1 --2- 172.21.15.22:0/118716454 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9e9c067fc0 0x7f9e9c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:29.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.540+0000 7f9eb2ffd640 1 --2- 172.21.15.22:0/118716454 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9e9c067fc0 0x7f9e9c06a480 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9eb416a1b0 tx=0x7f9eac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:29.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.544+0000 7f9eab7fe640 1 -- 172.21.15.22:0/118716454 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9ea40160d0 con 0x7f9eb415c0e0 2024-03-29T04:17:29.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.708+0000 7f9eb92de640 1 -- 172.21.15.22:0/118716454 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9eb40c8810 con 0x7f9eb415c0e0 2024-03-29T04:17:29.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.708+0000 7f9eab7fe640 1 -- 172.21.15.22:0/118716454 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12884 (secure 0 0 0) 0x7f9ea40f1340 con 0x7f9eb415c0e0 2024-03-29T04:17:29.690 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:29.691 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:17:26.357154+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:17:29.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.712+0000 7f9eb92de640 1 -- 172.21.15.22:0/118716454 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9e9c067fc0 msgr2=0x7f9e9c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:29.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.712+0000 7f9eb92de640 1 --2- 172.21.15.22:0/118716454 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9e9c067fc0 0x7f9e9c06a480 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9eb416a1b0 tx=0x7f9eac008040 comp rx=0 tx=0).stop 2024-03-29T04:17:29.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.712+0000 7f9eb92de640 1 -- 172.21.15.22:0/118716454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb415c0e0 msgr2=0x7f9eb4154cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:29.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.712+0000 7f9eb92de640 1 --2- 172.21.15.22:0/118716454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb415c0e0 0x7f9eb4154cd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40ae040 tx=0x7f9ea401ad20 comp rx=0 tx=0).stop 2024-03-29T04:17:29.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.712+0000 7f9eb92de640 1 -- 172.21.15.22:0/118716454 shutdown_connections 2024-03-29T04:17:29.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.712+0000 7f9eb92de640 1 --2- 172.21.15.22:0/118716454 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9e9c067fc0 0x7f9e9c06a480 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.712+0000 7f9eb92de640 1 --2- 172.21.15.22:0/118716454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb415c0e0 0x7f9eb4154cd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.712+0000 7f9eb92de640 1 --2- 172.21.15.22:0/118716454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9eb415bd00 0x7f9eb4154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.712+0000 7f9eb92de640 1 --2- 172.21.15.22:0/118716454 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9eb40cfc40 0x7f9eb4152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.712+0000 7f9eb92de640 1 -- 172.21.15.22:0/118716454 >> 172.21.15.22:0/118716454 conn(0x7f9eb40b3c80 msgr2=0x7f9eb415add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:29.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.712+0000 7f9eb92de640 1 -- 172.21.15.22:0/118716454 shutdown_connections 2024-03-29T04:17:29.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.712+0000 7f9eb92de640 1 -- 172.21.15.22:0/118716454 wait complete. 2024-03-29T04:17:29.709 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:17:29.710 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:17:29.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.860+0000 7ff0c4e87640 1 Processor -- start 2024-03-29T04:17:29.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.860+0000 7ff0c4e87640 1 -- start start 2024-03-29T04:17:29.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.860+0000 7ff0c4e87640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff0c015e2f0 0x7ff0c00b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:29.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.860+0000 7ff0c4e87640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0c015e6d0 0x7ff0c00ae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:29.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.860+0000 7ff0c4e87640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0c00b58c0 con 0x7ff0c00ad8e0 2024-03-29T04:17:29.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.860+0000 7ff0c4e87640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7ff0c00cd060 con 0x7ff0c015e2f0 2024-03-29T04:17:29.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.860+0000 7ff0c4e87640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff0c006b660 con 0x7ff0c015e6d0 2024-03-29T04:17:29.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.860+0000 7ff0bed9d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff0c00ad8e0 0x7ff0c01620e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54514/0 (socket says 172.21.15.22:54514) 2024-03-29T04:17:29.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.860+0000 7ff0bed9d640 1 -- 172.21.15.22:0/3658197920 learned_addr learned my addr 172.21.15.22:0/3658197920 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:29.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.860+0000 7ff0be59c640 1 --2- 172.21.15.22:0/3658197920 >> v2:172.21.15.22:3300/0 conn(0x7ff0c015e2f0 0x7ff0c00b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:29.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.860+0000 7ff0bdd9b640 1 --2- 172.21.15.22:0/3658197920 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0c015e6d0 0x7ff0c00ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:29.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.860+0000 7ff0bd59a640 1 -- 172.21.15.22:0/3658197920 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4114429409 0 0) 0x7ff0c00b58c0 con 0x7ff0c00ad8e0 2024-03-29T04:17:29.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.860+0000 7ff0bd59a640 1 -- 172.21.15.22:0/3658197920 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0a8003530 con 0x7ff0c00ad8e0 2024-03-29T04:17:29.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.860+0000 7ff0be59c640 1 -- 172.21.15.22:0/3658197920 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0c015e6d0 msgr2=0x7ff0c00ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:29.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.860+0000 7ff0be59c640 1 --2- 172.21.15.22:0/3658197920 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0c015e6d0 0x7ff0c00ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.860+0000 7ff0be59c640 1 -- 172.21.15.22:0/3658197920 >> v1:172.21.15.22:6789/0 conn(0x7ff0c00ad8e0 legacy=0x7ff0c01620e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:29.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.860+0000 7ff0be59c640 1 -- 172.21.15.22:0/3658197920 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c00b5ba0 con 0x7ff0c015e2f0 2024-03-29T04:17:29.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.864+0000 7ff0be59c640 1 --2- 172.21.15.22:0/3658197920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c015e2f0 0x7ff0c00b51e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff0b00097c0 tx=0x7ff0b001c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882264baf37f263d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:29.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.864+0000 7ff0bd59a640 1 -- 172.21.15.22:0/3658197920 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b00b0070 con 0x7ff0c015e2f0 2024-03-29T04:17:29.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.864+0000 7ff0bd59a640 1 -- 172.21.15.22:0/3658197920 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0b00ab600 con 0x7ff0c015e2f0 2024-03-29T04:17:29.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.864+0000 7ff0bd59a640 1 -- 172.21.15.22:0/3658197920 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b00ab9d0 con 0x7ff0c015e2f0 2024-03-29T04:17:29.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.864+0000 7ff0c4e87640 1 -- 172.21.15.22:0/3658197920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c015e2f0 msgr2=0x7ff0c00b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:29.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.864+0000 7ff0c4e87640 1 --2- 172.21.15.22:0/3658197920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c015e2f0 0x7ff0c00b51e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff0b00097c0 tx=0x7ff0b001c350 comp rx=0 tx=0).stop 2024-03-29T04:17:29.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.864+0000 7ff0c4e87640 1 -- 172.21.15.22:0/3658197920 shutdown_connections 2024-03-29T04:17:29.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.864+0000 7ff0c4e87640 1 --2- 172.21.15.22:0/3658197920 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0c015e6d0 0x7ff0c00ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.864+0000 7ff0c4e87640 1 --2- 172.21.15.22:0/3658197920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c015e2f0 0x7ff0c00b51e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.864+0000 7ff0c4e87640 1 -- 172.21.15.22:0/3658197920 >> 172.21.15.22:0/3658197920 conn(0x7ff0c00d0c90 msgr2=0x7ff0c00d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:29.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.864+0000 7ff0c4e87640 1 -- 172.21.15.22:0/3658197920 shutdown_connections 2024-03-29T04:17:29.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.864+0000 7ff0c4e87640 1 -- 172.21.15.22:0/3658197920 wait complete. 2024-03-29T04:17:29.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.864+0000 7ff0c4e87640 1 Processor -- start 2024-03-29T04:17:29.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0c4e87640 1 -- start start 2024-03-29T04:17:29.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0c4e87640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0c00ad8e0 0x7ff0c014d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:29.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0c4e87640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c015e2f0 0x7ff0c0153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:29.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0c4e87640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0c015e6d0 0x7ff0c01f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:29.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0c4e87640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c016b600 con 0x7ff0c015e2f0 2024-03-29T04:17:29.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0c4e87640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c016b480 con 0x7ff0c00ad8e0 2024-03-29T04:17:29.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0c4e87640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff0c016b780 con 0x7ff0c015e6d0 2024-03-29T04:17:29.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0bdd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c015e2f0 0x7ff0c0153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:29.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0bdd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c015e2f0 0x7ff0c0153960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53572/0 (socket says 172.21.15.22:53572) 2024-03-29T04:17:29.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0bdd9b640 1 -- 172.21.15.22:0/1467035949 learned_addr learned my addr 172.21.15.22:0/1467035949 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:29.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0be59c640 1 --2- 172.21.15.22:0/1467035949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0c00ad8e0 0x7ff0c014d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:29.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0bed9d640 1 --2- 172.21.15.22:0/1467035949 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0c015e6d0 0x7ff0c01f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:29.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0bdd9b640 1 -- 172.21.15.22:0/1467035949 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0c015e6d0 msgr2=0x7ff0c01f7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:29.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0bdd9b640 1 --2- 172.21.15.22:0/1467035949 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0c015e6d0 0x7ff0c01f7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0bdd9b640 1 -- 172.21.15.22:0/1467035949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0c00ad8e0 msgr2=0x7ff0c014d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:29.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0bdd9b640 1 --2- 172.21.15.22:0/1467035949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0c00ad8e0 0x7ff0c014d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:29.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0bdd9b640 1 -- 172.21.15.22:0/1467035949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c014dff0 con 0x7ff0c015e2f0 2024-03-29T04:17:29.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0bed9d640 1 --2- 172.21.15.22:0/1467035949 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0c015e6d0 0x7ff0c01f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:29.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0bdd9b640 1 --2- 172.21.15.22:0/1467035949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c015e2f0 0x7ff0c0153960 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff0b400d5d0 tx=0x7ff0b400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:29.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0aeffd640 1 -- 172.21.15.22:0/1467035949 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4004240 con 0x7ff0c015e2f0 2024-03-29T04:17:29.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0aeffd640 1 -- 172.21.15.22:0/1467035949 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0b40043e0 con 0x7ff0c015e2f0 2024-03-29T04:17:29.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0c4e87640 1 -- 172.21.15.22:0/1467035949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0c014e2d0 con 0x7ff0c015e2f0 2024-03-29T04:17:29.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0aeffd640 1 -- 172.21.15.22:0/1467035949 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4003840 con 0x7ff0c015e2f0 2024-03-29T04:17:29.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0c4e87640 1 -- 172.21.15.22:0/1467035949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0c014e890 con 0x7ff0c015e2f0 2024-03-29T04:17:29.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0c4e87640 1 -- 172.21.15.22:0/1467035949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0c0068900 con 0x7ff0c015e2f0 2024-03-29T04:17:29.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0aeffd640 1 -- 172.21.15.22:0/1467035949 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff0b40039e0 con 0x7ff0c015e2f0 2024-03-29T04:17:29.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0aeffd640 1 --2- 172.21.15.22:0/1467035949 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff0a0067fc0 0x7ff0a006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:29.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.868+0000 7ff0aeffd640 1 -- 172.21.15.22:0/1467035949 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7ff0b40864e0 con 0x7ff0c015e2f0 2024-03-29T04:17:29.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.872+0000 7ff0be59c640 1 --2- 172.21.15.22:0/1467035949 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff0a0067fc0 0x7ff0a006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:29.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.872+0000 7ff0be59c640 1 --2- 172.21.15.22:0/1467035949 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff0a0067fc0 0x7ff0a006a480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff0b00ad000 tx=0x7ff0b00a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:29.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:29.876+0000 7ff0aeffd640 1 -- 172.21.15.22:0/1467035949 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff0b401c360 con 0x7ff0c015e2f0 2024-03-29T04:17:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.036+0000 7ff0c4e87640 1 -- 172.21.15.22:0/1467035949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff0c0070680 con 0x7ff0c015e2f0 2024-03-29T04:17:30.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.040+0000 7ff0aeffd640 1 -- 172.21.15.22:0/1467035949 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7ff0b4056fd0 con 0x7ff0c015e2f0 2024-03-29T04:17:30.020 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:17:30.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.040+0000 7ff0c4e87640 1 -- 172.21.15.22:0/1467035949 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff0a0067fc0 msgr2=0x7ff0a006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:30.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.040+0000 7ff0c4e87640 1 --2- 172.21.15.22:0/1467035949 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff0a0067fc0 0x7ff0a006a480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff0b00ad000 tx=0x7ff0b00a6000 comp rx=0 tx=0).stop 2024-03-29T04:17:30.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.040+0000 7ff0c4e87640 1 -- 172.21.15.22:0/1467035949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c015e2f0 msgr2=0x7ff0c0153960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:30.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.040+0000 7ff0c4e87640 1 --2- 172.21.15.22:0/1467035949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c015e2f0 0x7ff0c0153960 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff0b400d5d0 tx=0x7ff0b400da90 comp rx=0 tx=0).stop 2024-03-29T04:17:30.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.040+0000 7ff0c4e87640 1 -- 172.21.15.22:0/1467035949 shutdown_connections 2024-03-29T04:17:30.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.040+0000 7ff0c4e87640 1 --2- 172.21.15.22:0/1467035949 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0c015e6d0 0x7ff0c01f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:30.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.040+0000 7ff0c4e87640 1 --2- 172.21.15.22:0/1467035949 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff0a0067fc0 0x7ff0a006a480 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:30.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.040+0000 7ff0c4e87640 1 --2- 172.21.15.22:0/1467035949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c015e2f0 0x7ff0c0153960 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:30.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.040+0000 7ff0c4e87640 1 --2- 172.21.15.22:0/1467035949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0c00ad8e0 0x7ff0c014d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:30.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.040+0000 7ff0c4e87640 1 -- 172.21.15.22:0/1467035949 >> 172.21.15.22:0/1467035949 conn(0x7ff0c00d0c90 msgr2=0x7ff0c0157590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:30.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.040+0000 7ff0c4e87640 1 -- 172.21.15.22:0/1467035949 shutdown_connections 2024-03-29T04:17:30.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.040+0000 7ff0c4e87640 1 -- 172.21.15.22:0/1467035949 wait complete. 2024-03-29T04:17:30.039 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:17:30.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e8f22d640 1 Processor -- start 2024-03-29T04:17:30.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e8f22d640 1 -- start start 2024-03-29T04:17:30.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e8f22d640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9e8806c670 0x7f9e880cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:30.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e8f22d640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e880ccef0 con 0x7f9e8806ca50 2024-03-29T04:17:30.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e8f22d640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e880b3710 con 0x7f9e880b41f0 2024-03-29T04:17:30.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e8f22d640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f9e880b34c0 con 0x7f9e8806c670 2024-03-29T04:17:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e8d7ca640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f9e8806ca50 0x7f9e880ce5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54526/0 (socket says 172.21.15.22:54526) 2024-03-29T04:17:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e8d7ca640 1 -- 172.21.15.22:0/2754592029 learned_addr learned my addr 172.21.15.22:0/2754592029 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e8cfc9640 1 --2- 172.21.15.22:0/2754592029 >> v2:172.21.15.151:3300/0 conn(0x7f9e8806c670 0x7f9e880cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e877fe640 1 -- 172.21.15.22:0/2754592029 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3682824719 0 0) 0x7f9e880ccef0 con 0x7f9e8806ca50 2024-03-29T04:17:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e877fe640 1 -- 172.21.15.22:0/2754592029 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e70003530 con 0x7f9e8806ca50 2024-03-29T04:17:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e877fe640 1 -- 172.21.15.22:0/2754592029 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3274559117 0 0) 0x7f9e70003530 con 0x7f9e8806ca50 2024-03-29T04:17:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e877fe640 1 -- 172.21.15.22:0/2754592029 >> v2:172.21.15.151:3300/0 conn(0x7f9e8806c670 msgr2=0x7f9e880cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e877fe640 1 --2- 172.21.15.22:0/2754592029 >> v2:172.21.15.151:3300/0 conn(0x7f9e8806c670 0x7f9e880cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e877fe640 1 -- 172.21.15.22:0/2754592029 >> v1:172.21.15.151:6789/0 conn(0x7f9e880b41f0 legacy=0x7f9e8806e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e877fe640 1 -- 172.21.15.22:0/2754592029 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e880b39f0 con 0x7f9e8806ca50 2024-03-29T04:17:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e877fe640 1 -- 172.21.15.22:0/2754592029 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f9e80001680 con 0x7f9e8806ca50 2024-03-29T04:17:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e877fe640 1 -- 172.21.15.22:0/2754592029 >> v1:172.21.15.22:6789/0 conn(0x7f9e8806ca50 legacy=0x7f9e880ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e877fe640 1 --2- 172.21.15.22:0/2754592029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e700039c0 0x7f9e70003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e877fe640 1 --2- 172.21.15.22:0/2754592029 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9e70004810 0x7f9e70004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e877fe640 1 --2- 172.21.15.22:0/2754592029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e700051d0 0x7f9e7000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e877fe640 1 -- 172.21.15.22:0/2754592029 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9e880b34c0 con 0x7f9e700039c0 2024-03-29T04:17:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e877fe640 1 -- 172.21.15.22:0/2754592029 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7000c070 con 0x7f9e700051d0 2024-03-29T04:17:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e877fe640 1 -- 172.21.15.22:0/2754592029 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9e7000c270 con 0x7f9e70004810 2024-03-29T04:17:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e8cfc9640 1 --2- 172.21.15.22:0/2754592029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e700039c0 0x7f9e70003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e87fff640 1 --2- 172.21.15.22:0/2754592029 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9e70004810 0x7f9e70004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e8d7ca640 1 --2- 172.21.15.22:0/2754592029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e700051d0 0x7f9e7000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e8cfc9640 1 -- 172.21.15.22:0/2754592029 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9e70004810 msgr2=0x7f9e70004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e8cfc9640 1 --2- 172.21.15.22:0/2754592029 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9e70004810 0x7f9e70004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:30.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e8cfc9640 1 -- 172.21.15.22:0/2754592029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e700051d0 msgr2=0x7f9e7000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:30.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e8cfc9640 1 --2- 172.21.15.22:0/2754592029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e700051d0 0x7f9e7000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:30.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e8d7ca640 1 -- 172.21.15.22:0/2754592029 reap_dead start 2024-03-29T04:17:30.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e8cfc9640 1 -- 172.21.15.22:0/2754592029 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e70003530 con 0x7f9e700039c0 2024-03-29T04:17:30.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.184+0000 7f9e87fff640 1 --2- 172.21.15.22:0/2754592029 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9e70004810 0x7f9e70004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.188+0000 7f9e8cfc9640 1 --2- 172.21.15.22:0/2754592029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e700039c0 0x7f9e70003de0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009fa0 tx=0x7f9e7801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ee74f15ac7db22 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.188+0000 7f9e877fe640 1 -- 172.21.15.22:0/2754592029 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e780b2070 con 0x7f9e700039c0 2024-03-29T04:17:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.188+0000 7f9e877fe640 1 -- 172.21.15.22:0/2754592029 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e780a5af0 con 0x7f9e700039c0 2024-03-29T04:17:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.188+0000 7f9e877fe640 1 -- 172.21.15.22:0/2754592029 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78009bf0 con 0x7f9e700039c0 2024-03-29T04:17:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.188+0000 7f9e8f22d640 1 -- 172.21.15.22:0/2754592029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e700039c0 msgr2=0x7f9e70003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.188+0000 7f9e8f22d640 1 --2- 172.21.15.22:0/2754592029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e700039c0 0x7f9e70003de0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009fa0 tx=0x7f9e7801cc40 comp rx=0 tx=0).stop 2024-03-29T04:17:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.188+0000 7f9e8f22d640 1 -- 172.21.15.22:0/2754592029 shutdown_connections 2024-03-29T04:17:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.188+0000 7f9e8f22d640 1 --2- 172.21.15.22:0/2754592029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e700039c0 0x7f9e70003de0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.188+0000 7f9e8f22d640 1 -- 172.21.15.22:0/2754592029 >> 172.21.15.22:0/2754592029 conn(0x7f9e880cfee0 msgr2=0x7f9e880d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.188+0000 7f9e8f22d640 1 -- 172.21.15.22:0/2754592029 shutdown_connections 2024-03-29T04:17:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.188+0000 7f9e8f22d640 1 -- 172.21.15.22:0/2754592029 wait complete. 2024-03-29T04:17:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.188+0000 7f9e8f22d640 1 Processor -- start 2024-03-29T04:17:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e8f22d640 1 -- start start 2024-03-29T04:17:30.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e8f22d640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9e700039c0 0x7f9e880cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:30.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e8f22d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e881f88d0 0x7f9e881450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:30.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e8f22d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e8815b5c0 0x7f9e8815ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:30.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e8f22d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8806cfb0 con 0x7f9e881f88d0 2024-03-29T04:17:30.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e8f22d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8806ce30 con 0x7f9e8815b5c0 2024-03-29T04:17:30.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e8f22d640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9e88160590 con 0x7f9e700039c0 2024-03-29T04:17:30.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e87fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e881f88d0 0x7f9e881450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:30.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e87fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e881f88d0 0x7f9e881450a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53594/0 (socket says 172.21.15.22:53594) 2024-03-29T04:17:30.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e87fff640 1 -- 172.21.15.22:0/69497062 learned_addr learned my addr 172.21.15.22:0/69497062 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:30.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e8d7ca640 1 --2- 172.21.15.22:0/69497062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e8815b5c0 0x7f9e8815ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:30.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e8cfc9640 1 --2- 172.21.15.22:0/69497062 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9e700039c0 0x7f9e880cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:30.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e87fff640 1 -- 172.21.15.22:0/69497062 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9e700039c0 msgr2=0x7f9e880cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:30.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e87fff640 1 --2- 172.21.15.22:0/69497062 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9e700039c0 0x7f9e880cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:30.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e87fff640 1 -- 172.21.15.22:0/69497062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e8815b5c0 msgr2=0x7f9e8815ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:30.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e87fff640 1 --2- 172.21.15.22:0/69497062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e8815b5c0 0x7f9e8815ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:30.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e87fff640 1 -- 172.21.15.22:0/69497062 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e881f8de0 con 0x7f9e881f88d0 2024-03-29T04:17:30.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e87fff640 1 --2- 172.21.15.22:0/69497062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e881f88d0 0x7f9e881450a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00d4d0 tx=0x7f9e7c00d990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:30.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e857fa640 1 -- 172.21.15.22:0/69497062 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c019070 con 0x7f9e881f88d0 2024-03-29T04:17:30.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e857fa640 1 -- 172.21.15.22:0/69497062 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e7c0054a0 con 0x7f9e881f88d0 2024-03-29T04:17:30.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e8f22d640 1 -- 172.21.15.22:0/69497062 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e881f90c0 con 0x7f9e881f88d0 2024-03-29T04:17:30.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e857fa640 1 -- 172.21.15.22:0/69497062 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c007270 con 0x7f9e881f88d0 2024-03-29T04:17:30.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e8f22d640 1 -- 172.21.15.22:0/69497062 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e881f9680 con 0x7f9e881f88d0 2024-03-29T04:17:30.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e8f22d640 1 -- 172.21.15.22:0/69497062 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e88068890 con 0x7f9e881f88d0 2024-03-29T04:17:30.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e857fa640 1 -- 172.21.15.22:0/69497062 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9e7c01c070 con 0x7f9e881f88d0 2024-03-29T04:17:30.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e857fa640 1 --2- 172.21.15.22:0/69497062 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9e74067fc0 0x7f9e7406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:30.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e857fa640 1 -- 172.21.15.22:0/69497062 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f9e7c085870 con 0x7f9e881f88d0 2024-03-29T04:17:30.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.192+0000 7f9e8cfc9640 1 --2- 172.21.15.22:0/69497062 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9e74067fc0 0x7f9e7406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:30.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.196+0000 7f9e8cfc9640 1 --2- 172.21.15.22:0/69497062 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9e74067fc0 0x7f9e7406a480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9e780af000 tx=0x7f9e78004ac0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:30.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.200+0000 7f9e857fa640 1 -- 172.21.15.22:0/69497062 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9e7c0564c0 con 0x7f9e881f88d0 2024-03-29T04:17:30.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.392+0000 7f9e8f22d640 1 -- 172.21.15.22:0/69497062 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e8815bf90 con 0x7f9e881f88d0 2024-03-29T04:17:30.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.392+0000 7f9e857fa640 1 -- 172.21.15.22:0/69497062 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7f9e7c055dc0 con 0x7f9e881f88d0 2024-03-29T04:17:30.375 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:30.375 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/1466916671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":1466916671},{"type":"v1","addr":"172.21.15.151:6837","nonce":1466916671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-29T04:17:30.375 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 2 2024-03-29T04:17:30.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.396+0000 7f9e8f22d640 1 -- 172.21.15.22:0/69497062 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9e74067fc0 msgr2=0x7f9e7406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:30.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.396+0000 7f9e8f22d640 1 --2- 172.21.15.22:0/69497062 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9e74067fc0 0x7f9e7406a480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9e780af000 tx=0x7f9e78004ac0 comp rx=0 tx=0).stop 2024-03-29T04:17:30.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.396+0000 7f9e8f22d640 1 -- 172.21.15.22:0/69497062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e881f88d0 msgr2=0x7f9e881450a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:30.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.396+0000 7f9e8f22d640 1 --2- 172.21.15.22:0/69497062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e881f88d0 0x7f9e881450a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00d4d0 tx=0x7f9e7c00d990 comp rx=0 tx=0).stop 2024-03-29T04:17:30.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.396+0000 7f9e8f22d640 1 -- 172.21.15.22:0/69497062 shutdown_connections 2024-03-29T04:17:30.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.396+0000 7f9e8f22d640 1 --2- 172.21.15.22:0/69497062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e8815b5c0 0x7f9e8815ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:30.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.396+0000 7f9e8f22d640 1 --2- 172.21.15.22:0/69497062 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9e74067fc0 0x7f9e7406a480 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:30.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.396+0000 7f9e8f22d640 1 --2- 172.21.15.22:0/69497062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e881f88d0 0x7f9e881450a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:30.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.396+0000 7f9e8f22d640 1 --2- 172.21.15.22:0/69497062 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9e700039c0 0x7f9e880cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:30.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.396+0000 7f9e8f22d640 1 -- 172.21.15.22:0/69497062 >> 172.21.15.22:0/69497062 conn(0x7f9e880cfee0 msgr2=0x7f9e8815e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:30.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.396+0000 7f9e8f22d640 1 -- 172.21.15.22:0/69497062 shutdown_connections 2024-03-29T04:17:30.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:30.396+0000 7f9e8f22d640 1 -- 172.21.15.22:0/69497062 wait complete. 2024-03-29T04:17:31.106 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:17:31.109 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:17:32.394 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:17:32.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.552+0000 7fe65b657640 1 Processor -- start 2024-03-29T04:17:32.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.552+0000 7fe65b657640 1 -- start start 2024-03-29T04:17:32.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.552+0000 7fe65b657640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe6540b55a0 0x7fe6540b3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:32.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.552+0000 7fe65b657640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fe6540b3d80 0x7fe654168300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:32.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.552+0000 7fe65b657640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6540b4500 con 0x7fe6540cef80 2024-03-29T04:17:32.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.552+0000 7fe65b657640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fe6540b47e0 con 0x7fe6540b55a0 2024-03-29T04:17:32.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.552+0000 7fe65b657640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fe6540b4ac0 con 0x7fe6540b3d80 2024-03-29T04:17:32.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.552+0000 7fe6593f3640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe6540cef80 0x7fe6540cf360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54534/0 (socket says 172.21.15.22:54534) 2024-03-29T04:17:32.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.552+0000 7fe6593f3640 1 -- 172.21.15.22:0/656477700 learned_addr learned my addr 172.21.15.22:0/656477700 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:32.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.552+0000 7fe658bf2640 1 --2- 172.21.15.22:0/656477700 >> v2:172.21.15.22:3300/0 conn(0x7fe6540b55a0 0x7fe6540b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:32.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.552+0000 7fe659bf4640 1 --2- 172.21.15.22:0/656477700 >> v2:172.21.15.151:3300/0 conn(0x7fe6540b3d80 0x7fe654168300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:32.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.552+0000 7fe64bfff640 1 -- 172.21.15.22:0/656477700 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 945376310 0 0) 0x7fe6540b4500 con 0x7fe6540cef80 2024-03-29T04:17:32.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.552+0000 7fe64bfff640 1 -- 172.21.15.22:0/656477700 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe640003530 con 0x7fe6540cef80 2024-03-29T04:17:32.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.552+0000 7fe658bf2640 1 -- 172.21.15.22:0/656477700 >> v2:172.21.15.151:3300/0 conn(0x7fe6540b3d80 msgr2=0x7fe654168300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:32.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.552+0000 7fe658bf2640 1 --2- 172.21.15.22:0/656477700 >> v2:172.21.15.151:3300/0 conn(0x7fe6540b3d80 0x7fe654168300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:32.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.552+0000 7fe658bf2640 1 -- 172.21.15.22:0/656477700 >> v1:172.21.15.22:6789/0 conn(0x7fe6540cef80 legacy=0x7fe6540cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:32.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.552+0000 7fe658bf2640 1 -- 172.21.15.22:0/656477700 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6540b4da0 con 0x7fe6540b55a0 2024-03-29T04:17:32.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.552+0000 7fe658bf2640 1 --2- 172.21.15.22:0/656477700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6540b55a0 0x7fe6540b3840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe6540cb370 tx=0x7fe64401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a65917adad0cb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:32.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe64bfff640 1 -- 172.21.15.22:0/656477700 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440b0070 con 0x7fe6540b55a0 2024-03-29T04:17:32.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe64bfff640 1 -- 172.21.15.22:0/656477700 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6440ab600 con 0x7fe6540b55a0 2024-03-29T04:17:32.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe64bfff640 1 -- 172.21.15.22:0/656477700 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440ab9d0 con 0x7fe6540b55a0 2024-03-29T04:17:32.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe65b657640 1 -- 172.21.15.22:0/656477700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6540b55a0 msgr2=0x7fe6540b3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:32.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe65b657640 1 --2- 172.21.15.22:0/656477700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6540b55a0 0x7fe6540b3840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe6540cb370 tx=0x7fe64401c480 comp rx=0 tx=0).stop 2024-03-29T04:17:32.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe65b657640 1 -- 172.21.15.22:0/656477700 shutdown_connections 2024-03-29T04:17:32.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe65b657640 1 --2- 172.21.15.22:0/656477700 >> v2:172.21.15.151:3300/0 conn(0x7fe6540b3d80 0x7fe654168300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:32.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe65b657640 1 --2- 172.21.15.22:0/656477700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6540b55a0 0x7fe6540b3840 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:32.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe65b657640 1 -- 172.21.15.22:0/656477700 >> 172.21.15.22:0/656477700 conn(0x7fe6540d4310 msgr2=0x7fe6540d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:32.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe65b657640 1 -- 172.21.15.22:0/656477700 shutdown_connections 2024-03-29T04:17:32.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe65b657640 1 -- 172.21.15.22:0/656477700 wait complete. 2024-03-29T04:17:32.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe65b657640 1 Processor -- start 2024-03-29T04:17:32.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe65b657640 1 -- start start 2024-03-29T04:17:32.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe65b657640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6540b3d80 0x7fe654152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:32.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe65b657640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6540b55a0 0x7fe65415e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:32.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe65b657640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6540cef80 0x7fe654153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:32.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe65b657640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe65416b1e0 con 0x7fe6540b55a0 2024-03-29T04:17:32.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe65b657640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe65416b060 con 0x7fe6540b3d80 2024-03-29T04:17:32.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe65b657640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe65416b360 con 0x7fe6540cef80 2024-03-29T04:17:32.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe658bf2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6540b55a0 0x7fe65415e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:32.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe658bf2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6540b55a0 0x7fe65415e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53608/0 (socket says 172.21.15.22:53608) 2024-03-29T04:17:32.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe658bf2640 1 -- 172.21.15.22:0/3406253378 learned_addr learned my addr 172.21.15.22:0/3406253378 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:32.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe658bf2640 1 -- 172.21.15.22:0/3406253378 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6540cef80 msgr2=0x7fe654153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:32.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe658bf2640 1 --2- 172.21.15.22:0/3406253378 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6540cef80 0x7fe654153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:32.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe658bf2640 1 -- 172.21.15.22:0/3406253378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6540b3d80 msgr2=0x7fe654152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:32.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe658bf2640 1 --2- 172.21.15.22:0/3406253378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6540b3d80 0x7fe654152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:32.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe658bf2640 1 -- 172.21.15.22:0/3406253378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6541617d0 con 0x7fe6540b55a0 2024-03-29T04:17:32.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe658bf2640 1 --2- 172.21.15.22:0/3406253378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6540b55a0 0x7fe65415e140 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe6440ae040 tx=0x7fe644019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:32.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe649ffb640 1 -- 172.21.15.22:0/3406253378 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440b0040 con 0x7fe6540b55a0 2024-03-29T04:17:32.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe649ffb640 1 -- 172.21.15.22:0/3406253378 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6440b9070 con 0x7fe6540b55a0 2024-03-29T04:17:32.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe65b657640 1 -- 172.21.15.22:0/3406253378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe654161ab0 con 0x7fe6540b55a0 2024-03-29T04:17:32.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe649ffb640 1 -- 172.21.15.22:0/3406253378 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440b4690 con 0x7fe6540b55a0 2024-03-29T04:17:32.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.556+0000 7fe65b657640 1 -- 172.21.15.22:0/3406253378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe65415f300 con 0x7fe6540b55a0 2024-03-29T04:17:32.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.560+0000 7fe65b657640 1 -- 172.21.15.22:0/3406253378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe65415f540 con 0x7fe6540b55a0 2024-03-29T04:17:32.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.560+0000 7fe649ffb640 1 -- 172.21.15.22:0/3406253378 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe6440b4830 con 0x7fe6540b55a0 2024-03-29T04:17:32.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.560+0000 7fe649ffb640 1 --2- 172.21.15.22:0/3406253378 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe638067fc0 0x7fe63806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:32.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.560+0000 7fe649ffb640 1 -- 172.21.15.22:0/3406253378 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fe64411fff0 con 0x7fe6540b55a0 2024-03-29T04:17:32.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.560+0000 7fe6593f3640 1 --2- 172.21.15.22:0/3406253378 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe638067fc0 0x7fe63806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:32.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.560+0000 7fe6593f3640 1 --2- 172.21.15.22:0/3406253378 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe638067fc0 0x7fe63806a480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe650007fd0 tx=0x7fe650007900 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:32.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.564+0000 7fe649ffb640 1 -- 172.21.15.22:0/3406253378 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fe65415f540 con 0x7fe6540b55a0 2024-03-29T04:17:32.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.764+0000 7fe65b657640 1 -- 172.21.15.22:0/3406253378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe654068890 con 0x7fe6540b55a0 2024-03-29T04:17:32.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.764+0000 7fe649ffb640 1 -- 172.21.15.22:0/3406253378 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3870 (secure 0 0 0) 0x7fe644016070 con 0x7fe6540b55a0 2024-03-29T04:17:32.745 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:32.745 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3094991863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3094991863},{"type":"v1","addr":"172.21.15.22:6834","nonce":3094991863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2464050026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2464050026},{"type":"v1","addr":"172.21.15.151:6835","nonce":2464050026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2999052384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2999052384},{"type":"v1","addr":"172.21.15.22:6835","nonce":2999052384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/1466916671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":1466916671},{"type":"v1","addr":"172.21.15.151:6837","nonce":1466916671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-29T04:17:32.746 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 3 2024-03-29T04:17:32.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.764+0000 7fe65b657640 1 -- 172.21.15.22:0/3406253378 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe638067fc0 msgr2=0x7fe63806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:32.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.764+0000 7fe65b657640 1 --2- 172.21.15.22:0/3406253378 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe638067fc0 0x7fe63806a480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe650007fd0 tx=0x7fe650007900 comp rx=0 tx=0).stop 2024-03-29T04:17:32.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.764+0000 7fe65b657640 1 -- 172.21.15.22:0/3406253378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6540b55a0 msgr2=0x7fe65415e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:32.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.764+0000 7fe65b657640 1 --2- 172.21.15.22:0/3406253378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6540b55a0 0x7fe65415e140 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe6440ae040 tx=0x7fe644019220 comp rx=0 tx=0).stop 2024-03-29T04:17:32.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.764+0000 7fe65b657640 1 -- 172.21.15.22:0/3406253378 shutdown_connections 2024-03-29T04:17:32.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.764+0000 7fe65b657640 1 --2- 172.21.15.22:0/3406253378 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe6540cef80 0x7fe654153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:32.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.764+0000 7fe65b657640 1 --2- 172.21.15.22:0/3406253378 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe638067fc0 0x7fe63806a480 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:32.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.764+0000 7fe65b657640 1 --2- 172.21.15.22:0/3406253378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6540b55a0 0x7fe65415e140 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:32.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.764+0000 7fe65b657640 1 --2- 172.21.15.22:0/3406253378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6540b3d80 0x7fe654152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:32.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.764+0000 7fe65b657640 1 -- 172.21.15.22:0/3406253378 >> 172.21.15.22:0/3406253378 conn(0x7fe6540d4310 msgr2=0x7fe6540d2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:32.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.768+0000 7fe65b657640 1 -- 172.21.15.22:0/3406253378 shutdown_connections 2024-03-29T04:17:32.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.768+0000 7fe65b657640 1 -- 172.21.15.22:0/3406253378 wait complete. 2024-03-29T04:17:32.763 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:17:32.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.892+0000 7f46da1eb640 1 Processor -- start 2024-03-29T04:17:32.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.892+0000 7f46da1eb640 1 -- start start 2024-03-29T04:17:32.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.896+0000 7f46da1eb640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f46d40d1810 0x7f46d40b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:32.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.896+0000 7f46da1eb640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46d40d3e60 con 0x7f46d40b9170 2024-03-29T04:17:32.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.896+0000 7f46da1eb640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46d40b7680 con 0x7f46d415e600 2024-03-29T04:17:32.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.896+0000 7f46da1eb640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f46d40b7450 con 0x7f46d40d1810 2024-03-29T04:17:32.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.896+0000 7f46d3fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f46d40b9170 0x7f46d40d2600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54550/0 (socket says 172.21.15.22:54550) 2024-03-29T04:17:32.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.896+0000 7f46d2ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f46d40d1810 0x7f46d40b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:32.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.896+0000 7f46d3fff640 1 -- 172.21.15.22:0/3832077446 learned_addr learned my addr 172.21.15.22:0/3832077446 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:32.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.896+0000 7f46d2ffd640 1 -- 172.21.15.22:0/3832077446 >> v1:172.21.15.151:6789/0 conn(0x7f46d415e600 legacy=0x7f46d40d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:32.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.896+0000 7f46d2ffd640 1 -- 172.21.15.22:0/3832077446 >> v1:172.21.15.22:6789/0 conn(0x7f46d40b9170 legacy=0x7f46d40d2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:32.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.896+0000 7f46d2ffd640 1 -- 172.21.15.22:0/3832077446 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d40b7960 con 0x7f46d40d1810 2024-03-29T04:17:32.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.896+0000 7f46d2ffd640 1 --2- 172.21.15.22:0/3832077446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d40d1810 0x7f46d40b9f00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f46c80097c0 tx=0x7f46c801c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d838431d4e3df5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:32.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.896+0000 7f46d27fc640 1 -- 172.21.15.22:0/3832077446 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c80b0070 con 0x7f46d40d1810 2024-03-29T04:17:32.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.896+0000 7f46d27fc640 1 -- 172.21.15.22:0/3832077446 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c80ab600 con 0x7f46d40d1810 2024-03-29T04:17:32.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.896+0000 7f46d27fc640 1 -- 172.21.15.22:0/3832077446 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c80ab9d0 con 0x7f46d40d1810 2024-03-29T04:17:32.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46da1eb640 1 -- 172.21.15.22:0/3832077446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d40d1810 msgr2=0x7f46d40b9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:32.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46da1eb640 1 --2- 172.21.15.22:0/3832077446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d40d1810 0x7f46d40b9f00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f46c80097c0 tx=0x7f46c801c2a0 comp rx=0 tx=0).stop 2024-03-29T04:17:32.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46da1eb640 1 -- 172.21.15.22:0/3832077446 shutdown_connections 2024-03-29T04:17:32.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46da1eb640 1 --2- 172.21.15.22:0/3832077446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d40d1810 0x7f46d40b9f00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:32.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46da1eb640 1 -- 172.21.15.22:0/3832077446 >> 172.21.15.22:0/3832077446 conn(0x7f46d40b6230 msgr2=0x7f46d40b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:32.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46da1eb640 1 -- 172.21.15.22:0/3832077446 shutdown_connections 2024-03-29T04:17:32.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46da1eb640 1 -- 172.21.15.22:0/3832077446 wait complete. 2024-03-29T04:17:32.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46da1eb640 1 Processor -- start 2024-03-29T04:17:32.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46da1eb640 1 -- start start 2024-03-29T04:17:32.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46da1eb640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f46d40b9170 0x7f46d4151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:32.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46da1eb640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46d40d1810 0x7f46d41523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:32.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46da1eb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d415e600 0x7f46d40c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:32.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46da1eb640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f46d416bb70 con 0x7f46d415e600 2024-03-29T04:17:32.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46da1eb640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f46d416b9f0 con 0x7f46d40d1810 2024-03-29T04:17:32.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46da1eb640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f46d416bcf0 con 0x7f46d40b9170 2024-03-29T04:17:32.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d415e600 0x7f46d40c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:32.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d415e600 0x7f46d40c8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53628/0 (socket says 172.21.15.22:53628) 2024-03-29T04:17:32.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46d3fff640 1 -- 172.21.15.22:0/2346412705 learned_addr learned my addr 172.21.15.22:0/2346412705 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:32.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46d2ffd640 1 --2- 172.21.15.22:0/2346412705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46d40d1810 0x7f46d41523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:32.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46d3fff640 1 -- 172.21.15.22:0/2346412705 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f46d40b9170 msgr2=0x7f46d4151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:32.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46d37fe640 1 --2- 172.21.15.22:0/2346412705 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f46d40b9170 0x7f46d4151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:32.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46d3fff640 1 --2- 172.21.15.22:0/2346412705 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f46d40b9170 0x7f46d4151e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:32.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46d3fff640 1 -- 172.21.15.22:0/2346412705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46d40d1810 msgr2=0x7f46d41523d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:32.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46d3fff640 1 --2- 172.21.15.22:0/2346412705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46d40d1810 0x7f46d41523d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:32.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46d3fff640 1 -- 172.21.15.22:0/2346412705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d4148c90 con 0x7f46d415e600 2024-03-29T04:17:32.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46d2ffd640 1 --2- 172.21.15.22:0/2346412705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46d40d1810 0x7f46d41523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:32.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46d3fff640 1 --2- 172.21.15.22:0/2346412705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d415e600 0x7f46d40c8160 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00caf0 tx=0x7f46cc002080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:32.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46bbfff640 1 -- 172.21.15.22:0/2346412705 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc018070 con 0x7f46d415e600 2024-03-29T04:17:32.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46bbfff640 1 -- 172.21.15.22:0/2346412705 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46cc00a700 con 0x7f46d415e600 2024-03-29T04:17:32.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46da1eb640 1 -- 172.21.15.22:0/2346412705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d4148f70 con 0x7f46d415e600 2024-03-29T04:17:32.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46bbfff640 1 -- 172.21.15.22:0/2346412705 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc00aaf0 con 0x7f46d415e600 2024-03-29T04:17:32.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46da1eb640 1 -- 172.21.15.22:0/2346412705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d4149530 con 0x7f46d415e600 2024-03-29T04:17:32.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.900+0000 7f46da1eb640 1 -- 172.21.15.22:0/2346412705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d4068890 con 0x7f46d415e600 2024-03-29T04:17:32.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.904+0000 7f46bbfff640 1 -- 172.21.15.22:0/2346412705 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f46cc01b070 con 0x7f46d415e600 2024-03-29T04:17:32.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.904+0000 7f46bbfff640 1 --2- 172.21.15.22:0/2346412705 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f46c0067fc0 0x7f46c006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:32.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.904+0000 7f46bbfff640 1 -- 172.21.15.22:0/2346412705 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f46cc0858c0 con 0x7f46d415e600 2024-03-29T04:17:32.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.904+0000 7f46d37fe640 1 --2- 172.21.15.22:0/2346412705 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f46c0067fc0 0x7f46c006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:32.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.904+0000 7f46d37fe640 1 --2- 172.21.15.22:0/2346412705 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f46c0067fc0 0x7f46c006a480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f46c4007fd0 tx=0x7f46c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:32.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:32.908+0000 7f46bbfff640 1 -- 172.21.15.22:0/2346412705 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f46cc056450 con 0x7f46d415e600 2024-03-29T04:17:33.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.068+0000 7f46da1eb640 1 -- 172.21.15.22:0/2346412705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f46d4074ad0 con 0x7f46d415e600 2024-03-29T04:17:33.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.068+0000 7f46bbfff640 1 -- 172.21.15.22:0/2346412705 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12884 (secure 0 0 0) 0x7f46cc055d50 con 0x7f46d415e600 2024-03-29T04:17:33.051 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:33.051 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:17:26.357154+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:17:33.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.072+0000 7f46da1eb640 1 -- 172.21.15.22:0/2346412705 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f46c0067fc0 msgr2=0x7f46c006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:33.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.072+0000 7f46da1eb640 1 --2- 172.21.15.22:0/2346412705 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f46c0067fc0 0x7f46c006a480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f46c4007fd0 tx=0x7f46c4009040 comp rx=0 tx=0).stop 2024-03-29T04:17:33.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.072+0000 7f46da1eb640 1 -- 172.21.15.22:0/2346412705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d415e600 msgr2=0x7f46d40c8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:33.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.072+0000 7f46da1eb640 1 --2- 172.21.15.22:0/2346412705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d415e600 0x7f46d40c8160 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00caf0 tx=0x7f46cc002080 comp rx=0 tx=0).stop 2024-03-29T04:17:33.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.072+0000 7f46da1eb640 1 -- 172.21.15.22:0/2346412705 shutdown_connections 2024-03-29T04:17:33.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.072+0000 7f46da1eb640 1 --2- 172.21.15.22:0/2346412705 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f46c0067fc0 0x7f46c006a480 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.072+0000 7f46da1eb640 1 --2- 172.21.15.22:0/2346412705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46d415e600 0x7f46d40c8160 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.072+0000 7f46da1eb640 1 --2- 172.21.15.22:0/2346412705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46d40d1810 0x7f46d41523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.072+0000 7f46da1eb640 1 --2- 172.21.15.22:0/2346412705 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f46d40b9170 0x7f46d4151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.072+0000 7f46da1eb640 1 -- 172.21.15.22:0/2346412705 >> 172.21.15.22:0/2346412705 conn(0x7f46d40b6230 msgr2=0x7f46d4152f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:33.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.072+0000 7f46da1eb640 1 -- 172.21.15.22:0/2346412705 shutdown_connections 2024-03-29T04:17:33.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.072+0000 7f46da1eb640 1 -- 172.21.15.22:0/2346412705 wait complete. 2024-03-29T04:17:33.069 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:17:33.069 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:17:33.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.216+0000 7fc03199f640 1 Processor -- start 2024-03-29T04:17:33.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.216+0000 7fc03199f640 1 -- start start 2024-03-29T04:17:33.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.216+0000 7fc03199f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc02c144930 0x7fc02c0b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:33.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.216+0000 7fc03199f640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc02c06cd80 0x7fc02c0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:33.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.216+0000 7fc03199f640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc02c0b33c0 con 0x7fc02c0b4870 2024-03-29T04:17:33.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.216+0000 7fc03199f640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fc02c0cdb90 con 0x7fc02c144930 2024-03-29T04:17:33.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.216+0000 7fc03199f640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fc02c0b3660 con 0x7fc02c06cd80 2024-03-29T04:17:33.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.216+0000 7fc023fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fc02c0b4870 0x7fc02c06c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54554/0 (socket says 172.21.15.22:54554) 2024-03-29T04:17:33.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.216+0000 7fc023fff640 1 -- 172.21.15.22:0/2404327409 learned_addr learned my addr 172.21.15.22:0/2404327409 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:33.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.216+0000 7fc02affd640 1 --2- 172.21.15.22:0/2404327409 >> v2:172.21.15.22:3300/0 conn(0x7fc02c144930 0x7fc02c0b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:33.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.216+0000 7fc02b7fe640 1 --2- 172.21.15.22:0/2404327409 >> v2:172.21.15.151:3300/0 conn(0x7fc02c06cd80 0x7fc02c0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:33.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.216+0000 7fc02a7fc640 1 -- 172.21.15.22:0/2404327409 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2806262210 0 0) 0x7fc02c0b33c0 con 0x7fc02c0b4870 2024-03-29T04:17:33.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.216+0000 7fc02a7fc640 1 -- 172.21.15.22:0/2404327409 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc018003530 con 0x7fc02c0b4870 2024-03-29T04:17:33.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.216+0000 7fc02affd640 1 -- 172.21.15.22:0/2404327409 >> v2:172.21.15.151:3300/0 conn(0x7fc02c06cd80 msgr2=0x7fc02c0ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:33.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.216+0000 7fc02affd640 1 --2- 172.21.15.22:0/2404327409 >> v2:172.21.15.151:3300/0 conn(0x7fc02c06cd80 0x7fc02c0ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.216+0000 7fc02affd640 1 -- 172.21.15.22:0/2404327409 >> v1:172.21.15.22:6789/0 conn(0x7fc02c0b4870 legacy=0x7fc02c06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:33.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.216+0000 7fc02affd640 1 -- 172.21.15.22:0/2404327409 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02c0b3940 con 0x7fc02c144930 2024-03-29T04:17:33.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc02affd640 1 --2- 172.21.15.22:0/2404327409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc02c144930 0x7fc02c0b2e80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0cdd10 tx=0x7fc01c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f532ab37ce483c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:33.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc02a7fc640 1 -- 172.21.15.22:0/2404327409 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c0b2070 con 0x7fc02c144930 2024-03-29T04:17:33.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc02a7fc640 1 -- 172.21.15.22:0/2404327409 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc01c0a9070 con 0x7fc02c144930 2024-03-29T04:17:33.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc02a7fc640 1 -- 172.21.15.22:0/2404327409 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c0ad860 con 0x7fc02c144930 2024-03-29T04:17:33.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc03199f640 1 -- 172.21.15.22:0/2404327409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc02c144930 msgr2=0x7fc02c0b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:33.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc03199f640 1 --2- 172.21.15.22:0/2404327409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc02c144930 0x7fc02c0b2e80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0cdd10 tx=0x7fc01c01c3a0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc03199f640 1 -- 172.21.15.22:0/2404327409 shutdown_connections 2024-03-29T04:17:33.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc03199f640 1 --2- 172.21.15.22:0/2404327409 >> v2:172.21.15.151:3300/0 conn(0x7fc02c06cd80 0x7fc02c0ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc03199f640 1 --2- 172.21.15.22:0/2404327409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc02c144930 0x7fc02c0b2e80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc03199f640 1 -- 172.21.15.22:0/2404327409 >> 172.21.15.22:0/2404327409 conn(0x7fc02c0d00a0 msgr2=0x7fc02c0d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:33.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc03199f640 1 -- 172.21.15.22:0/2404327409 shutdown_connections 2024-03-29T04:17:33.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc03199f640 1 -- 172.21.15.22:0/2404327409 wait complete. 2024-03-29T04:17:33.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc03199f640 1 Processor -- start 2024-03-29T04:17:33.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc03199f640 1 -- start start 2024-03-29T04:17:33.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc03199f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc02c06cd80 0x7fc02c0afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:33.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc03199f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc02c0b4870 0x7fc02c15d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:33.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc03199f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc02c144930 0x7fc02c15a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:33.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc03199f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c0ceef0 con 0x7fc02c144930 2024-03-29T04:17:33.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc03199f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c0ced70 con 0x7fc02c0b4870 2024-03-29T04:17:33.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc03199f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc02c0cf070 con 0x7fc02c06cd80 2024-03-29T04:17:33.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc02b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc02c144930 0x7fc02c15a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:33.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc02b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc02c144930 0x7fc02c15a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53644/0 (socket says 172.21.15.22:53644) 2024-03-29T04:17:33.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc02b7fe640 1 -- 172.21.15.22:0/3860999573 learned_addr learned my addr 172.21.15.22:0/3860999573 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:33.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc023fff640 1 --2- 172.21.15.22:0/3860999573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc02c0b4870 0x7fc02c15d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:33.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc02affd640 1 --2- 172.21.15.22:0/3860999573 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc02c06cd80 0x7fc02c0afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:33.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc02b7fe640 1 -- 172.21.15.22:0/3860999573 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc02c06cd80 msgr2=0x7fc02c0afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:33.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc02b7fe640 1 --2- 172.21.15.22:0/3860999573 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc02c06cd80 0x7fc02c0afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc02b7fe640 1 -- 172.21.15.22:0/3860999573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc02c0b4870 msgr2=0x7fc02c15d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:33.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc02b7fe640 1 --2- 172.21.15.22:0/3860999573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc02c0b4870 0x7fc02c15d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc02b7fe640 1 -- 172.21.15.22:0/3860999573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02c06c0c0 con 0x7fc02c144930 2024-03-29T04:17:33.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc023fff640 1 --2- 172.21.15.22:0/3860999573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc02c0b4870 0x7fc02c15d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:33.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc02affd640 1 --2- 172.21.15.22:0/3860999573 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc02c06cd80 0x7fc02c0afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:33.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc02b7fe640 1 --2- 172.21.15.22:0/3860999573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc02c144930 0x7fc02c15a620 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc024007e60 tx=0x7fc02400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:33.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc028ff9640 1 -- 172.21.15.22:0/3860999573 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024014620 con 0x7fc02c144930 2024-03-29T04:17:33.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc028ff9640 1 -- 172.21.15.22:0/3860999573 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc024019070 con 0x7fc02c144930 2024-03-29T04:17:33.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc03199f640 1 -- 172.21.15.22:0/3860999573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02c15da60 con 0x7fc02c144930 2024-03-29T04:17:33.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc028ff9640 1 -- 172.21.15.22:0/3860999573 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024014a00 con 0x7fc02c144930 2024-03-29T04:17:33.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.220+0000 7fc03199f640 1 -- 172.21.15.22:0/3860999573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc02c15dfe0 con 0x7fc02c144930 2024-03-29T04:17:33.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.224+0000 7fc03199f640 1 -- 172.21.15.22:0/3860999573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc02c068900 con 0x7fc02c144930 2024-03-29T04:17:33.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.224+0000 7fc028ff9640 1 -- 172.21.15.22:0/3860999573 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fc024014ba0 con 0x7fc02c144930 2024-03-29T04:17:33.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.224+0000 7fc028ff9640 1 --2- 172.21.15.22:0/3860999573 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc00c067fc0 0x7fc00c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:33.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.224+0000 7fc028ff9640 1 -- 172.21.15.22:0/3860999573 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fc024086a20 con 0x7fc02c144930 2024-03-29T04:17:33.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.224+0000 7fc02affd640 1 --2- 172.21.15.22:0/3860999573 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc00c067fc0 0x7fc00c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:33.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.224+0000 7fc02affd640 1 --2- 172.21.15.22:0/3860999573 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc00c067fc0 0x7fc00c06a480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0af000 tx=0x7fc01c0a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:33.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.228+0000 7fc028ff9640 1 -- 172.21.15.22:0/3860999573 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fc024011260 con 0x7fc02c144930 2024-03-29T04:17:33.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.388+0000 7fc03199f640 1 -- 172.21.15.22:0/3860999573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc02c0aeda0 con 0x7fc02c144930 2024-03-29T04:17:33.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.392+0000 7fc028ff9640 1 -- 172.21.15.22:0/3860999573 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7fc0240575b0 con 0x7fc02c144930 2024-03-29T04:17:33.372 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:17:33.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.392+0000 7fc03199f640 1 -- 172.21.15.22:0/3860999573 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc00c067fc0 msgr2=0x7fc00c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:33.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.392+0000 7fc03199f640 1 --2- 172.21.15.22:0/3860999573 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc00c067fc0 0x7fc00c06a480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0af000 tx=0x7fc01c0a7000 comp rx=0 tx=0).stop 2024-03-29T04:17:33.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.392+0000 7fc03199f640 1 -- 172.21.15.22:0/3860999573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc02c144930 msgr2=0x7fc02c15a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:33.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.392+0000 7fc03199f640 1 --2- 172.21.15.22:0/3860999573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc02c144930 0x7fc02c15a620 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc024007e60 tx=0x7fc02400e4b0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.392+0000 7fc03199f640 1 -- 172.21.15.22:0/3860999573 shutdown_connections 2024-03-29T04:17:33.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.392+0000 7fc03199f640 1 --2- 172.21.15.22:0/3860999573 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc00c067fc0 0x7fc00c06a480 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.392+0000 7fc03199f640 1 --2- 172.21.15.22:0/3860999573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc02c144930 0x7fc02c15a620 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.392+0000 7fc03199f640 1 --2- 172.21.15.22:0/3860999573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc02c0b4870 0x7fc02c15d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.392+0000 7fc03199f640 1 --2- 172.21.15.22:0/3860999573 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc02c06cd80 0x7fc02c0afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.392+0000 7fc03199f640 1 -- 172.21.15.22:0/3860999573 >> 172.21.15.22:0/3860999573 conn(0x7fc02c0d00a0 msgr2=0x7fc02c06b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:33.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.392+0000 7fc03199f640 1 -- 172.21.15.22:0/3860999573 shutdown_connections 2024-03-29T04:17:33.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.392+0000 7fc03199f640 1 -- 172.21.15.22:0/3860999573 wait complete. 2024-03-29T04:17:33.390 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-29T04:17:33.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.528+0000 7f535d0c3640 1 Processor -- start 2024-03-29T04:17:33.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.528+0000 7f535d0c3640 1 -- start start 2024-03-29T04:17:33.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.532+0000 7f535d0c3640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f53580b5b20 0x7f535806d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:33.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.532+0000 7f535d0c3640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f535806dcc0 0x7f535806e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:33.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.532+0000 7f535d0c3640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f535815edd0 con 0x7f5358070490 2024-03-29T04:17:33.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.532+0000 7f535d0c3640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f535815f0b0 con 0x7f535806dcc0 2024-03-29T04:17:33.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.532+0000 7f535d0c3640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f535815f390 con 0x7f53580b5b20 2024-03-29T04:17:33.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.532+0000 7f535759e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f535806dcc0 0x7f535806e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:33.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.532+0000 7f5356d9d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f5358070490 0x7f535815c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:54570/0 (socket says 172.21.15.22:54570) 2024-03-29T04:17:33.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.532+0000 7f5356d9d640 1 -- 172.21.15.22:0/2473032549 learned_addr learned my addr 172.21.15.22:0/2473032549 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:33.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.532+0000 7f535659c640 1 --2- 172.21.15.22:0/2473032549 >> v2:172.21.15.151:3300/0 conn(0x7f53580b5b20 0x7f535806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:33.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.532+0000 7f5355d9b640 1 -- 172.21.15.22:0/2473032549 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3463122309 0 0) 0x7f535815edd0 con 0x7f5358070490 2024-03-29T04:17:33.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.532+0000 7f5355d9b640 1 -- 172.21.15.22:0/2473032549 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5340003530 con 0x7f5358070490 2024-03-29T04:17:33.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.532+0000 7f535759e640 1 -- 172.21.15.22:0/2473032549 >> v2:172.21.15.151:3300/0 conn(0x7f53580b5b20 msgr2=0x7f535806d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:33.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.532+0000 7f535759e640 1 --2- 172.21.15.22:0/2473032549 >> v2:172.21.15.151:3300/0 conn(0x7f53580b5b20 0x7f535806d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.532+0000 7f535759e640 1 -- 172.21.15.22:0/2473032549 >> v1:172.21.15.22:6789/0 conn(0x7f5358070490 legacy=0x7f535815c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:33.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.532+0000 7f535759e640 1 -- 172.21.15.22:0/2473032549 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f535815f670 con 0x7f535806dcc0 2024-03-29T04:17:33.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.532+0000 7f535659c640 1 --2- 172.21.15.22:0/2473032549 >> v2:172.21.15.151:3300/0 conn(0x7f53580b5b20 0x7f535806d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:17:33.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535759e640 1 --2- 172.21.15.22:0/2473032549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f535806dcc0 0x7f535806e810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5358073d50 tx=0x7f535001c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9716a7d244d635 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:33.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f5355d9b640 1 -- 172.21.15.22:0/2473032549 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53500a66e0 con 0x7f535806dcc0 2024-03-29T04:17:33.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f5355d9b640 1 -- 172.21.15.22:0/2473032549 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53500a6880 con 0x7f535806dcc0 2024-03-29T04:17:33.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f5355d9b640 1 -- 172.21.15.22:0/2473032549 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53500a6c50 con 0x7f535806dcc0 2024-03-29T04:17:33.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535d0c3640 1 -- 172.21.15.22:0/2473032549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f535806dcc0 msgr2=0x7f535806e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:33.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535d0c3640 1 --2- 172.21.15.22:0/2473032549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f535806dcc0 0x7f535806e810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5358073d50 tx=0x7f535001c810 comp rx=0 tx=0).stop 2024-03-29T04:17:33.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535d0c3640 1 -- 172.21.15.22:0/2473032549 shutdown_connections 2024-03-29T04:17:33.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535d0c3640 1 --2- 172.21.15.22:0/2473032549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f535806dcc0 0x7f535806e810 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535d0c3640 1 --2- 172.21.15.22:0/2473032549 >> v2:172.21.15.151:3300/0 conn(0x7f53580b5b20 0x7f535806d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535d0c3640 1 -- 172.21.15.22:0/2473032549 >> 172.21.15.22:0/2473032549 conn(0x7f5358072500 msgr2=0x7f5358072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:33.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535d0c3640 1 -- 172.21.15.22:0/2473032549 shutdown_connections 2024-03-29T04:17:33.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535d0c3640 1 -- 172.21.15.22:0/2473032549 wait complete. 2024-03-29T04:17:33.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535d0c3640 1 Processor -- start 2024-03-29T04:17:33.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535d0c3640 1 -- start start 2024-03-29T04:17:33.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535d0c3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f535806dcc0 0x7f535814f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:33.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535d0c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5358070490 0x7f535814ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:33.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535d0c3640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53580b5b20 0x7f53580b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:33.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535d0c3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f535806f8f0 con 0x7f5358070490 2024-03-29T04:17:33.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535d0c3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f535806f770 con 0x7f535806dcc0 2024-03-29T04:17:33.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535d0c3640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f535806fa70 con 0x7f53580b5b20 2024-03-29T04:17:33.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5358070490 0x7f535814ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:33.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5358070490 0x7f535814ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53660/0 (socket says 172.21.15.22:53660) 2024-03-29T04:17:33.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535659c640 1 -- 172.21.15.22:0/3070678445 learned_addr learned my addr 172.21.15.22:0/3070678445 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:33.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f5356d9d640 1 --2- 172.21.15.22:0/3070678445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f535806dcc0 0x7f535814f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:33.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535659c640 1 -- 172.21.15.22:0/3070678445 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53580b5b20 msgr2=0x7f53580b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:33.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535659c640 1 --2- 172.21.15.22:0/3070678445 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53580b5b20 0x7f53580b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535659c640 1 -- 172.21.15.22:0/3070678445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f535806dcc0 msgr2=0x7f535814f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:33.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535659c640 1 --2- 172.21.15.22:0/3070678445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f535806dcc0 0x7f535814f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535659c640 1 -- 172.21.15.22:0/3070678445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f535814fe10 con 0x7f5358070490 2024-03-29T04:17:33.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f5356d9d640 1 --2- 172.21.15.22:0/3070678445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f535806dcc0 0x7f535814f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:33.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535659c640 1 --2- 172.21.15.22:0/3070678445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5358070490 0x7f535814ef80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f534c00d7d0 tx=0x7f534c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:33.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f533f7fe640 1 -- 172.21.15.22:0/3070678445 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c00b660 con 0x7f5358070490 2024-03-29T04:17:33.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f533f7fe640 1 -- 172.21.15.22:0/3070678445 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f534c019070 con 0x7f5358070490 2024-03-29T04:17:33.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535d0c3640 1 -- 172.21.15.22:0/3070678445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53581515f0 con 0x7f5358070490 2024-03-29T04:17:33.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f533f7fe640 1 -- 172.21.15.22:0/3070678445 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c00ba40 con 0x7f5358070490 2024-03-29T04:17:33.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.536+0000 7f535d0c3640 1 -- 172.21.15.22:0/3070678445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f535815d5c0 con 0x7f5358070490 2024-03-29T04:17:33.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.540+0000 7f535d0c3640 1 -- 172.21.15.22:0/3070678445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5358068890 con 0x7f5358070490 2024-03-29T04:17:33.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.540+0000 7f533f7fe640 1 -- 172.21.15.22:0/3070678445 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f534c00bbe0 con 0x7f5358070490 2024-03-29T04:17:33.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.540+0000 7f533f7fe640 1 --2- 172.21.15.22:0/3070678445 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5344067fc0 0x7f534406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:33.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.540+0000 7f533f7fe640 1 -- 172.21.15.22:0/3070678445 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f534c086580 con 0x7f5358070490 2024-03-29T04:17:33.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.540+0000 7f5356d9d640 1 --2- 172.21.15.22:0/3070678445 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5344067fc0 0x7f534406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:33.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.540+0000 7f5356d9d640 1 --2- 172.21.15.22:0/3070678445 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5344067fc0 0x7f534406a480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f53580cd350 tx=0x7f5348009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:33.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.544+0000 7f533f7fe640 1 -- 172.21.15.22:0/3070678445 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f535814fe10 con 0x7f5358070490 2024-03-29T04:17:33.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.708+0000 7f535d0c3640 1 -- 172.21.15.22:0/3070678445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f53580b0540 con 0x7f5358070490 2024-03-29T04:17:33.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.708+0000 7f533f7fe640 1 -- 172.21.15.22:0/3070678445 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f534c057070 con 0x7f5358070490 2024-03-29T04:17:33.689 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:33.690 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2024-03-29T04:17:33.690 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:17:33.690 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 1, 2024-03-29T04:17:33.690 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": ".mgr" 2024-03-29T04:17:33.690 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:17:33.690 INFO:teuthology.orchestra.run.smithi022.stdout:] 2024-03-29T04:17:33.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.712+0000 7f535d0c3640 1 -- 172.21.15.22:0/3070678445 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5344067fc0 msgr2=0x7f534406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:33.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.712+0000 7f535d0c3640 1 --2- 172.21.15.22:0/3070678445 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5344067fc0 0x7f534406a480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f53580cd350 tx=0x7f5348009040 comp rx=0 tx=0).stop 2024-03-29T04:17:33.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.712+0000 7f535d0c3640 1 -- 172.21.15.22:0/3070678445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5358070490 msgr2=0x7f535814ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:33.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.712+0000 7f535d0c3640 1 --2- 172.21.15.22:0/3070678445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5358070490 0x7f535814ef80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f534c00d7d0 tx=0x7f534c00dc90 comp rx=0 tx=0).stop 2024-03-29T04:17:33.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.712+0000 7f535d0c3640 1 -- 172.21.15.22:0/3070678445 shutdown_connections 2024-03-29T04:17:33.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.712+0000 7f535d0c3640 1 --2- 172.21.15.22:0/3070678445 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53580b5b20 0x7f53580b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.712+0000 7f535d0c3640 1 --2- 172.21.15.22:0/3070678445 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5344067fc0 0x7f534406a480 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.712+0000 7f535d0c3640 1 --2- 172.21.15.22:0/3070678445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5358070490 0x7f535814ef80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.712+0000 7f535d0c3640 1 --2- 172.21.15.22:0/3070678445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f535806dcc0 0x7f535814f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.712+0000 7f535d0c3640 1 -- 172.21.15.22:0/3070678445 >> 172.21.15.22:0/3070678445 conn(0x7f5358072500 msgr2=0x7f53580b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:33.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.712+0000 7f535d0c3640 1 -- 172.21.15.22:0/3070678445 shutdown_connections 2024-03-29T04:17:33.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.712+0000 7f535d0c3640 1 -- 172.21.15.22:0/3070678445 wait complete. 2024-03-29T04:17:33.709 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-29T04:17:33.709 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-29T04:17:33.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.848+0000 7fb094c56640 1 Processor -- start 2024-03-29T04:17:33.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.848+0000 7fb094c56640 1 -- start start 2024-03-29T04:17:33.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.848+0000 7fb094c56640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0900b5550 0x7fb090160610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:33.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.848+0000 7fb094c56640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fb0900b5930 0x7fb09015c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:33.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.848+0000 7fb094c56640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb090073490 0x7fb09006c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:33.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.848+0000 7fb094c56640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fb09015d090 con 0x7fb090073490 2024-03-29T04:17:33.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.848+0000 7fb094c56640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fb09015d370 con 0x7fb0900b5930 2024-03-29T04:17:33.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.848+0000 7fb094c56640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb09015fe10 con 0x7fb0900b5550 2024-03-29T04:17:33.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.848+0000 7fb08ed9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb090073490 0x7fb09006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:33.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.848+0000 7fb08ed9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb090073490 0x7fb09006c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53674/0 (socket says 172.21.15.22:53674) 2024-03-29T04:17:33.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.848+0000 7fb08ed9d640 1 -- 172.21.15.22:0/2414423090 learned_addr learned my addr 172.21.15.22:0/2414423090 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:33.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.848+0000 7fb08dd9b640 1 --2- 172.21.15.22:0/2414423090 >> v2:172.21.15.151:3300/0 conn(0x7fb0900b5930 0x7fb09015c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:33.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.848+0000 7fb08e59c640 1 --2- 172.21.15.22:0/2414423090 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0900b5550 0x7fb090160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:33.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.848+0000 7fb08ed9d640 1 -- 172.21.15.22:0/2414423090 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0900b5550 msgr2=0x7fb090160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:33.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.848+0000 7fb08ed9d640 1 --2- 172.21.15.22:0/2414423090 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0900b5550 0x7fb090160610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.848+0000 7fb08ed9d640 1 -- 172.21.15.22:0/2414423090 >> v2:172.21.15.151:3300/0 conn(0x7fb0900b5930 msgr2=0x7fb09015c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:33.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.848+0000 7fb08ed9d640 1 --2- 172.21.15.22:0/2414423090 >> v2:172.21.15.151:3300/0 conn(0x7fb0900b5930 0x7fb09015c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.848+0000 7fb08ed9d640 1 -- 172.21.15.22:0/2414423090 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0901600f0 con 0x7fb090073490 2024-03-29T04:17:33.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.848+0000 7fb08dd9b640 1 --2- 172.21.15.22:0/2414423090 >> v2:172.21.15.151:3300/0 conn(0x7fb0900b5930 0x7fb09015c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:33.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.848+0000 7fb08e59c640 1 --2- 172.21.15.22:0/2414423090 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0900b5550 0x7fb090160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:33.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.852+0000 7fb08ed9d640 1 --2- 172.21.15.22:0/2414423090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb090073490 0x7fb09006c670 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb0900cd8d0 tx=0x7fb08801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae94c46e9792140 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:33.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.852+0000 7fb08d59a640 1 -- 172.21.15.22:0/2414423090 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0880a56c0 con 0x7fb090073490 2024-03-29T04:17:33.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.852+0000 7fb08d59a640 1 -- 172.21.15.22:0/2414423090 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0880a5860 con 0x7fb090073490 2024-03-29T04:17:33.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.852+0000 7fb08d59a640 1 -- 172.21.15.22:0/2414423090 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0880a5c30 con 0x7fb090073490 2024-03-29T04:17:33.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.852+0000 7fb094c56640 1 -- 172.21.15.22:0/2414423090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb090073490 msgr2=0x7fb09006c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:33.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.852+0000 7fb094c56640 1 --2- 172.21.15.22:0/2414423090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb090073490 0x7fb09006c670 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb0900cd8d0 tx=0x7fb08801c930 comp rx=0 tx=0).stop 2024-03-29T04:17:33.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.852+0000 7fb094c56640 1 -- 172.21.15.22:0/2414423090 shutdown_connections 2024-03-29T04:17:33.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.852+0000 7fb094c56640 1 --2- 172.21.15.22:0/2414423090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb090073490 0x7fb09006c670 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.852+0000 7fb094c56640 1 --2- 172.21.15.22:0/2414423090 >> v2:172.21.15.151:3300/0 conn(0x7fb0900b5930 0x7fb09015c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.852+0000 7fb094c56640 1 --2- 172.21.15.22:0/2414423090 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0900b5550 0x7fb090160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.852+0000 7fb094c56640 1 -- 172.21.15.22:0/2414423090 >> 172.21.15.22:0/2414423090 conn(0x7fb0900b1b80 msgr2=0x7fb0900b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:33.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.852+0000 7fb094c56640 1 -- 172.21.15.22:0/2414423090 shutdown_connections 2024-03-29T04:17:33.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.852+0000 7fb094c56640 1 -- 172.21.15.22:0/2414423090 wait complete. 2024-03-29T04:17:33.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.852+0000 7fb094c56640 1 Processor -- start 2024-03-29T04:17:33.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb094c56640 1 -- start start 2024-03-29T04:17:33.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb094c56640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb090073490 0x7fb090142a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:33.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb094c56640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0900b5550 0x7fb090152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:33.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb094c56640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0900b5930 0x7fb090158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:33.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb094c56640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb09006d7f0 con 0x7fb090073490 2024-03-29T04:17:33.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb094c56640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb09006d670 con 0x7fb0900b5930 2024-03-29T04:17:33.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb094c56640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb09006d970 con 0x7fb0900b5550 2024-03-29T04:17:33.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb08e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb090073490 0x7fb090142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:33.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb08e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb090073490 0x7fb090142a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53676/0 (socket says 172.21.15.22:53676) 2024-03-29T04:17:33.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb08e59c640 1 -- 172.21.15.22:0/646276137 learned_addr learned my addr 172.21.15.22:0/646276137 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:33.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb08dd9b640 1 --2- 172.21.15.22:0/646276137 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0900b5550 0x7fb090152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:33.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb08ed9d640 1 --2- 172.21.15.22:0/646276137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0900b5930 0x7fb090158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:33.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb08e59c640 1 -- 172.21.15.22:0/646276137 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0900b5550 msgr2=0x7fb090152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:33.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb08e59c640 1 --2- 172.21.15.22:0/646276137 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0900b5550 0x7fb090152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb08e59c640 1 -- 172.21.15.22:0/646276137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0900b5930 msgr2=0x7fb090158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:33.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb08e59c640 1 --2- 172.21.15.22:0/646276137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0900b5930 0x7fb090158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:33.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb08e59c640 1 -- 172.21.15.22:0/646276137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb090142f50 con 0x7fb090073490 2024-03-29T04:17:33.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb08dd9b640 1 --2- 172.21.15.22:0/646276137 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0900b5550 0x7fb090152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:33.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb08ed9d640 1 --2- 172.21.15.22:0/646276137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0900b5930 0x7fb090158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:33.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb08e59c640 1 --2- 172.21.15.22:0/646276137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb090073490 0x7fb090142a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb08000c7c0 tx=0x7fb08000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:33.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb07effd640 1 -- 172.21.15.22:0/646276137 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080007a30 con 0x7fb090073490 2024-03-29T04:17:33.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb07effd640 1 -- 172.21.15.22:0/646276137 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb080007bd0 con 0x7fb090073490 2024-03-29T04:17:33.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb07effd640 1 -- 172.21.15.22:0/646276137 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0800113f0 con 0x7fb090073490 2024-03-29T04:17:33.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb094c56640 1 -- 172.21.15.22:0/646276137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb090153710 con 0x7fb090073490 2024-03-29T04:17:33.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb094c56640 1 -- 172.21.15.22:0/646276137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb090158530 con 0x7fb090073490 2024-03-29T04:17:33.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb094c56640 1 -- 172.21.15.22:0/646276137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb090158970 con 0x7fb090073490 2024-03-29T04:17:33.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb07effd640 1 -- 172.21.15.22:0/646276137 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb080011590 con 0x7fb090073490 2024-03-29T04:17:33.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb07effd640 1 --2- 172.21.15.22:0/646276137 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb070067fc0 0x7fb07006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:33.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb07effd640 1 -- 172.21.15.22:0/646276137 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fb080086360 con 0x7fb090073490 2024-03-29T04:17:33.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.856+0000 7fb08dd9b640 1 --2- 172.21.15.22:0/646276137 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb070067fc0 0x7fb07006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:33.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.860+0000 7fb08dd9b640 1 --2- 172.21.15.22:0/646276137 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb070067fc0 0x7fb07006a480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb090152a20 tx=0x7fb084008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:33.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:33.860+0000 7fb07effd640 1 -- 172.21.15.22:0/646276137 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fb080015020 con 0x7fb090073490 2024-03-29T04:17:34.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.024+0000 7fb094c56640 1 -- 172.21.15.22:0/646276137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb0900687f0 con 0x7fb090073490 2024-03-29T04:17:34.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.532+0000 7fb07effd640 1 -- 172.21.15.22:0/646276137 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7fb080056ef0 con 0x7fb090073490 2024-03-29T04:17:34.515 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' created 2024-03-29T04:17:34.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.536+0000 7fb094c56640 1 -- 172.21.15.22:0/646276137 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb070067fc0 msgr2=0x7fb07006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:34.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.536+0000 7fb094c56640 1 --2- 172.21.15.22:0/646276137 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb070067fc0 0x7fb07006a480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb090152a20 tx=0x7fb084008040 comp rx=0 tx=0).stop 2024-03-29T04:17:34.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.536+0000 7fb094c56640 1 -- 172.21.15.22:0/646276137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb090073490 msgr2=0x7fb090142a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:34.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.536+0000 7fb094c56640 1 --2- 172.21.15.22:0/646276137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb090073490 0x7fb090142a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb08000c7c0 tx=0x7fb08000cc80 comp rx=0 tx=0).stop 2024-03-29T04:17:34.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.536+0000 7fb094c56640 1 -- 172.21.15.22:0/646276137 shutdown_connections 2024-03-29T04:17:34.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.536+0000 7fb094c56640 1 --2- 172.21.15.22:0/646276137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0900b5930 0x7fb090158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:34.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.536+0000 7fb094c56640 1 --2- 172.21.15.22:0/646276137 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0900b5550 0x7fb090152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:34.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.536+0000 7fb094c56640 1 --2- 172.21.15.22:0/646276137 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb070067fc0 0x7fb07006a480 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:34.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.536+0000 7fb094c56640 1 --2- 172.21.15.22:0/646276137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb090073490 0x7fb090142a10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:34.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.536+0000 7fb094c56640 1 -- 172.21.15.22:0/646276137 >> 172.21.15.22:0/646276137 conn(0x7fb0900b1b80 msgr2=0x7fb090156b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:34.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.536+0000 7fb094c56640 1 -- 172.21.15.22:0/646276137 shutdown_connections 2024-03-29T04:17:34.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.536+0000 7fb094c56640 1 -- 172.21.15.22:0/646276137 wait complete. 2024-03-29T04:17:34.547 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-29T04:17:34.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.696+0000 7f7d316b5640 1 Processor -- start 2024-03-29T04:17:34.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.696+0000 7f7d316b5640 1 -- start start 2024-03-29T04:17:34.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.696+0000 7f7d316b5640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7d2c15d130 0x7f7d2c06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:34.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.696+0000 7f7d316b5640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7d2c141fa0 0x7f7d2c06d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:34.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.696+0000 7f7d316b5640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d2c142550 con 0x7f7d2c06c2e0 2024-03-29T04:17:34.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.696+0000 7f7d316b5640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f7d2c0cf830 con 0x7f7d2c141fa0 2024-03-29T04:17:34.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.696+0000 7f7d316b5640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7d2c06b800 con 0x7f7d2c15d130 2024-03-29T04:17:34.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.696+0000 7f7d2b7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7d2c141fa0 0x7f7d2c06d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:34.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.696+0000 7f7d2b7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7d2c141fa0 0x7f7d2c06d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53686/0 (socket says 172.21.15.22:53686) 2024-03-29T04:17:34.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.696+0000 7f7d2b7fe640 1 -- 172.21.15.22:0/3856901356 learned_addr learned my addr 172.21.15.22:0/3856901356 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:34.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.696+0000 7f7d2affd640 1 --2- 172.21.15.22:0/3856901356 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7d2c15d130 0x7f7d2c06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:34.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.700+0000 7f7d2b7fe640 1 -- 172.21.15.22:0/3856901356 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7d2c15d130 msgr2=0x7f7d2c06d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:34.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.700+0000 7f7d2b7fe640 1 --2- 172.21.15.22:0/3856901356 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7d2c15d130 0x7f7d2c06d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:34.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.700+0000 7f7d2b7fe640 1 -- 172.21.15.22:0/3856901356 >> v1:172.21.15.151:6789/0 conn(0x7f7d2c06c2e0 legacy=0x7f7d2c141890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:34.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.700+0000 7f7d2b7fe640 1 -- 172.21.15.22:0/3856901356 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2c06bae0 con 0x7f7d2c141fa0 2024-03-29T04:17:34.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.700+0000 7f7d2b7fe640 1 --2- 172.21.15.22:0/3856901356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d2c141fa0 0x7f7d2c06d8f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009870 tx=0x7f7d2401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922b289cd64475dc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:34.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.700+0000 7f7d29ffb640 1 -- 172.21.15.22:0/3856901356 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d240a7070 con 0x7f7d2c141fa0 2024-03-29T04:17:34.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.700+0000 7f7d29ffb640 1 -- 172.21.15.22:0/3856901356 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d240b0070 con 0x7f7d2c141fa0 2024-03-29T04:17:34.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.700+0000 7f7d29ffb640 1 -- 172.21.15.22:0/3856901356 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d240ab890 con 0x7f7d2c141fa0 2024-03-29T04:17:34.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.700+0000 7f7d316b5640 1 -- 172.21.15.22:0/3856901356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d2c141fa0 msgr2=0x7f7d2c06d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:34.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.700+0000 7f7d316b5640 1 --2- 172.21.15.22:0/3856901356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d2c141fa0 0x7f7d2c06d8f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009870 tx=0x7f7d2401c560 comp rx=0 tx=0).stop 2024-03-29T04:17:34.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.700+0000 7f7d316b5640 1 -- 172.21.15.22:0/3856901356 shutdown_connections 2024-03-29T04:17:34.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.700+0000 7f7d316b5640 1 --2- 172.21.15.22:0/3856901356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d2c141fa0 0x7f7d2c06d8f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:34.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.700+0000 7f7d316b5640 1 --2- 172.21.15.22:0/3856901356 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7d2c15d130 0x7f7d2c06d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:34.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.700+0000 7f7d316b5640 1 -- 172.21.15.22:0/3856901356 >> 172.21.15.22:0/3856901356 conn(0x7f7d2c0b1c60 msgr2=0x7f7d2c0b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:34.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.700+0000 7f7d316b5640 1 -- 172.21.15.22:0/3856901356 shutdown_connections 2024-03-29T04:17:34.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.700+0000 7f7d316b5640 1 -- 172.21.15.22:0/3856901356 wait complete. 2024-03-29T04:17:34.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d316b5640 1 Processor -- start 2024-03-29T04:17:34.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d316b5640 1 -- start start 2024-03-29T04:17:34.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d316b5640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7d2c06c2e0 0x7f7d2c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:34.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d316b5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d2c141fa0 0x7f7d2c138a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:34.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d316b5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d2c15d130 0x7f7d2c0b0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:34.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d316b5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c0cf3c0 con 0x7f7d2c141fa0 2024-03-29T04:17:34.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d316b5640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c0cf240 con 0x7f7d2c15d130 2024-03-29T04:17:34.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d316b5640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7d2c0cf540 con 0x7f7d2c06c2e0 2024-03-29T04:17:34.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d2a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d2c141fa0 0x7f7d2c138a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:34.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d2a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d2c141fa0 0x7f7d2c138a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53690/0 (socket says 172.21.15.22:53690) 2024-03-29T04:17:34.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d2a7fc640 1 -- 172.21.15.22:0/2177897491 learned_addr learned my addr 172.21.15.22:0/2177897491 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:34.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d2affd640 1 --2- 172.21.15.22:0/2177897491 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7d2c06c2e0 0x7f7d2c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:34.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d2b7fe640 1 --2- 172.21.15.22:0/2177897491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d2c15d130 0x7f7d2c0b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:34.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d2a7fc640 1 -- 172.21.15.22:0/2177897491 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7d2c06c2e0 msgr2=0x7f7d2c0b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:34.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d2a7fc640 1 --2- 172.21.15.22:0/2177897491 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7d2c06c2e0 0x7f7d2c0b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:34.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d2a7fc640 1 -- 172.21.15.22:0/2177897491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d2c15d130 msgr2=0x7f7d2c0b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:34.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d2a7fc640 1 --2- 172.21.15.22:0/2177897491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d2c15d130 0x7f7d2c0b0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:34.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d2a7fc640 1 -- 172.21.15.22:0/2177897491 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2c153cd0 con 0x7f7d2c141fa0 2024-03-29T04:17:34.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d2a7fc640 1 --2- 172.21.15.22:0/2177897491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d2c141fa0 0x7f7d2c138a20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400c7c0 tx=0x7f7d1400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:34.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d237fe640 1 -- 172.21.15.22:0/2177897491 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14016020 con 0x7f7d2c141fa0 2024-03-29T04:17:34.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d237fe640 1 -- 172.21.15.22:0/2177897491 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d14006d60 con 0x7f7d2c141fa0 2024-03-29T04:17:34.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d316b5640 1 -- 172.21.15.22:0/2177897491 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2c13b000 con 0x7f7d2c141fa0 2024-03-29T04:17:34.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d237fe640 1 -- 172.21.15.22:0/2177897491 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14003bc0 con 0x7f7d2c141fa0 2024-03-29T04:17:34.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.704+0000 7f7d316b5640 1 -- 172.21.15.22:0/2177897491 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d2c13b5c0 con 0x7f7d2c141fa0 2024-03-29T04:17:34.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.708+0000 7f7d316b5640 1 -- 172.21.15.22:0/2177897491 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d2c068900 con 0x7f7d2c141fa0 2024-03-29T04:17:34.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.708+0000 7f7d237fe640 1 -- 172.21.15.22:0/2177897491 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f7d1400a7f0 con 0x7f7d2c141fa0 2024-03-29T04:17:34.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.708+0000 7f7d237fe640 1 --2- 172.21.15.22:0/2177897491 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7d0c067ff0 0x7f7d0c06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:34.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.708+0000 7f7d237fe640 1 -- 172.21.15.22:0/2177897491 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f7d14085fe0 con 0x7f7d2c141fa0 2024-03-29T04:17:34.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.708+0000 7f7d2affd640 1 --2- 172.21.15.22:0/2177897491 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7d0c067ff0 0x7f7d0c06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:34.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.708+0000 7f7d2affd640 1 --2- 172.21.15.22:0/2177897491 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7d0c067ff0 0x7f7d0c06a4b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00b930 tx=0x7f7d1c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:34.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.712+0000 7f7d237fe640 1 -- 172.21.15.22:0/2177897491 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f7d14020020 con 0x7f7d2c141fa0 2024-03-29T04:17:34.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:34.876+0000 7f7d316b5640 1 -- 172.21.15.22:0/2177897491 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f7d2c06cba0 con 0x7f7d2c141fa0 2024-03-29T04:17:35.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.536+0000 7f7d237fe640 1 -- 172.21.15.22:0/2177897491 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) ==== 171+0+0 (secure 0 0 0) 0x7f7d140569c0 con 0x7f7d2c141fa0 2024-03-29T04:17:35.520 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' created 2024-03-29T04:17:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.540+0000 7f7d316b5640 1 -- 172.21.15.22:0/2177897491 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7d0c067ff0 msgr2=0x7f7d0c06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.540+0000 7f7d316b5640 1 --2- 172.21.15.22:0/2177897491 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7d0c067ff0 0x7f7d0c06a4b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00b930 tx=0x7f7d1c008040 comp rx=0 tx=0).stop 2024-03-29T04:17:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.540+0000 7f7d316b5640 1 -- 172.21.15.22:0/2177897491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d2c141fa0 msgr2=0x7f7d2c138a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.540+0000 7f7d316b5640 1 --2- 172.21.15.22:0/2177897491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d2c141fa0 0x7f7d2c138a20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400c7c0 tx=0x7f7d1400cc80 comp rx=0 tx=0).stop 2024-03-29T04:17:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.540+0000 7f7d316b5640 1 -- 172.21.15.22:0/2177897491 shutdown_connections 2024-03-29T04:17:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.540+0000 7f7d316b5640 1 --2- 172.21.15.22:0/2177897491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d2c15d130 0x7f7d2c0b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.540+0000 7f7d316b5640 1 --2- 172.21.15.22:0/2177897491 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7d0c067ff0 0x7f7d0c06a4b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.540+0000 7f7d316b5640 1 --2- 172.21.15.22:0/2177897491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d2c141fa0 0x7f7d2c138a20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.540+0000 7f7d316b5640 1 --2- 172.21.15.22:0/2177897491 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7d2c06c2e0 0x7f7d2c0b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.540+0000 7f7d316b5640 1 -- 172.21.15.22:0/2177897491 >> 172.21.15.22:0/2177897491 conn(0x7f7d2c0b1c60 msgr2=0x7f7d2c15bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.540+0000 7f7d316b5640 1 -- 172.21.15.22:0/2177897491 shutdown_connections 2024-03-29T04:17:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.544+0000 7f7d316b5640 1 -- 172.21.15.22:0/2177897491 wait complete. 2024-03-29T04:17:35.557 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-29T04:17:35.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.700+0000 7f51b88ce640 1 Processor -- start 2024-03-29T04:17:35.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.700+0000 7f51b88ce640 1 -- start start 2024-03-29T04:17:35.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.700+0000 7f51b88ce640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f51b40cf870 0x7f51b415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:35.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.700+0000 7f51b88ce640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51b40726d0 0x7f51b40b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:35.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.700+0000 7f51b88ce640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f51b40b4090 0x7f51b40b4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:35.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.700+0000 7f51b88ce640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f51b40b3c40 con 0x7f51b40cf870 2024-03-29T04:17:35.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.700+0000 7f51b88ce640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f51b4071910 con 0x7f51b40b4090 2024-03-29T04:17:35.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.700+0000 7f51b88ce640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f51b4071bf0 con 0x7f51b40726d0 2024-03-29T04:17:35.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.700+0000 7f51b259c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f51b40cf870 0x7f51b415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:35.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.700+0000 7f51b259c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f51b40cf870 0x7f51b415baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46370/0 (socket says 172.21.15.22:46370) 2024-03-29T04:17:35.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.700+0000 7f51b259c640 1 -- 172.21.15.22:0/1961445455 learned_addr learned my addr 172.21.15.22:0/1961445455 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:35.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.704+0000 7f51b2d9d640 1 --2- 172.21.15.22:0/1961445455 >> v2:172.21.15.151:3300/0 conn(0x7f51b40b4090 0x7f51b40b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:35.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.704+0000 7f51b1d9b640 1 --2- 172.21.15.22:0/1961445455 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51b40726d0 0x7f51b40b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:35.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.704+0000 7f51b259c640 1 -- 172.21.15.22:0/1961445455 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51b40726d0 msgr2=0x7f51b40b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:35.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.704+0000 7f51b259c640 1 --2- 172.21.15.22:0/1961445455 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51b40726d0 0x7f51b40b3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:35.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.704+0000 7f51b259c640 1 -- 172.21.15.22:0/1961445455 >> v2:172.21.15.151:3300/0 conn(0x7f51b40b4090 msgr2=0x7f51b40b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:35.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.704+0000 7f51b259c640 1 --2- 172.21.15.22:0/1961445455 >> v2:172.21.15.151:3300/0 conn(0x7f51b40b4090 0x7f51b40b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:35.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.704+0000 7f51b259c640 1 -- 172.21.15.22:0/1961445455 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b4071ed0 con 0x7f51b40cf870 2024-03-29T04:17:35.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.704+0000 7f51b2d9d640 1 --2- 172.21.15.22:0/1961445455 >> v2:172.21.15.151:3300/0 conn(0x7f51b40b4090 0x7f51b40b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:35.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.704+0000 7f51b259c640 1 --2- 172.21.15.22:0/1961445455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51b40cf870 0x7f51b415baa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f51b40b3690 tx=0x7f51a401cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f08ec4d0981a0b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:35.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.704+0000 7f51b159a640 1 -- 172.21.15.22:0/1961445455 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a40b0070 con 0x7f51b40cf870 2024-03-29T04:17:35.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.704+0000 7f51b159a640 1 -- 172.21.15.22:0/1961445455 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a40ab600 con 0x7f51b40cf870 2024-03-29T04:17:35.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.704+0000 7f51b159a640 1 -- 172.21.15.22:0/1961445455 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a40ab9d0 con 0x7f51b40cf870 2024-03-29T04:17:35.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.704+0000 7f51b88ce640 1 -- 172.21.15.22:0/1961445455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51b40cf870 msgr2=0x7f51b415baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:35.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.704+0000 7f51b88ce640 1 --2- 172.21.15.22:0/1961445455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51b40cf870 0x7f51b415baa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f51b40b3690 tx=0x7f51a401cf80 comp rx=0 tx=0).stop 2024-03-29T04:17:35.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.704+0000 7f51b88ce640 1 -- 172.21.15.22:0/1961445455 shutdown_connections 2024-03-29T04:17:35.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.704+0000 7f51b88ce640 1 --2- 172.21.15.22:0/1961445455 >> v2:172.21.15.151:3300/0 conn(0x7f51b40b4090 0x7f51b40b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:35.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.704+0000 7f51b88ce640 1 --2- 172.21.15.22:0/1961445455 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51b40726d0 0x7f51b40b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:35.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.704+0000 7f51b88ce640 1 --2- 172.21.15.22:0/1961445455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51b40cf870 0x7f51b415baa0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:35.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.704+0000 7f51b88ce640 1 -- 172.21.15.22:0/1961445455 >> 172.21.15.22:0/1961445455 conn(0x7f51b40d00a0 msgr2=0x7f51b40d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:35.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.704+0000 7f51b88ce640 1 -- 172.21.15.22:0/1961445455 shutdown_connections 2024-03-29T04:17:35.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.704+0000 7f51b88ce640 1 -- 172.21.15.22:0/1961445455 wait complete. 2024-03-29T04:17:35.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b88ce640 1 Processor -- start 2024-03-29T04:17:35.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b88ce640 1 -- start start 2024-03-29T04:17:35.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b88ce640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51b40726d0 0x7f51b41377d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:35.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b88ce640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51b40b4090 0x7f51b406dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:35.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b88ce640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b40cf870 0x7f51b406e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:35.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b88ce640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f51b406d7f0 con 0x7f51b40726d0 2024-03-29T04:17:35.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b88ce640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f51b406d670 con 0x7f51b40cf870 2024-03-29T04:17:35.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b88ce640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f51b406d970 con 0x7f51b40b4090 2024-03-29T04:17:35.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51b40726d0 0x7f51b41377d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:35.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51b40726d0 0x7f51b41377d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46386/0 (socket says 172.21.15.22:46386) 2024-03-29T04:17:35.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b259c640 1 -- 172.21.15.22:0/1463724874 learned_addr learned my addr 172.21.15.22:0/1463724874 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:35.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b1d9b640 1 --2- 172.21.15.22:0/1463724874 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51b40b4090 0x7f51b406dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:35.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b2d9d640 1 --2- 172.21.15.22:0/1463724874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b40cf870 0x7f51b406e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:35.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b259c640 1 -- 172.21.15.22:0/1463724874 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51b40b4090 msgr2=0x7f51b406dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:35.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b259c640 1 --2- 172.21.15.22:0/1463724874 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51b40b4090 0x7f51b406dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:35.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b259c640 1 -- 172.21.15.22:0/1463724874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b40cf870 msgr2=0x7f51b406e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:35.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b259c640 1 --2- 172.21.15.22:0/1463724874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b40cf870 0x7f51b406e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:35.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b259c640 1 -- 172.21.15.22:0/1463724874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b41f4e60 con 0x7f51b40726d0 2024-03-29T04:17:35.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b2d9d640 1 --2- 172.21.15.22:0/1463724874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b40cf870 0x7f51b406e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:35.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b1d9b640 1 --2- 172.21.15.22:0/1463724874 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51b40b4090 0x7f51b406dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:35.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b259c640 1 --2- 172.21.15.22:0/1463724874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51b40726d0 0x7f51b41377d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f51a40ae040 tx=0x7f51a401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:35.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51a2ffd640 1 -- 172.21.15.22:0/1463724874 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a40ba020 con 0x7f51b40726d0 2024-03-29T04:17:35.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51a2ffd640 1 -- 172.21.15.22:0/1463724874 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a40b6640 con 0x7f51b40726d0 2024-03-29T04:17:35.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b88ce640 1 -- 172.21.15.22:0/1463724874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b41f5140 con 0x7f51b40726d0 2024-03-29T04:17:35.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51a2ffd640 1 -- 172.21.15.22:0/1463724874 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a40b0040 con 0x7f51b40726d0 2024-03-29T04:17:35.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b88ce640 1 -- 172.21.15.22:0/1463724874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b41f5700 con 0x7f51b40726d0 2024-03-29T04:17:35.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51b88ce640 1 -- 172.21.15.22:0/1463724874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b4068900 con 0x7f51b40726d0 2024-03-29T04:17:35.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.708+0000 7f51a2ffd640 1 -- 172.21.15.22:0/1463724874 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f51a40bd020 con 0x7f51b40726d0 2024-03-29T04:17:35.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.712+0000 7f51a2ffd640 1 --2- 172.21.15.22:0/1463724874 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5194067fc0 0x7f519406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:35.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.712+0000 7f51a2ffd640 1 -- 172.21.15.22:0/1463724874 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f51a4120790 con 0x7f51b40726d0 2024-03-29T04:17:35.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.712+0000 7f51b1d9b640 1 --2- 172.21.15.22:0/1463724874 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5194067fc0 0x7f519406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:35.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.712+0000 7f51b1d9b640 1 --2- 172.21.15.22:0/1463724874 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5194067fc0 0x7f519406a480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f51a8006fd0 tx=0x7f51a8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:35.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.716+0000 7f51a2ffd640 1 -- 172.21.15.22:0/1463724874 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f51a40c4030 con 0x7f51b40726d0 2024-03-29T04:17:35.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:35.912+0000 7f51b88ce640 1 -- 172.21.15.22:0/1463724874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f51b406ebe0 con 0x7f51b40726d0 2024-03-29T04:17:36.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.568+0000 7f51a2ffd640 1 -- 172.21.15.22:0/1463724874 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f51a4016110 con 0x7f51b40726d0 2024-03-29T04:17:36.551 INFO:teuthology.orchestra.run.smithi022.stderr:new fs with metadata pool 2 and data pool 3 2024-03-29T04:17:36.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.576+0000 7f51b88ce640 1 -- 172.21.15.22:0/1463724874 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5194067fc0 msgr2=0x7f519406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:36.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.576+0000 7f51b88ce640 1 --2- 172.21.15.22:0/1463724874 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5194067fc0 0x7f519406a480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f51a8006fd0 tx=0x7f51a8009040 comp rx=0 tx=0).stop 2024-03-29T04:17:36.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.576+0000 7f51b88ce640 1 -- 172.21.15.22:0/1463724874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51b40726d0 msgr2=0x7f51b41377d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:36.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.576+0000 7f51b88ce640 1 --2- 172.21.15.22:0/1463724874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51b40726d0 0x7f51b41377d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f51a40ae040 tx=0x7f51a401ad60 comp rx=0 tx=0).stop 2024-03-29T04:17:36.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.576+0000 7f51b88ce640 1 -- 172.21.15.22:0/1463724874 shutdown_connections 2024-03-29T04:17:36.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.576+0000 7f51b88ce640 1 --2- 172.21.15.22:0/1463724874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b40cf870 0x7f51b406e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:36.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.576+0000 7f51b88ce640 1 --2- 172.21.15.22:0/1463724874 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51b40b4090 0x7f51b406dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:36.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.576+0000 7f51b88ce640 1 --2- 172.21.15.22:0/1463724874 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5194067fc0 0x7f519406a480 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:36.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.576+0000 7f51b88ce640 1 --2- 172.21.15.22:0/1463724874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51b40726d0 0x7f51b41377d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:36.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.576+0000 7f51b88ce640 1 -- 172.21.15.22:0/1463724874 >> 172.21.15.22:0/1463724874 conn(0x7f51b40d00a0 msgr2=0x7f51b40b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:36.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.576+0000 7f51b88ce640 1 -- 172.21.15.22:0/1463724874 shutdown_connections 2024-03-29T04:17:36.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.576+0000 7f51b88ce640 1 -- 172.21.15.22:0/1463724874 wait complete. 2024-03-29T04:17:36.599 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-29T04:17:36.599 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-29T04:17:36.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.744+0000 7f34b8e79640 1 Processor -- start 2024-03-29T04:17:36.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.744+0000 7f34b8e79640 1 -- start start 2024-03-29T04:17:36.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.744+0000 7f34b8e79640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f34b40b5550 0x7f34b4160610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:36.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.744+0000 7f34b8e79640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34b40b5930 0x7f34b415c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:36.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.744+0000 7f34b8e79640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34b415d090 con 0x7f34b4073490 2024-03-29T04:17:36.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.744+0000 7f34b8e79640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f34b415d370 con 0x7f34b40b5550 2024-03-29T04:17:36.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.744+0000 7f34b8e79640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f34b415fe10 con 0x7f34b40b5930 2024-03-29T04:17:36.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.744+0000 7f34b259c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f34b40b5550 0x7f34b4160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:36.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.744+0000 7f34b259c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f34b40b5550 0x7f34b4160610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46390/0 (socket says 172.21.15.22:46390) 2024-03-29T04:17:36.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.744+0000 7f34b259c640 1 -- 172.21.15.22:0/1008469965 learned_addr learned my addr 172.21.15.22:0/1008469965 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:36.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.744+0000 7f34b259c640 1 -- 172.21.15.22:0/1008469965 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34b40b5930 msgr2=0x7f34b415c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:36.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.744+0000 7f34b259c640 1 --2- 172.21.15.22:0/1008469965 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34b40b5930 0x7f34b415c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:36.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.744+0000 7f34b259c640 1 -- 172.21.15.22:0/1008469965 >> v1:172.21.15.151:6789/0 conn(0x7f34b4073490 legacy=0x7f34b406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:36.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.744+0000 7f34b259c640 1 -- 172.21.15.22:0/1008469965 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b41600f0 con 0x7f34b40b5550 2024-03-29T04:17:36.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.744+0000 7f34b259c640 1 --2- 172.21.15.22:0/1008469965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34b40b5550 0x7f34b4160610 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009870 tx=0x7f34a401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27514da3783bc764 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:36.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b159a640 1 -- 172.21.15.22:0/1008469965 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a40ab640 con 0x7f34b40b5550 2024-03-29T04:17:36.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b159a640 1 -- 172.21.15.22:0/1008469965 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a40b0070 con 0x7f34b40b5550 2024-03-29T04:17:36.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b159a640 1 -- 172.21.15.22:0/1008469965 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a40aba00 con 0x7f34b40b5550 2024-03-29T04:17:36.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b8e79640 1 -- 172.21.15.22:0/1008469965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34b40b5550 msgr2=0x7f34b4160610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:36.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b8e79640 1 --2- 172.21.15.22:0/1008469965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34b40b5550 0x7f34b4160610 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009870 tx=0x7f34a401c460 comp rx=0 tx=0).stop 2024-03-29T04:17:36.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b8e79640 1 -- 172.21.15.22:0/1008469965 shutdown_connections 2024-03-29T04:17:36.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b8e79640 1 --2- 172.21.15.22:0/1008469965 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34b40b5930 0x7f34b415c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:36.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b8e79640 1 --2- 172.21.15.22:0/1008469965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34b40b5550 0x7f34b4160610 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:36.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b8e79640 1 -- 172.21.15.22:0/1008469965 >> 172.21.15.22:0/1008469965 conn(0x7f34b40b1b80 msgr2=0x7f34b40b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:36.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b8e79640 1 -- 172.21.15.22:0/1008469965 shutdown_connections 2024-03-29T04:17:36.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b8e79640 1 -- 172.21.15.22:0/1008469965 wait complete. 2024-03-29T04:17:36.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b8e79640 1 Processor -- start 2024-03-29T04:17:36.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b8e79640 1 -- start start 2024-03-29T04:17:36.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b8e79640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34b4073490 0x7f34b4154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:36.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b8e79640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34b40b5550 0x7f34b4157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:36.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b8e79640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34b40b5930 0x7f34b415a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:36.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b8e79640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f34b40cb010 con 0x7f34b40b5550 2024-03-29T04:17:36.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b8e79640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f34b40cae90 con 0x7f34b40b5930 2024-03-29T04:17:36.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b8e79640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f34b406cd80 con 0x7f34b4073490 2024-03-29T04:17:36.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b1d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34b40b5550 0x7f34b4157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:36.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b1d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34b40b5550 0x7f34b4157cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46404/0 (socket says 172.21.15.22:46404) 2024-03-29T04:17:36.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b1d9b640 1 -- 172.21.15.22:0/2206305339 learned_addr learned my addr 172.21.15.22:0/2206305339 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:36.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b2d9d640 1 --2- 172.21.15.22:0/2206305339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34b40b5930 0x7f34b415a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:36.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b259c640 1 --2- 172.21.15.22:0/2206305339 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34b4073490 0x7f34b4154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b1d9b640 1 -- 172.21.15.22:0/2206305339 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34b4073490 msgr2=0x7f34b4154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b1d9b640 1 --2- 172.21.15.22:0/2206305339 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34b4073490 0x7f34b4154e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b1d9b640 1 -- 172.21.15.22:0/2206305339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34b40b5930 msgr2=0x7f34b415a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b1d9b640 1 --2- 172.21.15.22:0/2206305339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34b40b5930 0x7f34b415a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b1d9b640 1 -- 172.21.15.22:0/2206305339 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b4155d40 con 0x7f34b40b5550 2024-03-29T04:17:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b2d9d640 1 --2- 172.21.15.22:0/2206305339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34b40b5930 0x7f34b415a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b259c640 1 --2- 172.21.15.22:0/2206305339 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34b4073490 0x7f34b4154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.748+0000 7f34b1d9b640 1 --2- 172.21.15.22:0/2206305339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34b40b5550 0x7f34b4157cf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f34a800b4b0 tx=0x7f34a800b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.752+0000 7f34a2ffd640 1 -- 172.21.15.22:0/2206305339 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a8014070 con 0x7f34b40b5550 2024-03-29T04:17:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.752+0000 7f34a2ffd640 1 -- 172.21.15.22:0/2206305339 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a8010610 con 0x7f34b40b5550 2024-03-29T04:17:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.752+0000 7f34b8e79640 1 -- 172.21.15.22:0/2206305339 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b41554a0 con 0x7f34b40b5550 2024-03-29T04:17:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.752+0000 7f34a2ffd640 1 -- 172.21.15.22:0/2206305339 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a8010a00 con 0x7f34b40b5550 2024-03-29T04:17:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.752+0000 7f34b8e79640 1 -- 172.21.15.22:0/2206305339 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b4155a60 con 0x7f34b40b5550 2024-03-29T04:17:36.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.752+0000 7f34b8e79640 1 -- 172.21.15.22:0/2206305339 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b40687f0 con 0x7f34b40b5550 2024-03-29T04:17:36.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.752+0000 7f34a2ffd640 1 -- 172.21.15.22:0/2206305339 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f34a8017070 con 0x7f34b40b5550 2024-03-29T04:17:36.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.752+0000 7f34a2ffd640 1 --2- 172.21.15.22:0/2206305339 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3494067fc0 0x7f349406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:36.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.752+0000 7f34a2ffd640 1 -- 172.21.15.22:0/2206305339 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f34a8085ee0 con 0x7f34b40b5550 2024-03-29T04:17:36.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.752+0000 7f34b259c640 1 --2- 172.21.15.22:0/2206305339 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3494067fc0 0x7f349406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:36.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.752+0000 7f34b259c640 1 --2- 172.21.15.22:0/2206305339 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3494067fc0 0x7f349406a480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f34a40af000 tx=0x7f34a4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:36.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.756+0000 7f34a2ffd640 1 -- 172.21.15.22:0/2206305339 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f34a8056720 con 0x7f34b40b5550 2024-03-29T04:17:36.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:36.928+0000 7f34b8e79640 1 -- 172.21.15.22:0/2206305339 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f34b40af8d0 con 0x7f34b40b5550 2024-03-29T04:17:37.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.572+0000 7f34a2ffd640 1 -- 172.21.15.22:0/2206305339 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) ==== 157+0+0 (secure 0 0 0) 0x7f34a8056020 con 0x7f34b40b5550 2024-03-29T04:17:37.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.576+0000 7f34b8e79640 1 -- 172.21.15.22:0/2206305339 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3494067fc0 msgr2=0x7f349406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:37.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.576+0000 7f34b8e79640 1 --2- 172.21.15.22:0/2206305339 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3494067fc0 0x7f349406a480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f34a40af000 tx=0x7f34a4008040 comp rx=0 tx=0).stop 2024-03-29T04:17:37.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.576+0000 7f34b8e79640 1 -- 172.21.15.22:0/2206305339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34b40b5550 msgr2=0x7f34b4157cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:37.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.576+0000 7f34b8e79640 1 --2- 172.21.15.22:0/2206305339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34b40b5550 0x7f34b4157cf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f34a800b4b0 tx=0x7f34a800b970 comp rx=0 tx=0).stop 2024-03-29T04:17:37.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.576+0000 7f34b8e79640 1 -- 172.21.15.22:0/2206305339 shutdown_connections 2024-03-29T04:17:37.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.576+0000 7f34b8e79640 1 --2- 172.21.15.22:0/2206305339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34b40b5930 0x7f34b415a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:37.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.576+0000 7f34b8e79640 1 --2- 172.21.15.22:0/2206305339 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3494067fc0 0x7f349406a480 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:37.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.576+0000 7f34b8e79640 1 --2- 172.21.15.22:0/2206305339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34b40b5550 0x7f34b4157cf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:37.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.576+0000 7f34b8e79640 1 --2- 172.21.15.22:0/2206305339 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34b4073490 0x7f34b4154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:37.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.576+0000 7f34b8e79640 1 -- 172.21.15.22:0/2206305339 >> 172.21.15.22:0/2206305339 conn(0x7f34b40b1b80 msgr2=0x7f34b4159e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:37.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.576+0000 7f34b8e79640 1 -- 172.21.15.22:0/2206305339 shutdown_connections 2024-03-29T04:17:37.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.576+0000 7f34b8e79640 1 -- 172.21.15.22:0/2206305339 wait complete. 2024-03-29T04:17:37.578 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-29T04:17:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.736+0000 7ff864842640 1 Processor -- start 2024-03-29T04:17:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.736+0000 7ff864842640 1 -- start start 2024-03-29T04:17:37.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.740+0000 7ff864842640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff860147eb0 0x7ff86006b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:37.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.740+0000 7ff864842640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff8600700f0 0x7ff86015fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:37.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.740+0000 7ff864842640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff86015ebc0 con 0x7ff86006bba0 2024-03-29T04:17:37.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.740+0000 7ff864842640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7ff86015ee60 con 0x7ff8600700f0 2024-03-29T04:17:37.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.740+0000 7ff864842640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7ff86015f140 con 0x7ff860147eb0 2024-03-29T04:17:37.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.740+0000 7ff85ed9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff8600700f0 0x7ff86015fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:37.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.740+0000 7ff85dd9b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff86006bba0 0x7ff86006f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:43092/0 (socket says 172.21.15.22:43092) 2024-03-29T04:17:37.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.740+0000 7ff85dd9b640 1 -- 172.21.15.22:0/1998862948 learned_addr learned my addr 172.21.15.22:0/1998862948 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:37.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.740+0000 7ff85e59c640 1 --2- 172.21.15.22:0/1998862948 >> v2:172.21.15.151:3300/0 conn(0x7ff860147eb0 0x7ff86006b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:37.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.740+0000 7ff85ed9d640 1 -- 172.21.15.22:0/1998862948 >> v2:172.21.15.151:3300/0 conn(0x7ff860147eb0 msgr2=0x7ff86006b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:37.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.740+0000 7ff85ed9d640 1 --2- 172.21.15.22:0/1998862948 >> v2:172.21.15.151:3300/0 conn(0x7ff860147eb0 0x7ff86006b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:37.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.740+0000 7ff85ed9d640 1 -- 172.21.15.22:0/1998862948 >> v1:172.21.15.22:6789/0 conn(0x7ff86006bba0 legacy=0x7ff86006f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:37.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.740+0000 7ff85ed9d640 1 -- 172.21.15.22:0/1998862948 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86015f420 con 0x7ff8600700f0 2024-03-29T04:17:37.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.740+0000 7ff85d59a640 1 -- 172.21.15.22:0/1998862948 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 521665934 0 0) 0x7ff86015ebc0 con 0x7ff86006bba0 2024-03-29T04:17:37.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.740+0000 7ff85ed9d640 1 --2- 172.21.15.22:0/1998862948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8600700f0 0x7ff86015fc70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff8600b3a70 tx=0x7ff85801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbcf7c99f6bfa05a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:37.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.740+0000 7ff85d59a640 1 -- 172.21.15.22:0/1998862948 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8580b2070 con 0x7ff8600700f0 2024-03-29T04:17:37.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.740+0000 7ff85d59a640 1 -- 172.21.15.22:0/1998862948 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8580a57f0 con 0x7ff8600700f0 2024-03-29T04:17:37.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.740+0000 7ff85d59a640 1 -- 172.21.15.22:0/1998862948 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8580a5bc0 con 0x7ff8600700f0 2024-03-29T04:17:37.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.744+0000 7ff864842640 1 -- 172.21.15.22:0/1998862948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8600700f0 msgr2=0x7ff86015fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:37.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.744+0000 7ff864842640 1 --2- 172.21.15.22:0/1998862948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8600700f0 0x7ff86015fc70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff8600b3a70 tx=0x7ff85801c930 comp rx=0 tx=0).stop 2024-03-29T04:17:37.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.744+0000 7ff864842640 1 -- 172.21.15.22:0/1998862948 shutdown_connections 2024-03-29T04:17:37.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.744+0000 7ff864842640 1 --2- 172.21.15.22:0/1998862948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8600700f0 0x7ff86015fc70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:37.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.744+0000 7ff864842640 1 --2- 172.21.15.22:0/1998862948 >> v2:172.21.15.151:3300/0 conn(0x7ff860147eb0 0x7ff86006b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:37.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.744+0000 7ff864842640 1 -- 172.21.15.22:0/1998862948 >> 172.21.15.22:0/1998862948 conn(0x7ff8600b32e0 msgr2=0x7ff8600d1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:37.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.744+0000 7ff864842640 1 -- 172.21.15.22:0/1998862948 shutdown_connections 2024-03-29T04:17:37.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.744+0000 7ff864842640 1 -- 172.21.15.22:0/1998862948 wait complete. 2024-03-29T04:17:37.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.744+0000 7ff864842640 1 Processor -- start 2024-03-29T04:17:37.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.744+0000 7ff864842640 1 -- start start 2024-03-29T04:17:37.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.744+0000 7ff864842640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff86006bba0 0x7ff860148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:37.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.744+0000 7ff864842640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8600700f0 0x7ff86013fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:37.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.744+0000 7ff864842640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff860147eb0 0x7ff8600ae470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:37.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.744+0000 7ff864842640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff860160dd0 con 0x7ff8600700f0 2024-03-29T04:17:37.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.744+0000 7ff864842640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff860160c50 con 0x7ff86006bba0 2024-03-29T04:17:37.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.744+0000 7ff864842640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff860160f50 con 0x7ff860147eb0 2024-03-29T04:17:37.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.748+0000 7ff85dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8600700f0 0x7ff86013fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:37.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.748+0000 7ff85dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8600700f0 0x7ff86013fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46420/0 (socket says 172.21.15.22:46420) 2024-03-29T04:17:37.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.748+0000 7ff85dd9b640 1 -- 172.21.15.22:0/1278593969 learned_addr learned my addr 172.21.15.22:0/1278593969 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:37.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.748+0000 7ff85e59c640 1 --2- 172.21.15.22:0/1278593969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff86006bba0 0x7ff860148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:37.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.748+0000 7ff85dd9b640 1 -- 172.21.15.22:0/1278593969 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff860147eb0 msgr2=0x7ff8600ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:37.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.748+0000 7ff85dd9b640 1 --2- 172.21.15.22:0/1278593969 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff860147eb0 0x7ff8600ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:37.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.748+0000 7ff85dd9b640 1 -- 172.21.15.22:0/1278593969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff86006bba0 msgr2=0x7ff860148780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:37.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.748+0000 7ff85dd9b640 1 --2- 172.21.15.22:0/1278593969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff86006bba0 0x7ff860148780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:37.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.748+0000 7ff85dd9b640 1 -- 172.21.15.22:0/1278593969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff860140270 con 0x7ff8600700f0 2024-03-29T04:17:37.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.748+0000 7ff85dd9b640 1 --2- 172.21.15.22:0/1278593969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8600700f0 0x7ff86013fd30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff85400c4b0 tx=0x7ff85400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:37.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.748+0000 7ff84effd640 1 -- 172.21.15.22:0/1278593969 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff854018070 con 0x7ff8600700f0 2024-03-29T04:17:37.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.748+0000 7ff84effd640 1 -- 172.21.15.22:0/1278593969 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff854007500 con 0x7ff8600700f0 2024-03-29T04:17:37.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.748+0000 7ff864842640 1 -- 172.21.15.22:0/1278593969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8600aefe0 con 0x7ff8600700f0 2024-03-29T04:17:37.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.748+0000 7ff84effd640 1 -- 172.21.15.22:0/1278593969 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff854004a30 con 0x7ff8600700f0 2024-03-29T04:17:37.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.748+0000 7ff864842640 1 -- 172.21.15.22:0/1278593969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8600af5a0 con 0x7ff8600700f0 2024-03-29T04:17:37.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.748+0000 7ff864842640 1 -- 172.21.15.22:0/1278593969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff860068900 con 0x7ff8600700f0 2024-03-29T04:17:37.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.748+0000 7ff84effd640 1 -- 172.21.15.22:0/1278593969 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff85401b070 con 0x7ff8600700f0 2024-03-29T04:17:37.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.748+0000 7ff84effd640 1 --2- 172.21.15.22:0/1278593969 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff840067ff0 0x7ff84006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:37.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.748+0000 7ff84effd640 1 -- 172.21.15.22:0/1278593969 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6007+0+0 (secure 0 0 0) 0x7ff854085950 con 0x7ff8600700f0 2024-03-29T04:17:37.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.748+0000 7ff85e59c640 1 --2- 172.21.15.22:0/1278593969 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff840067ff0 0x7ff84006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:37.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.752+0000 7ff85e59c640 1 --2- 172.21.15.22:0/1278593969 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff840067ff0 0x7ff84006a4b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff85000b930 tx=0x7ff850008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:37.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.752+0000 7ff84effd640 1 -- 172.21.15.22:0/1278593969 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff854056090 con 0x7ff8600700f0 2024-03-29T04:17:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:37.916+0000 7ff864842640 1 -- 172.21.15.22:0/1278593969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff8600b6240 con 0x7ff8600700f0 2024-03-29T04:17:39.490 INFO:tasks.ceph.mds.b.smithi151.stderr:2024-03-29T04:17:39.507+0000 7f68f6146640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T04:17:39.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.616+0000 7ff84effd640 1 -- 172.21.15.22:0/1278593969 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) ==== 228+0+0 (secure 0 0 0) 0x7ff854055990 con 0x7ff8600700f0 2024-03-29T04:17:39.597 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data_ec' created 2024-03-29T04:17:39.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.620+0000 7ff864842640 1 -- 172.21.15.22:0/1278593969 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff840067ff0 msgr2=0x7ff84006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:39.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.620+0000 7ff864842640 1 --2- 172.21.15.22:0/1278593969 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff840067ff0 0x7ff84006a4b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff85000b930 tx=0x7ff850008040 comp rx=0 tx=0).stop 2024-03-29T04:17:39.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.620+0000 7ff864842640 1 -- 172.21.15.22:0/1278593969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8600700f0 msgr2=0x7ff86013fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:39.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.620+0000 7ff864842640 1 --2- 172.21.15.22:0/1278593969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8600700f0 0x7ff86013fd30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff85400c4b0 tx=0x7ff85400c970 comp rx=0 tx=0).stop 2024-03-29T04:17:39.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.620+0000 7ff864842640 1 -- 172.21.15.22:0/1278593969 shutdown_connections 2024-03-29T04:17:39.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.620+0000 7ff864842640 1 --2- 172.21.15.22:0/1278593969 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff860147eb0 0x7ff8600ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:39.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.620+0000 7ff864842640 1 --2- 172.21.15.22:0/1278593969 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff840067ff0 0x7ff84006a4b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:39.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.620+0000 7ff864842640 1 --2- 172.21.15.22:0/1278593969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8600700f0 0x7ff86013fd30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:39.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.620+0000 7ff864842640 1 --2- 172.21.15.22:0/1278593969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff86006bba0 0x7ff860148780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:39.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.620+0000 7ff864842640 1 -- 172.21.15.22:0/1278593969 >> 172.21.15.22:0/1278593969 conn(0x7ff8600b32e0 msgr2=0x7ff8600b5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:39.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.620+0000 7ff864842640 1 -- 172.21.15.22:0/1278593969 shutdown_connections 2024-03-29T04:17:39.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.620+0000 7ff864842640 1 -- 172.21.15.22:0/1278593969 wait complete. 2024-03-29T04:17:39.630 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-29T04:17:39.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.776+0000 7fe0999c2640 1 Processor -- start 2024-03-29T04:17:39.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.776+0000 7fe0999c2640 1 -- start start 2024-03-29T04:17:39.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.776+0000 7fe0999c2640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fe0940d3cf0 0x7fe0940d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:39.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.776+0000 7fe0999c2640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe0940b7cd0 0x7fe09406b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:39.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.776+0000 7fe0999c2640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe094074030 con 0x7fe094161af0 2024-03-29T04:17:39.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.776+0000 7fe0999c2640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fe0940d4150 con 0x7fe0940b7cd0 2024-03-29T04:17:39.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.776+0000 7fe0999c2640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fe0940d2a20 con 0x7fe0940d3cf0 2024-03-29T04:17:39.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.776+0000 7fe092ffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe094161af0 0x7fe0940b75c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:43094/0 (socket says 172.21.15.22:43094) 2024-03-29T04:17:39.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.776+0000 7fe092ffd640 1 -- 172.21.15.22:0/3431535120 learned_addr learned my addr 172.21.15.22:0/3431535120 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:39.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.776+0000 7fe0937fe640 1 --2- 172.21.15.22:0/3431535120 >> v2:172.21.15.22:3300/0 conn(0x7fe0940b7cd0 0x7fe09406b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:39.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.776+0000 7fe0927fc640 1 --2- 172.21.15.22:0/3431535120 >> v2:172.21.15.151:3300/0 conn(0x7fe0940d3cf0 0x7fe0940d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:39.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.776+0000 7fe0937fe640 1 -- 172.21.15.22:0/3431535120 >> v2:172.21.15.151:3300/0 conn(0x7fe0940d3cf0 msgr2=0x7fe0940d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:39.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.776+0000 7fe0937fe640 1 --2- 172.21.15.22:0/3431535120 >> v2:172.21.15.151:3300/0 conn(0x7fe0940d3cf0 0x7fe0940d24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:39.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.776+0000 7fe0937fe640 1 -- 172.21.15.22:0/3431535120 >> v1:172.21.15.22:6789/0 conn(0x7fe094161af0 legacy=0x7fe0940b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:39.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.776+0000 7fe0937fe640 1 -- 172.21.15.22:0/3431535120 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0940d2c50 con 0x7fe0940b7cd0 2024-03-29T04:17:39.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.780+0000 7fe0937fe640 1 --2- 172.21.15.22:0/3431535120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0940b7cd0 0x7fe09406b660 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe09406d820 tx=0x7fe08c01f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62b61e2dba54343 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:39.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.780+0000 7fe091ffb640 1 -- 172.21.15.22:0/3431535120 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c0b4070 con 0x7fe0940b7cd0 2024-03-29T04:17:39.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.780+0000 7fe091ffb640 1 -- 172.21.15.22:0/3431535120 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe08c0af600 con 0x7fe0940b7cd0 2024-03-29T04:17:39.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.780+0000 7fe091ffb640 1 -- 172.21.15.22:0/3431535120 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c0af9d0 con 0x7fe0940b7cd0 2024-03-29T04:17:39.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.780+0000 7fe0999c2640 1 -- 172.21.15.22:0/3431535120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0940b7cd0 msgr2=0x7fe09406b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:39.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.780+0000 7fe0999c2640 1 --2- 172.21.15.22:0/3431535120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0940b7cd0 0x7fe09406b660 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe09406d820 tx=0x7fe08c01f110 comp rx=0 tx=0).stop 2024-03-29T04:17:39.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.780+0000 7fe0999c2640 1 -- 172.21.15.22:0/3431535120 shutdown_connections 2024-03-29T04:17:39.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.780+0000 7fe0999c2640 1 --2- 172.21.15.22:0/3431535120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0940b7cd0 0x7fe09406b660 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:39.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.780+0000 7fe0999c2640 1 --2- 172.21.15.22:0/3431535120 >> v2:172.21.15.151:3300/0 conn(0x7fe0940d3cf0 0x7fe0940d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:39.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.780+0000 7fe0999c2640 1 -- 172.21.15.22:0/3431535120 >> 172.21.15.22:0/3431535120 conn(0x7fe0940d42e0 msgr2=0x7fe0940d46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:39.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.780+0000 7fe0999c2640 1 -- 172.21.15.22:0/3431535120 shutdown_connections 2024-03-29T04:17:39.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.780+0000 7fe0999c2640 1 -- 172.21.15.22:0/3431535120 wait complete. 2024-03-29T04:17:39.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.780+0000 7fe0999c2640 1 Processor -- start 2024-03-29T04:17:39.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.780+0000 7fe0999c2640 1 -- start start 2024-03-29T04:17:39.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.780+0000 7fe0999c2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0940b7cd0 0x7fe0940ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:39.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.780+0000 7fe0999c2640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe0940d3cf0 0x7fe09414f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:39.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.780+0000 7fe0999c2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe094161af0 0x7fe09415c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:39.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe0999c2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe0940d34f0 con 0x7fe094161af0 2024-03-29T04:17:39.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe0999c2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe0940d3210 con 0x7fe0940b7cd0 2024-03-29T04:17:39.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe0999c2640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe0940d37d0 con 0x7fe0940d3cf0 2024-03-29T04:17:39.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe0937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe094161af0 0x7fe09415c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:39.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe0937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe094161af0 0x7fe09415c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46438/0 (socket says 172.21.15.22:46438) 2024-03-29T04:17:39.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe0937fe640 1 -- 172.21.15.22:0/2037426666 learned_addr learned my addr 172.21.15.22:0/2037426666 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:39.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe092ffd640 1 --2- 172.21.15.22:0/2037426666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0940b7cd0 0x7fe0940ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:39.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe0927fc640 1 --2- 172.21.15.22:0/2037426666 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe0940d3cf0 0x7fe09414f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:39.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe0937fe640 1 -- 172.21.15.22:0/2037426666 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe0940d3cf0 msgr2=0x7fe09414f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:39.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe0937fe640 1 --2- 172.21.15.22:0/2037426666 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe0940d3cf0 0x7fe09414f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:39.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe0937fe640 1 -- 172.21.15.22:0/2037426666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0940b7cd0 msgr2=0x7fe0940ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:39.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe0937fe640 1 --2- 172.21.15.22:0/2037426666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0940b7cd0 0x7fe0940ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:39.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe0937fe640 1 -- 172.21.15.22:0/2037426666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0940ca830 con 0x7fe094161af0 2024-03-29T04:17:39.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe0937fe640 1 --2- 172.21.15.22:0/2037426666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe094161af0 0x7fe09415c980 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe08c01bd20 tx=0x7fe08c01beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:39.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe07bfff640 1 -- 172.21.15.22:0/2037426666 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c0b4040 con 0x7fe094161af0 2024-03-29T04:17:39.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe07bfff640 1 -- 172.21.15.22:0/2037426666 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe08c0c1070 con 0x7fe094161af0 2024-03-29T04:17:39.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe0999c2640 1 -- 172.21.15.22:0/2037426666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0940b4cb0 con 0x7fe094161af0 2024-03-29T04:17:39.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe07bfff640 1 -- 172.21.15.22:0/2037426666 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c009450 con 0x7fe094161af0 2024-03-29T04:17:39.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe0999c2640 1 -- 172.21.15.22:0/2037426666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0940b5270 con 0x7fe094161af0 2024-03-29T04:17:39.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe0999c2640 1 -- 172.21.15.22:0/2037426666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0940687f0 con 0x7fe094161af0 2024-03-29T04:17:39.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe07bfff640 1 -- 172.21.15.22:0/2037426666 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe08c0ba6b0 con 0x7fe094161af0 2024-03-29T04:17:39.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe07bfff640 1 --2- 172.21.15.22:0/2037426666 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe080067ff0 0x7fe08006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:39.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe07bfff640 1 -- 172.21.15.22:0/2037426666 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7fe08c125bd0 con 0x7fe094161af0 2024-03-29T04:17:39.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.784+0000 7fe092ffd640 1 --2- 172.21.15.22:0/2037426666 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe080067ff0 0x7fe08006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:39.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.788+0000 7fe092ffd640 1 --2- 172.21.15.22:0/2037426666 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe080067ff0 0x7fe08006a4b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe088000f70 tx=0x7fe088007e90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:39.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.788+0000 7fe07bfff640 1 -- 172.21.15.22:0/2037426666 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fe08c018c70 con 0x7fe094161af0 2024-03-29T04:17:39.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:39.956+0000 7fe0999c2640 1 -- 172.21.15.22:0/2037426666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fe0940d49a0 con 0x7fe094161af0 2024-03-29T04:17:40.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.616+0000 7fe07bfff640 1 -- 172.21.15.22:0/2037426666 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) ==== 169+0+0 (secure 0 0 0) 0x7fe08c0f5c10 con 0x7fe094161af0 2024-03-29T04:17:40.599 INFO:teuthology.orchestra.run.smithi022.stderr:set pool 4 allow_ec_overwrites to true 2024-03-29T04:17:40.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.624+0000 7fe0999c2640 1 -- 172.21.15.22:0/2037426666 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe080067ff0 msgr2=0x7fe08006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:40.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.624+0000 7fe0999c2640 1 --2- 172.21.15.22:0/2037426666 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe080067ff0 0x7fe08006a4b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe088000f70 tx=0x7fe088007e90 comp rx=0 tx=0).stop 2024-03-29T04:17:40.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.624+0000 7fe0999c2640 1 -- 172.21.15.22:0/2037426666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe094161af0 msgr2=0x7fe09415c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:40.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.624+0000 7fe0999c2640 1 --2- 172.21.15.22:0/2037426666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe094161af0 0x7fe09415c980 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe08c01bd20 tx=0x7fe08c01beb0 comp rx=0 tx=0).stop 2024-03-29T04:17:40.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.624+0000 7fe0999c2640 1 -- 172.21.15.22:0/2037426666 shutdown_connections 2024-03-29T04:17:40.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.624+0000 7fe0999c2640 1 --2- 172.21.15.22:0/2037426666 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe080067ff0 0x7fe08006a4b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:40.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.624+0000 7fe0999c2640 1 --2- 172.21.15.22:0/2037426666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe094161af0 0x7fe09415c980 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:40.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.624+0000 7fe0999c2640 1 --2- 172.21.15.22:0/2037426666 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe0940d3cf0 0x7fe09414f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:40.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.624+0000 7fe0999c2640 1 --2- 172.21.15.22:0/2037426666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0940b7cd0 0x7fe0940ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:40.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.624+0000 7fe0999c2640 1 -- 172.21.15.22:0/2037426666 >> 172.21.15.22:0/2037426666 conn(0x7fe0940d42e0 msgr2=0x7fe09415e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:40.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.624+0000 7fe0999c2640 1 -- 172.21.15.22:0/2037426666 shutdown_connections 2024-03-29T04:17:40.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.628+0000 7fe0999c2640 1 -- 172.21.15.22:0/2037426666 wait complete. 2024-03-29T04:17:40.651 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-29T04:17:40.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.792+0000 7f30c4445640 1 Processor -- start 2024-03-29T04:17:40.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.792+0000 7f30c4445640 1 -- start start 2024-03-29T04:17:40.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.792+0000 7f30c4445640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f30bc15fe20 0x7f30bc160290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:40.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.792+0000 7f30c4445640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc0ccaa0 0x7f30bc0cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:40.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.792+0000 7f30c4445640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30bc0cd410 con 0x7f30bc15c160 2024-03-29T04:17:40.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.792+0000 7f30c4445640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f30bc1607d0 con 0x7f30bc15fe20 2024-03-29T04:17:40.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.792+0000 7f30c4445640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f30bc160c80 con 0x7f30bc0ccaa0 2024-03-29T04:17:40.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.796+0000 7f30c29e2640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc0ccaa0 0x7f30bc0cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:40.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.796+0000 7f30c21e1640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f30bc15c160 0x7f30bc06b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.22:51502/0 (socket says 172.21.15.22:51502) 2024-03-29T04:17:40.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.796+0000 7f30c21e1640 1 -- 172.21.15.22:0/1537298239 learned_addr learned my addr 172.21.15.22:0/1537298239 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:40.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.796+0000 7f30c19e0640 1 --2- 172.21.15.22:0/1537298239 >> v2:172.21.15.151:3300/0 conn(0x7f30bc15fe20 0x7f30bc160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:40.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.796+0000 7f30c11df640 1 -- 172.21.15.22:0/1537298239 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 31334805 0 0) 0x7f30bc0cd410 con 0x7f30bc15c160 2024-03-29T04:17:40.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.796+0000 7f30c11df640 1 -- 172.21.15.22:0/1537298239 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30a8003530 con 0x7f30bc15c160 2024-03-29T04:17:40.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.796+0000 7f30c29e2640 1 -- 172.21.15.22:0/1537298239 >> v2:172.21.15.151:3300/0 conn(0x7f30bc15fe20 msgr2=0x7f30bc160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:40.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.796+0000 7f30c29e2640 1 --2- 172.21.15.22:0/1537298239 >> v2:172.21.15.151:3300/0 conn(0x7f30bc15fe20 0x7f30bc160290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:40.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.796+0000 7f30c29e2640 1 -- 172.21.15.22:0/1537298239 >> v1:172.21.15.151:6789/0 conn(0x7f30bc15c160 legacy=0x7f30bc06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:40.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.796+0000 7f30c29e2640 1 -- 172.21.15.22:0/1537298239 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30bc0cd680 con 0x7f30bc0ccaa0 2024-03-29T04:17:40.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.796+0000 7f30c29e2640 1 --2- 172.21.15.22:0/1537298239 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc0ccaa0 0x7f30bc0cced0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f30b40097c0 tx=0x7f30b401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33b69812629a2531 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:40.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.796+0000 7f30c11df640 1 -- 172.21.15.22:0/1537298239 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b40a7070 con 0x7f30bc0ccaa0 2024-03-29T04:17:40.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.796+0000 7f30c11df640 1 -- 172.21.15.22:0/1537298239 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b40b0070 con 0x7f30bc0ccaa0 2024-03-29T04:17:40.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.796+0000 7f30c11df640 1 -- 172.21.15.22:0/1537298239 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b40ab870 con 0x7f30bc0ccaa0 2024-03-29T04:17:40.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c4445640 1 -- 172.21.15.22:0/1537298239 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc0ccaa0 msgr2=0x7f30bc0cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:40.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c4445640 1 --2- 172.21.15.22:0/1537298239 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc0ccaa0 0x7f30bc0cced0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f30b40097c0 tx=0x7f30b401c470 comp rx=0 tx=0).stop 2024-03-29T04:17:40.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c4445640 1 -- 172.21.15.22:0/1537298239 shutdown_connections 2024-03-29T04:17:40.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c4445640 1 --2- 172.21.15.22:0/1537298239 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc0ccaa0 0x7f30bc0cced0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:40.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c4445640 1 --2- 172.21.15.22:0/1537298239 >> v2:172.21.15.151:3300/0 conn(0x7f30bc15fe20 0x7f30bc160290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:40.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c4445640 1 -- 172.21.15.22:0/1537298239 >> 172.21.15.22:0/1537298239 conn(0x7f30bc0d0c90 msgr2=0x7f30bc0d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:40.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c4445640 1 -- 172.21.15.22:0/1537298239 shutdown_connections 2024-03-29T04:17:40.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c4445640 1 -- 172.21.15.22:0/1537298239 wait complete. 2024-03-29T04:17:40.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c4445640 1 Processor -- start 2024-03-29T04:17:40.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c4445640 1 -- start start 2024-03-29T04:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c4445640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc0ccaa0 0x7f30bc146370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c4445640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30bc15c160 0x7f30bc165e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c4445640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc15fe20 0x7f30bc14b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c4445640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f30bc0cf420 con 0x7f30bc15fe20 2024-03-29T04:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c4445640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f30bc0cf2a0 con 0x7f30bc15c160 2024-03-29T04:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c4445640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f30bc15d550 con 0x7f30bc0ccaa0 2024-03-29T04:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c29e2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc15fe20 0x7f30bc14b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c29e2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc15fe20 0x7f30bc14b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46452/0 (socket says 172.21.15.22:46452) 2024-03-29T04:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c29e2640 1 -- 172.21.15.22:0/3555846945 learned_addr learned my addr 172.21.15.22:0/3555846945 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c21e1640 1 --2- 172.21.15.22:0/3555846945 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc0ccaa0 0x7f30bc146370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c19e0640 1 --2- 172.21.15.22:0/3555846945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30bc15c160 0x7f30bc165e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:40.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c29e2640 1 -- 172.21.15.22:0/3555846945 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc0ccaa0 msgr2=0x7f30bc146370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:40.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c29e2640 1 --2- 172.21.15.22:0/3555846945 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc0ccaa0 0x7f30bc146370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:40.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c29e2640 1 -- 172.21.15.22:0/3555846945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30bc15c160 msgr2=0x7f30bc165e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:40.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c29e2640 1 --2- 172.21.15.22:0/3555846945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30bc15c160 0x7f30bc165e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:40.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c29e2640 1 -- 172.21.15.22:0/3555846945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30bc14c3a0 con 0x7f30bc15fe20 2024-03-29T04:17:40.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c21e1640 1 --2- 172.21.15.22:0/3555846945 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc0ccaa0 0x7f30bc146370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:17:40.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.800+0000 7f30c29e2640 1 --2- 172.21.15.22:0/3555846945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc15fe20 0x7f30bc14b2e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f30b40ad040 tx=0x7f30b4019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:40.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.804+0000 7f30b2ffd640 1 -- 172.21.15.22:0/3555846945 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4003820 con 0x7f30bc15fe20 2024-03-29T04:17:40.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.804+0000 7f30b2ffd640 1 -- 172.21.15.22:0/3555846945 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b40039c0 con 0x7f30bc15fe20 2024-03-29T04:17:40.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.804+0000 7f30c4445640 1 -- 172.21.15.22:0/3555846945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30bc14c680 con 0x7f30bc15fe20 2024-03-29T04:17:40.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.804+0000 7f30b2ffd640 1 -- 172.21.15.22:0/3555846945 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4003d90 con 0x7f30bc15fe20 2024-03-29T04:17:40.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.804+0000 7f30c4445640 1 -- 172.21.15.22:0/3555846945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30bc1468b0 con 0x7f30bc15fe20 2024-03-29T04:17:40.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.804+0000 7f30c4445640 1 -- 172.21.15.22:0/3555846945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30bc068900 con 0x7f30bc15fe20 2024-03-29T04:17:40.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.804+0000 7f30b2ffd640 1 -- 172.21.15.22:0/3555846945 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f30b40c3060 con 0x7f30bc15fe20 2024-03-29T04:17:40.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.804+0000 7f30b2ffd640 1 --2- 172.21.15.22:0/3555846945 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f30a0067ff0 0x7f30a006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:40.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.804+0000 7f30b2ffd640 1 -- 172.21.15.22:0/3555846945 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f30b41207c0 con 0x7f30bc15fe20 2024-03-29T04:17:40.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.804+0000 7f30c21e1640 1 --2- 172.21.15.22:0/3555846945 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f30a0067ff0 0x7f30a006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:40.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.804+0000 7f30c21e1640 1 --2- 172.21.15.22:0/3555846945 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f30a0067ff0 0x7f30a006a4b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f30b8005c10 tx=0x7f30b8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:40.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:40.808+0000 7f30b2ffd640 1 -- 172.21.15.22:0/3555846945 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f30b4016090 con 0x7f30bc15fe20 2024-03-29T04:17:40.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.012+0000 7f30c4445640 1 -- 172.21.15.22:0/3555846945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f30bc165ad0 con 0x7f30bc15fe20 2024-03-29T04:17:41.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.640+0000 7f30b2ffd640 1 -- 172.21.15.22:0/3555846945 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f30b40b3380 con 0x7f30bc15fe20 2024-03-29T04:17:41.623 INFO:teuthology.orchestra.run.smithi022.stderr:added data pool 4 to fsmap 2024-03-29T04:17:41.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.648+0000 7f30c4445640 1 -- 172.21.15.22:0/3555846945 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f30a0067ff0 msgr2=0x7f30a006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:41.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.648+0000 7f30c4445640 1 --2- 172.21.15.22:0/3555846945 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f30a0067ff0 0x7f30a006a4b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f30b8005c10 tx=0x7f30b8009040 comp rx=0 tx=0).stop 2024-03-29T04:17:41.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.648+0000 7f30c4445640 1 -- 172.21.15.22:0/3555846945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc15fe20 msgr2=0x7f30bc14b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:41.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.648+0000 7f30c4445640 1 --2- 172.21.15.22:0/3555846945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc15fe20 0x7f30bc14b2e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f30b40ad040 tx=0x7f30b4019210 comp rx=0 tx=0).stop 2024-03-29T04:17:41.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.652+0000 7f30c4445640 1 -- 172.21.15.22:0/3555846945 shutdown_connections 2024-03-29T04:17:41.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.652+0000 7f30c4445640 1 --2- 172.21.15.22:0/3555846945 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f30a0067ff0 0x7f30a006a4b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:41.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.652+0000 7f30c4445640 1 --2- 172.21.15.22:0/3555846945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc15fe20 0x7f30bc14b2e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:41.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.652+0000 7f30c4445640 1 --2- 172.21.15.22:0/3555846945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30bc15c160 0x7f30bc165e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:41.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.652+0000 7f30c4445640 1 --2- 172.21.15.22:0/3555846945 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc0ccaa0 0x7f30bc146370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:41.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.652+0000 7f30c4445640 1 -- 172.21.15.22:0/3555846945 >> 172.21.15.22:0/3555846945 conn(0x7f30bc0d0c90 msgr2=0x7f30bc06be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:41.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.652+0000 7f30c4445640 1 -- 172.21.15.22:0/3555846945 shutdown_connections 2024-03-29T04:17:41.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.652+0000 7f30c4445640 1 -- 172.21.15.22:0/3555846945 wait complete. 2024-03-29T04:17:41.655 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:17:41.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.800+0000 7f633036d640 1 Processor -- start 2024-03-29T04:17:41.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.800+0000 7f633036d640 1 -- start start 2024-03-29T04:17:41.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.800+0000 7f633036d640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f632815e600 0x7f63280b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:41.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.800+0000 7f633036d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f63280d1100 0x7f63280d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:41.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.800+0000 7f633036d640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63280d3e60 con 0x7f63280b8160 2024-03-29T04:17:41.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.800+0000 7f633036d640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f63280b7450 con 0x7f63280d1100 2024-03-29T04:17:41.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.800+0000 7f633036d640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f63280d1aa0 con 0x7f632815e600 2024-03-29T04:17:41.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.800+0000 7f632d908640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f63280d1100 0x7f63280d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:41.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.800+0000 7f632e90a640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f63280b8160 0x7f63280b85b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:43096/0 (socket says 172.21.15.22:43096) 2024-03-29T04:17:41.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.800+0000 7f632e90a640 1 -- 172.21.15.22:0/2855483890 learned_addr learned my addr 172.21.15.22:0/2855483890 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:41.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.800+0000 7f632e109640 1 --2- 172.21.15.22:0/2855483890 >> v2:172.21.15.151:3300/0 conn(0x7f632815e600 0x7f63280b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:41.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.800+0000 7f632d908640 1 -- 172.21.15.22:0/2855483890 >> v2:172.21.15.151:3300/0 conn(0x7f632815e600 msgr2=0x7f63280b9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:41.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.800+0000 7f632d908640 1 --2- 172.21.15.22:0/2855483890 >> v2:172.21.15.151:3300/0 conn(0x7f632815e600 0x7f63280b9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:41.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.800+0000 7f632d908640 1 -- 172.21.15.22:0/2855483890 >> v1:172.21.15.22:6789/0 conn(0x7f63280b8160 legacy=0x7f63280b85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:41.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.800+0000 7f632d908640 1 -- 172.21.15.22:0/2855483890 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63280b7680 con 0x7f63280d1100 2024-03-29T04:17:41.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.804+0000 7f632d908640 1 --2- 172.21.15.22:0/2855483890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63280d1100 0x7f63280d1560 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f63280b53e0 tx=0x7f631801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6971557bf982de3f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:41.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.804+0000 7f632d107640 1 -- 172.21.15.22:0/2855483890 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63180b0070 con 0x7f63280d1100 2024-03-29T04:17:41.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.804+0000 7f632d107640 1 -- 172.21.15.22:0/2855483890 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63180ab600 con 0x7f63280d1100 2024-03-29T04:17:41.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.804+0000 7f632d107640 1 -- 172.21.15.22:0/2855483890 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63180ab9d0 con 0x7f63280d1100 2024-03-29T04:17:41.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.804+0000 7f633036d640 1 -- 172.21.15.22:0/2855483890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63280d1100 msgr2=0x7f63280d1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:41.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.804+0000 7f633036d640 1 --2- 172.21.15.22:0/2855483890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63280d1100 0x7f63280d1560 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f63280b53e0 tx=0x7f631801c480 comp rx=0 tx=0).stop 2024-03-29T04:17:41.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.804+0000 7f633036d640 1 -- 172.21.15.22:0/2855483890 shutdown_connections 2024-03-29T04:17:41.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.804+0000 7f633036d640 1 --2- 172.21.15.22:0/2855483890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63280d1100 0x7f63280d1560 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:41.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.804+0000 7f633036d640 1 --2- 172.21.15.22:0/2855483890 >> v2:172.21.15.151:3300/0 conn(0x7f632815e600 0x7f63280b9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:41.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.804+0000 7f633036d640 1 -- 172.21.15.22:0/2855483890 >> 172.21.15.22:0/2855483890 conn(0x7f63280b6230 msgr2=0x7f63280b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:41.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.804+0000 7f633036d640 1 -- 172.21.15.22:0/2855483890 shutdown_connections 2024-03-29T04:17:41.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.804+0000 7f633036d640 1 -- 172.21.15.22:0/2855483890 wait complete. 2024-03-29T04:17:41.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f633036d640 1 Processor -- start 2024-03-29T04:17:41.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f633036d640 1 -- start start 2024-03-29T04:17:41.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f633036d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63280b8160 0x7f63281531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:41.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f633036d640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f63280d1100 0x7f6328156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:41.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f633036d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f632815e600 0x7f6328159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:41.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f633036d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f632816b150 con 0x7f63280b8160 2024-03-29T04:17:41.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f633036d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f632816afd0 con 0x7f632815e600 2024-03-29T04:17:41.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f633036d640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f632816b2d0 con 0x7f63280d1100 2024-03-29T04:17:41.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f632e109640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63280b8160 0x7f63281531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:41.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f632e109640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63280b8160 0x7f63281531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46476/0 (socket says 172.21.15.22:46476) 2024-03-29T04:17:41.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f632e109640 1 -- 172.21.15.22:0/2044303046 learned_addr learned my addr 172.21.15.22:0/2044303046 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:41.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f632e90a640 1 --2- 172.21.15.22:0/2044303046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f632815e600 0x7f6328159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:41.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f632d908640 1 --2- 172.21.15.22:0/2044303046 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f63280d1100 0x7f6328156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:41.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f632e109640 1 -- 172.21.15.22:0/2044303046 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f63280d1100 msgr2=0x7f6328156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:41.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f632e109640 1 --2- 172.21.15.22:0/2044303046 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f63280d1100 0x7f6328156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:41.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f632e109640 1 -- 172.21.15.22:0/2044303046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f632815e600 msgr2=0x7f6328159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:41.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f632e109640 1 --2- 172.21.15.22:0/2044303046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f632815e600 0x7f6328159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:41.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f632e109640 1 -- 172.21.15.22:0/2044303046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6328157090 con 0x7f63280b8160 2024-03-29T04:17:41.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f632e90a640 1 --2- 172.21.15.22:0/2044303046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f632815e600 0x7f6328159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:17:41.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f632e109640 1 --2- 172.21.15.22:0/2044303046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63280b8160 0x7f63281531c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f632400d5f0 tx=0x7f632400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:41.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f631effd640 1 -- 172.21.15.22:0/2044303046 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63240125f0 con 0x7f63280b8160 2024-03-29T04:17:41.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f631effd640 1 -- 172.21.15.22:0/2044303046 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6324019070 con 0x7f63280b8160 2024-03-29T04:17:41.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f631effd640 1 -- 172.21.15.22:0/2044303046 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63240129b0 con 0x7f63280b8160 2024-03-29T04:17:41.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f633036d640 1 -- 172.21.15.22:0/2044303046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63280c7f60 con 0x7f63280b8160 2024-03-29T04:17:41.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f633036d640 1 -- 172.21.15.22:0/2044303046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63280c8520 con 0x7f63280b8160 2024-03-29T04:17:41.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f633036d640 1 -- 172.21.15.22:0/2044303046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6328068890 con 0x7f63280b8160 2024-03-29T04:17:41.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.808+0000 7f631effd640 1 -- 172.21.15.22:0/2044303046 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f6324012b50 con 0x7f63280b8160 2024-03-29T04:17:41.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.812+0000 7f631effd640 1 --2- 172.21.15.22:0/2044303046 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f630c067ff0 0x7f630c06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:41.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.812+0000 7f631effd640 1 -- 172.21.15.22:0/2044303046 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6324086bb0 con 0x7f63280b8160 2024-03-29T04:17:41.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.812+0000 7f632d908640 1 --2- 172.21.15.22:0/2044303046 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f630c067ff0 0x7f630c06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:41.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.812+0000 7f632d908640 1 --2- 172.21.15.22:0/2044303046 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f630c067ff0 0x7f630c06a4b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f63180ad000 tx=0x7f6318008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:41.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:41.816+0000 7f631effd640 1 -- 172.21.15.22:0/2044303046 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f63240570b0 con 0x7f63280b8160 2024-03-29T04:17:41.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.012+0000 7f633036d640 1 -- 172.21.15.22:0/2044303046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6328074ad0 con 0x7f63280b8160 2024-03-29T04:17:41.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.016+0000 7f631effd640 1 -- 172.21.15.22:0/2044303046 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5183 (secure 0 0 0) 0x7f63240569b0 con 0x7f63280b8160 2024-03-29T04:17:41.996 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:41.996 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3094991863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3094991863},{"type":"v1","addr":"172.21.15.22:6834","nonce":3094991863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2464050026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2464050026},{"type":"v1","addr":"172.21.15.151:6835","nonce":2464050026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2999052384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2999052384},{"type":"v1","addr":"172.21.15.22:6835","nonce":2999052384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:17:35.918069+0000","modified":"2024-03-29T04:17:41.016073+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6837/1466916671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":1466916671},{"type":"v1","addr":"172.21.15.151:6837","nonce":1466916671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-03-29T04:17:41.997 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 7 2024-03-29T04:17:41.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.016+0000 7f633036d640 1 -- 172.21.15.22:0/2044303046 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f630c067ff0 msgr2=0x7f630c06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:42.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.016+0000 7f633036d640 1 --2- 172.21.15.22:0/2044303046 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f630c067ff0 0x7f630c06a4b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f63180ad000 tx=0x7f6318008040 comp rx=0 tx=0).stop 2024-03-29T04:17:42.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.016+0000 7f633036d640 1 -- 172.21.15.22:0/2044303046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63280b8160 msgr2=0x7f63281531c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:42.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.016+0000 7f633036d640 1 --2- 172.21.15.22:0/2044303046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63280b8160 0x7f63281531c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f632400d5f0 tx=0x7f632400dab0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.016+0000 7f633036d640 1 -- 172.21.15.22:0/2044303046 shutdown_connections 2024-03-29T04:17:42.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.016+0000 7f633036d640 1 --2- 172.21.15.22:0/2044303046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f632815e600 0x7f6328159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.016+0000 7f633036d640 1 --2- 172.21.15.22:0/2044303046 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f63280d1100 0x7f6328156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.016+0000 7f633036d640 1 --2- 172.21.15.22:0/2044303046 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f630c067ff0 0x7f630c06a4b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.016+0000 7f633036d640 1 --2- 172.21.15.22:0/2044303046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63280b8160 0x7f63281531c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.016+0000 7f633036d640 1 -- 172.21.15.22:0/2044303046 >> 172.21.15.22:0/2044303046 conn(0x7f63280b6230 msgr2=0x7f632806c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:42.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.016+0000 7f633036d640 1 -- 172.21.15.22:0/2044303046 shutdown_connections 2024-03-29T04:17:42.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.016+0000 7f633036d640 1 -- 172.21.15.22:0/2044303046 wait complete. 2024-03-29T04:17:42.017 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:17:42.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.152+0000 7f5527d69640 1 Processor -- start 2024-03-29T04:17:42.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.152+0000 7f5527d69640 1 -- start start 2024-03-29T04:17:42.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.152+0000 7f5527d69640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f552006c670 0x7f55200cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:42.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.152+0000 7f5527d69640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f552006ca50 0x7f55200ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.152+0000 7f5527d69640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55200ccef0 con 0x7f55200b41f0 2024-03-29T04:17:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.152+0000 7f5527d69640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f55200b34c0 con 0x7f552006ca50 2024-03-29T04:17:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.152+0000 7f5527d69640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f55200b3710 con 0x7f552006c670 2024-03-29T04:17:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.152+0000 7f5526306640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f552006ca50 0x7f55200ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.152+0000 7f5525304640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f55200b41f0 0x7f552006e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:43102/0 (socket says 172.21.15.22:43102) 2024-03-29T04:17:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.152+0000 7f5525304640 1 -- 172.21.15.22:0/3631570057 learned_addr learned my addr 172.21.15.22:0/3631570057 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.152+0000 7f5525b05640 1 --2- 172.21.15.22:0/3631570057 >> v2:172.21.15.151:3300/0 conn(0x7f552006c670 0x7f55200cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.152+0000 7f5526306640 1 -- 172.21.15.22:0/3631570057 >> v2:172.21.15.151:3300/0 conn(0x7f552006c670 msgr2=0x7f55200cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.152+0000 7f5526306640 1 --2- 172.21.15.22:0/3631570057 >> v2:172.21.15.151:3300/0 conn(0x7f552006c670 0x7f55200cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.152+0000 7f5526306640 1 -- 172.21.15.22:0/3631570057 >> v1:172.21.15.22:6789/0 conn(0x7f55200b41f0 legacy=0x7f552006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.152+0000 7f5524b03640 1 -- 172.21.15.22:0/3631570057 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1009697617 0 0) 0x7f55200ccef0 con 0x7f55200b41f0 2024-03-29T04:17:42.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.152+0000 7f5526306640 1 -- 172.21.15.22:0/3631570057 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55200b39f0 con 0x7f552006ca50 2024-03-29T04:17:42.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5526306640 1 --2- 172.21.15.22:0/3631570057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f552006ca50 0x7f55200ce5b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f552006ea10 tx=0x7f551801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d811a9ba7ef8ed88 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:42.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5524b03640 1 -- 172.21.15.22:0/3631570057 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55180b2070 con 0x7f552006ca50 2024-03-29T04:17:42.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5524b03640 1 -- 172.21.15.22:0/3631570057 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55180a5900 con 0x7f552006ca50 2024-03-29T04:17:42.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5524b03640 1 -- 172.21.15.22:0/3631570057 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55180a5cd0 con 0x7f552006ca50 2024-03-29T04:17:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5527d69640 1 -- 172.21.15.22:0/3631570057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f552006ca50 msgr2=0x7f55200ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5527d69640 1 --2- 172.21.15.22:0/3631570057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f552006ca50 0x7f55200ce5b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f552006ea10 tx=0x7f551801c930 comp rx=0 tx=0).stop 2024-03-29T04:17:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5527d69640 1 -- 172.21.15.22:0/3631570057 shutdown_connections 2024-03-29T04:17:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5527d69640 1 --2- 172.21.15.22:0/3631570057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f552006ca50 0x7f55200ce5b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5527d69640 1 --2- 172.21.15.22:0/3631570057 >> v2:172.21.15.151:3300/0 conn(0x7f552006c670 0x7f55200cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5527d69640 1 -- 172.21.15.22:0/3631570057 >> 172.21.15.22:0/3631570057 conn(0x7f55200cfee0 msgr2=0x7f55200d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5527d69640 1 -- 172.21.15.22:0/3631570057 shutdown_connections 2024-03-29T04:17:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5527d69640 1 -- 172.21.15.22:0/3631570057 wait complete. 2024-03-29T04:17:42.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5527d69640 1 Processor -- start 2024-03-29T04:17:42.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5527d69640 1 -- start start 2024-03-29T04:17:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5527d69640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f552006c670 0x7f552015b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5527d69640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f552006ca50 0x7f55201666a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5527d69640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f55200b41f0 0x7f5520143480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5527d69640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5520169070 con 0x7f552006ca50 2024-03-29T04:17:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5527d69640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5520168ef0 con 0x7f552006c670 2024-03-29T04:17:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5527d69640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f55201691f0 con 0x7f55200b41f0 2024-03-29T04:17:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5525304640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f552006ca50 0x7f55201666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5525304640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f552006ca50 0x7f55201666a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46486/0 (socket says 172.21.15.22:46486) 2024-03-29T04:17:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5525304640 1 -- 172.21.15.22:0/3707639573 learned_addr learned my addr 172.21.15.22:0/3707639573 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5525b05640 1 --2- 172.21.15.22:0/3707639573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f552006c670 0x7f552015b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5526306640 1 --2- 172.21.15.22:0/3707639573 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f55200b41f0 0x7f5520143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:42.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5525304640 1 -- 172.21.15.22:0/3707639573 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f55200b41f0 msgr2=0x7f5520143480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:42.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5525304640 1 --2- 172.21.15.22:0/3707639573 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f55200b41f0 0x7f5520143480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5525304640 1 -- 172.21.15.22:0/3707639573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f552006c670 msgr2=0x7f552015b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:42.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5525304640 1 --2- 172.21.15.22:0/3707639573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f552006c670 0x7f552015b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5525304640 1 -- 172.21.15.22:0/3707639573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5520145a50 con 0x7f552006ca50 2024-03-29T04:17:42.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.156+0000 7f5525304640 1 --2- 172.21.15.22:0/3707639573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f552006ca50 0x7f55201666a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f55100011f0 tx=0x7f551000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:42.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.160+0000 7f55167fc640 1 -- 172.21.15.22:0/3707639573 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5510018070 con 0x7f552006ca50 2024-03-29T04:17:42.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.160+0000 7f55167fc640 1 -- 172.21.15.22:0/3707639573 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55100074e0 con 0x7f552006ca50 2024-03-29T04:17:42.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.160+0000 7f55167fc640 1 -- 172.21.15.22:0/3707639573 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5510004a10 con 0x7f552006ca50 2024-03-29T04:17:42.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.160+0000 7f5527d69640 1 -- 172.21.15.22:0/3707639573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f552013f950 con 0x7f552006ca50 2024-03-29T04:17:42.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.160+0000 7f5527d69640 1 -- 172.21.15.22:0/3707639573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5520165ad0 con 0x7f552006ca50 2024-03-29T04:17:42.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.160+0000 7f5527d69640 1 -- 172.21.15.22:0/3707639573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5520068890 con 0x7f552006ca50 2024-03-29T04:17:42.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.160+0000 7f55167fc640 1 -- 172.21.15.22:0/3707639573 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f5510004bb0 con 0x7f552006ca50 2024-03-29T04:17:42.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.160+0000 7f55167fc640 1 --2- 172.21.15.22:0/3707639573 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5504067fc0 0x7f550406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:42.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.160+0000 7f55167fc640 1 -- 172.21.15.22:0/3707639573 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5510085cc0 con 0x7f552006ca50 2024-03-29T04:17:42.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.160+0000 7f5525b05640 1 --2- 172.21.15.22:0/3707639573 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5504067fc0 0x7f550406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:42.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.160+0000 7f5525b05640 1 --2- 172.21.15.22:0/3707639573 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5504067fc0 0x7f550406a480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f551c009ac0 tx=0x7f551c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:42.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.164+0000 7f55167fc640 1 -- 172.21.15.22:0/3707639573 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f551008c080 con 0x7f552006ca50 2024-03-29T04:17:42.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.324+0000 7f5527d69640 1 -- 172.21.15.22:0/3707639573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55200b32a0 con 0x7f552006ca50 2024-03-29T04:17:42.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.328+0000 7f55167fc640 1 -- 172.21.15.22:0/3707639573 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18527 (secure 0 0 0) 0x7f55100559b0 con 0x7f552006ca50 2024-03-29T04:17:42.308 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:42.309 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":25,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:17:41.015613+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:17:42.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.328+0000 7f5527d69640 1 -- 172.21.15.22:0/3707639573 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5504067fc0 msgr2=0x7f550406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:42.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.328+0000 7f5527d69640 1 --2- 172.21.15.22:0/3707639573 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5504067fc0 0x7f550406a480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f551c009ac0 tx=0x7f551c009190 comp rx=0 tx=0).stop 2024-03-29T04:17:42.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.328+0000 7f5527d69640 1 -- 172.21.15.22:0/3707639573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f552006ca50 msgr2=0x7f55201666a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:42.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.328+0000 7f5527d69640 1 --2- 172.21.15.22:0/3707639573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f552006ca50 0x7f55201666a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f55100011f0 tx=0x7f551000c9b0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.328+0000 7f5527d69640 1 -- 172.21.15.22:0/3707639573 shutdown_connections 2024-03-29T04:17:42.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.328+0000 7f5527d69640 1 --2- 172.21.15.22:0/3707639573 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f55200b41f0 0x7f5520143480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.328+0000 7f5527d69640 1 --2- 172.21.15.22:0/3707639573 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5504067fc0 0x7f550406a480 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.328+0000 7f5527d69640 1 --2- 172.21.15.22:0/3707639573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f552006ca50 0x7f55201666a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.328+0000 7f5527d69640 1 --2- 172.21.15.22:0/3707639573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f552006c670 0x7f552015b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.328+0000 7f5527d69640 1 -- 172.21.15.22:0/3707639573 >> 172.21.15.22:0/3707639573 conn(0x7f55200cfee0 msgr2=0x7f552013b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:42.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.328+0000 7f5527d69640 1 -- 172.21.15.22:0/3707639573 shutdown_connections 2024-03-29T04:17:42.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.328+0000 7f5527d69640 1 -- 172.21.15.22:0/3707639573 wait complete. 2024-03-29T04:17:42.327 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:17:42.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9ad54ae640 1 Processor -- start 2024-03-29T04:17:42.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9ad54ae640 1 -- start start 2024-03-29T04:17:42.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9ad54ae640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9ad00b3f50 0x7f9ad0169310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:42.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9ad54ae640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ad00b4500 con 0x7f9ad00b55a0 2024-03-29T04:17:42.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9ad54ae640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ad00b47e0 con 0x7f9ad00cef80 2024-03-29T04:17:42.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9ad54ae640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f9ad00b4ac0 con 0x7f9ad00b3f50 2024-03-29T04:17:42.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9ace7fc640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f9ad00b55a0 0x7f9ad00b3840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:43106/0 (socket says 172.21.15.22:43106) 2024-03-29T04:17:42.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9ace7fc640 1 -- 172.21.15.22:0/3586237516 learned_addr learned my addr 172.21.15.22:0/3586237516 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:42.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acdffb640 1 -- 172.21.15.22:0/3586237516 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 125973084 0 0) 0x7f9ad00b4500 con 0x7f9ad00b55a0 2024-03-29T04:17:42.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acf7fe640 1 --2- 172.21.15.22:0/3586237516 >> v2:172.21.15.151:3300/0 conn(0x7f9ad00b3f50 0x7f9ad0169310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:42.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acdffb640 1 -- 172.21.15.22:0/3586237516 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ab8003530 con 0x7f9ad00b55a0 2024-03-29T04:17:42.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acdffb640 1 -- 172.21.15.22:0/3586237516 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 369451632 0 0) 0x7f9ab8003530 con 0x7f9ad00b55a0 2024-03-29T04:17:42.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acdffb640 1 -- 172.21.15.22:0/3586237516 >> v2:172.21.15.151:3300/0 conn(0x7f9ad00b3f50 msgr2=0x7f9ad0169310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:42.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acdffb640 1 --2- 172.21.15.22:0/3586237516 >> v2:172.21.15.151:3300/0 conn(0x7f9ad00b3f50 0x7f9ad0169310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acdffb640 1 -- 172.21.15.22:0/3586237516 >> v1:172.21.15.151:6789/0 conn(0x7f9ad00cef80 legacy=0x7f9ad00cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:42.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acdffb640 1 -- 172.21.15.22:0/3586237516 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad00b4da0 con 0x7f9ad00b55a0 2024-03-29T04:17:42.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acdffb640 1 -- 172.21.15.22:0/3586237516 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f9ac4001680 con 0x7f9ad00b55a0 2024-03-29T04:17:42.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acdffb640 1 -- 172.21.15.22:0/3586237516 >> v1:172.21.15.22:6789/0 conn(0x7f9ad00b55a0 legacy=0x7f9ad00b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:42.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acdffb640 1 --2- 172.21.15.22:0/3586237516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ab80039c0 0x7f9ab8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:42.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acdffb640 1 --2- 172.21.15.22:0/3586237516 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ab8004810 0x7f9ab8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:42.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acdffb640 1 --2- 172.21.15.22:0/3586237516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab80051d0 0x7f9ab800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:42.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acdffb640 1 -- 172.21.15.22:0/3586237516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad00b4ac0 con 0x7f9ab80051d0 2024-03-29T04:17:42.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acdffb640 1 -- 172.21.15.22:0/3586237516 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab800c070 con 0x7f9ab80039c0 2024-03-29T04:17:42.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acdffb640 1 -- 172.21.15.22:0/3586237516 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab800c270 con 0x7f9ab8004810 2024-03-29T04:17:42.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acf7fe640 1 --2- 172.21.15.22:0/3586237516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab80051d0 0x7f9ab800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:42.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9ace7fc640 1 --2- 172.21.15.22:0/3586237516 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ab8004810 0x7f9ab8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:42.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9aceffd640 1 --2- 172.21.15.22:0/3586237516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ab80039c0 0x7f9ab8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:42.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acf7fe640 1 -- 172.21.15.22:0/3586237516 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ab8004810 msgr2=0x7f9ab8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:42.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acf7fe640 1 --2- 172.21.15.22:0/3586237516 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ab8004810 0x7f9ab8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acf7fe640 1 -- 172.21.15.22:0/3586237516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ab80039c0 msgr2=0x7f9ab8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:42.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acf7fe640 1 --2- 172.21.15.22:0/3586237516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ab80039c0 0x7f9ab8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9acf7fe640 1 -- 172.21.15.22:0/3586237516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad00b4da0 con 0x7f9ab80051d0 2024-03-29T04:17:42.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.456+0000 7f9aceffd640 1 --2- 172.21.15.22:0/3586237516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ab80039c0 0x7f9ab8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:42.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9acf7fe640 1 -- 172.21.15.22:0/3586237516 reap_dead start 2024-03-29T04:17:42.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9acf7fe640 1 --2- 172.21.15.22:0/3586237516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab80051d0 0x7f9ab800b9d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009ae0 tx=0x7f9ac801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f789985476bd15f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:42.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9acdffb640 1 -- 172.21.15.22:0/3586237516 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac80096b0 con 0x7f9ab80051d0 2024-03-29T04:17:42.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9acdffb640 1 -- 172.21.15.22:0/3586237516 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ac8004020 con 0x7f9ab80051d0 2024-03-29T04:17:42.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9acdffb640 1 -- 172.21.15.22:0/3586237516 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8004410 con 0x7f9ab80051d0 2024-03-29T04:17:42.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ad54ae640 1 -- 172.21.15.22:0/3586237516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab80051d0 msgr2=0x7f9ab800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:42.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ad54ae640 1 --2- 172.21.15.22:0/3586237516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab80051d0 0x7f9ab800b9d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009ae0 tx=0x7f9ac801cab0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ad54ae640 1 -- 172.21.15.22:0/3586237516 shutdown_connections 2024-03-29T04:17:42.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ad54ae640 1 --2- 172.21.15.22:0/3586237516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab80051d0 0x7f9ab800b9d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ad54ae640 1 -- 172.21.15.22:0/3586237516 >> 172.21.15.22:0/3586237516 conn(0x7f9ad00d4310 msgr2=0x7f9ad00d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:42.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ad54ae640 1 -- 172.21.15.22:0/3586237516 shutdown_connections 2024-03-29T04:17:42.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ad54ae640 1 -- 172.21.15.22:0/3586237516 wait complete. 2024-03-29T04:17:42.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ad54ae640 1 Processor -- start 2024-03-29T04:17:42.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ad54ae640 1 -- start start 2024-03-29T04:17:42.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ad54ae640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ab80051d0 0x7f9ad0160050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:42.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ad54ae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ad00c93d0 0x7f9ad01577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:42.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ad54ae640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ad00cd5b0 0x7f9ad00cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:42.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ad54ae640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad00b3840 con 0x7f9ad00c93d0 2024-03-29T04:17:42.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ad54ae640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad00b4380 con 0x7f9ab80051d0 2024-03-29T04:17:42.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ad54ae640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad00b39c0 con 0x7f9ad00cd5b0 2024-03-29T04:17:42.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ace7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ad00c93d0 0x7f9ad01577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:42.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ace7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ad00c93d0 0x7f9ad01577d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46506/0 (socket says 172.21.15.22:46506) 2024-03-29T04:17:42.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ace7fc640 1 -- 172.21.15.22:0/1407483278 learned_addr learned my addr 172.21.15.22:0/1407483278 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:42.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9aceffd640 1 --2- 172.21.15.22:0/1407483278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ab80051d0 0x7f9ad0160050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:42.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9acf7fe640 1 --2- 172.21.15.22:0/1407483278 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ad00cd5b0 0x7f9ad00cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:42.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ace7fc640 1 -- 172.21.15.22:0/1407483278 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ad00cd5b0 msgr2=0x7f9ad00cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:42.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ace7fc640 1 --2- 172.21.15.22:0/1407483278 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ad00cd5b0 0x7f9ad00cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ace7fc640 1 -- 172.21.15.22:0/1407483278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ab80051d0 msgr2=0x7f9ad0160050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:42.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ace7fc640 1 --2- 172.21.15.22:0/1407483278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ab80051d0 0x7f9ad0160050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.460+0000 7f9ace7fc640 1 -- 172.21.15.22:0/1407483278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad00ca120 con 0x7f9ad00c93d0 2024-03-29T04:17:42.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.464+0000 7f9ace7fc640 1 --2- 172.21.15.22:0/1407483278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ad00c93d0 0x7f9ad01577d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4000fc0 tx=0x7f9ac400ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:42.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.464+0000 7f9ab77fe640 1 -- 172.21.15.22:0/1407483278 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400ee30 con 0x7f9ad00c93d0 2024-03-29T04:17:42.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.464+0000 7f9ab77fe640 1 -- 172.21.15.22:0/1407483278 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ac4018070 con 0x7f9ad00c93d0 2024-03-29T04:17:42.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.464+0000 7f9ad54ae640 1 -- 172.21.15.22:0/1407483278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad00ca400 con 0x7f9ad00c93d0 2024-03-29T04:17:42.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.464+0000 7f9ab77fe640 1 -- 172.21.15.22:0/1407483278 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400cb10 con 0x7f9ad00c93d0 2024-03-29T04:17:42.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.464+0000 7f9ad54ae640 1 -- 172.21.15.22:0/1407483278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad00ca9c0 con 0x7f9ad00c93d0 2024-03-29T04:17:42.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.464+0000 7f9ad54ae640 1 -- 172.21.15.22:0/1407483278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad00cdf70 con 0x7f9ad00c93d0 2024-03-29T04:17:42.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.464+0000 7f9ab77fe640 1 -- 172.21.15.22:0/1407483278 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9ac400ccb0 con 0x7f9ad00c93d0 2024-03-29T04:17:42.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.464+0000 7f9ab77fe640 1 --2- 172.21.15.22:0/1407483278 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9abc067ff0 0x7f9abc06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:42.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.464+0000 7f9ab77fe640 1 -- 172.21.15.22:0/1407483278 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9ac4086330 con 0x7f9ad00c93d0 2024-03-29T04:17:42.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.464+0000 7f9aceffd640 1 --2- 172.21.15.22:0/1407483278 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9abc067ff0 0x7f9abc06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:42.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.464+0000 7f9aceffd640 1 --2- 172.21.15.22:0/1407483278 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9abc067ff0 0x7f9abc06a4b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0002020 tx=0x7f9ac0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:42.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.468+0000 7f9ab77fe640 1 -- 172.21.15.22:0/1407483278 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9ac40567a0 con 0x7f9ad00c93d0 2024-03-29T04:17:42.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.624+0000 7f9ad54ae640 1 -- 172.21.15.22:0/1407483278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ad0068890 con 0x7f9ad00c93d0 2024-03-29T04:17:42.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.628+0000 7f9ab77fe640 1 -- 172.21.15.22:0/1407483278 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18527 (secure 0 0 0) 0x7f9ac40560a0 con 0x7f9ad00c93d0 2024-03-29T04:17:42.610 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:42.610 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":25,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:17:41.015613+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:17:42.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.632+0000 7f9ad54ae640 1 -- 172.21.15.22:0/1407483278 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9abc067ff0 msgr2=0x7f9abc06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:42.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.632+0000 7f9ad54ae640 1 --2- 172.21.15.22:0/1407483278 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9abc067ff0 0x7f9abc06a4b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0002020 tx=0x7f9ac0009040 comp rx=0 tx=0).stop 2024-03-29T04:17:42.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.632+0000 7f9ad54ae640 1 -- 172.21.15.22:0/1407483278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ad00c93d0 msgr2=0x7f9ad01577d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:42.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.632+0000 7f9ad54ae640 1 --2- 172.21.15.22:0/1407483278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ad00c93d0 0x7f9ad01577d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4000fc0 tx=0x7f9ac400ebe0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.632+0000 7f9ad54ae640 1 -- 172.21.15.22:0/1407483278 shutdown_connections 2024-03-29T04:17:42.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.632+0000 7f9ad54ae640 1 --2- 172.21.15.22:0/1407483278 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ad00cd5b0 0x7f9ad00cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.632+0000 7f9ad54ae640 1 --2- 172.21.15.22:0/1407483278 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9abc067ff0 0x7f9abc06a4b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.632+0000 7f9ad54ae640 1 --2- 172.21.15.22:0/1407483278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ad00c93d0 0x7f9ad01577d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.632+0000 7f9ad54ae640 1 --2- 172.21.15.22:0/1407483278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ab80051d0 0x7f9ad0160050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.632+0000 7f9ad54ae640 1 -- 172.21.15.22:0/1407483278 >> 172.21.15.22:0/1407483278 conn(0x7f9ad00d4310 msgr2=0x7f9ad00b8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:42.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.632+0000 7f9ad54ae640 1 -- 172.21.15.22:0/1407483278 shutdown_connections 2024-03-29T04:17:42.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.632+0000 7f9ad54ae640 1 -- 172.21.15.22:0/1407483278 wait complete. 2024-03-29T04:17:42.647 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:17:42.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.784+0000 7f9f6135a640 1 Processor -- start 2024-03-29T04:17:42.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.788+0000 7f9f6135a640 1 -- start start 2024-03-29T04:17:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.788+0000 7f9f6135a640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9f5c144270 0x7f9f5c144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.788+0000 7f9f6135a640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9f5c0b2ef0 0x7f9f5c15f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.788+0000 7f9f6135a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9f5c0b4490 0x7f9f5c06c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.788+0000 7f9f6135a640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f9f5c144fa0 con 0x7f9f5c0b4490 2024-03-29T04:17:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.788+0000 7f9f6135a640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f9f5c15fce0 con 0x7f9f5c0b2ef0 2024-03-29T04:17:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.788+0000 7f9f6135a640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9f5c0b33f0 con 0x7f9f5c144270 2024-03-29T04:17:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.788+0000 7f9f5b7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9f5c0b4490 0x7f9f5c06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:42.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.788+0000 7f9f5b7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9f5c0b4490 0x7f9f5c06c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46520/0 (socket says 172.21.15.22:46520) 2024-03-29T04:17:42.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.788+0000 7f9f5b7fe640 1 -- 172.21.15.22:0/2910156040 learned_addr learned my addr 172.21.15.22:0/2910156040 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:42.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.788+0000 7f9f5affd640 1 --2- 172.21.15.22:0/2910156040 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9f5c144270 0x7f9f5c144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:42.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.788+0000 7f9f527fc640 1 --2- 172.21.15.22:0/2910156040 >> v2:172.21.15.151:3300/0 conn(0x7f9f5c0b2ef0 0x7f9f5c15f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:42.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.788+0000 7f9f5b7fe640 1 -- 172.21.15.22:0/2910156040 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9f5c144270 msgr2=0x7f9f5c144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:42.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.788+0000 7f9f5b7fe640 1 --2- 172.21.15.22:0/2910156040 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9f5c144270 0x7f9f5c144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.788+0000 7f9f5b7fe640 1 -- 172.21.15.22:0/2910156040 >> v2:172.21.15.151:3300/0 conn(0x7f9f5c0b2ef0 msgr2=0x7f9f5c15f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:42.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.788+0000 7f9f5b7fe640 1 --2- 172.21.15.22:0/2910156040 >> v2:172.21.15.151:3300/0 conn(0x7f9f5c0b2ef0 0x7f9f5c15f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.788+0000 7f9f5b7fe640 1 -- 172.21.15.22:0/2910156040 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5c0b36d0 con 0x7f9f5c0b4490 2024-03-29T04:17:42.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f5b7fe640 1 --2- 172.21.15.22:0/2910156040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f5c0b4490 0x7f9f5c06c670 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c0d0fa0 tx=0x7f9f5401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f941bd1ab2ede server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:42.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f5a7fc640 1 -- 172.21.15.22:0/2910156040 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f540a5840 con 0x7f9f5c0b4490 2024-03-29T04:17:42.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f5a7fc640 1 -- 172.21.15.22:0/2910156040 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f540a59e0 con 0x7f9f5c0b4490 2024-03-29T04:17:42.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f5a7fc640 1 -- 172.21.15.22:0/2910156040 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f540a5db0 con 0x7f9f5c0b4490 2024-03-29T04:17:42.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f6135a640 1 -- 172.21.15.22:0/2910156040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f5c0b4490 msgr2=0x7f9f5c06c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:42.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f6135a640 1 --2- 172.21.15.22:0/2910156040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f5c0b4490 0x7f9f5c06c670 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c0d0fa0 tx=0x7f9f5401c9c0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f6135a640 1 -- 172.21.15.22:0/2910156040 shutdown_connections 2024-03-29T04:17:42.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f6135a640 1 --2- 172.21.15.22:0/2910156040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f5c0b4490 0x7f9f5c06c670 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f6135a640 1 --2- 172.21.15.22:0/2910156040 >> v2:172.21.15.151:3300/0 conn(0x7f9f5c0b2ef0 0x7f9f5c15f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f6135a640 1 --2- 172.21.15.22:0/2910156040 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9f5c144270 0x7f9f5c144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f6135a640 1 -- 172.21.15.22:0/2910156040 >> 172.21.15.22:0/2910156040 conn(0x7f9f5c0cfee0 msgr2=0x7f9f5c0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:42.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f6135a640 1 -- 172.21.15.22:0/2910156040 shutdown_connections 2024-03-29T04:17:42.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f6135a640 1 -- 172.21.15.22:0/2910156040 wait complete. 2024-03-29T04:17:42.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f6135a640 1 Processor -- start 2024-03-29T04:17:42.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f6135a640 1 -- start start 2024-03-29T04:17:42.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f6135a640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9f5c0b2ef0 0x7f9f5c157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:42.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f6135a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f5c0b4490 0x7f9f5c139df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:42.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f6135a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f5c144270 0x7f9f5c15a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:42.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f6135a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c0cdd10 con 0x7f9f5c0b4490 2024-03-29T04:17:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f6135a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c06cfc0 con 0x7f9f5c144270 2024-03-29T04:17:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f6135a640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9f5c0cde90 con 0x7f9f5c0b2ef0 2024-03-29T04:17:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f527fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f5c0b4490 0x7f9f5c139df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f527fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f5c0b4490 0x7f9f5c139df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46522/0 (socket says 172.21.15.22:46522) 2024-03-29T04:17:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.792+0000 7f9f527fc640 1 -- 172.21.15.22:0/423439597 learned_addr learned my addr 172.21.15.22:0/423439597 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.796+0000 7f9f5affd640 1 --2- 172.21.15.22:0/423439597 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9f5c0b2ef0 0x7f9f5c157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.796+0000 7f9f5b7fe640 1 --2- 172.21.15.22:0/423439597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f5c144270 0x7f9f5c15a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.796+0000 7f9f527fc640 1 -- 172.21.15.22:0/423439597 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9f5c0b2ef0 msgr2=0x7f9f5c157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.796+0000 7f9f527fc640 1 --2- 172.21.15.22:0/423439597 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9f5c0b2ef0 0x7f9f5c157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.796+0000 7f9f527fc640 1 -- 172.21.15.22:0/423439597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f5c144270 msgr2=0x7f9f5c15a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.796+0000 7f9f527fc640 1 --2- 172.21.15.22:0/423439597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f5c144270 0x7f9f5c15a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.796+0000 7f9f527fc640 1 -- 172.21.15.22:0/423439597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5c140c60 con 0x7f9f5c0b4490 2024-03-29T04:17:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.796+0000 7f9f5affd640 1 --2- 172.21.15.22:0/423439597 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9f5c0b2ef0 0x7f9f5c157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:17:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.796+0000 7f9f5b7fe640 1 --2- 172.21.15.22:0/423439597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f5c144270 0x7f9f5c15a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:17:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.796+0000 7f9f527fc640 1 --2- 172.21.15.22:0/423439597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f5c0b4490 0x7f9f5c139df0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400e870 tx=0x7f9f4400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:42.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.796+0000 7f9f53fff640 1 -- 172.21.15.22:0/423439597 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44017070 con 0x7f9f5c0b4490 2024-03-29T04:17:42.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.796+0000 7f9f53fff640 1 -- 172.21.15.22:0/423439597 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f4400ee60 con 0x7f9f5c0b4490 2024-03-29T04:17:42.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.796+0000 7f9f6135a640 1 -- 172.21.15.22:0/423439597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5c140f40 con 0x7f9f5c0b4490 2024-03-29T04:17:42.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.796+0000 7f9f53fff640 1 -- 172.21.15.22:0/423439597 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4400cb80 con 0x7f9f5c0b4490 2024-03-29T04:17:42.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.796+0000 7f9f6135a640 1 -- 172.21.15.22:0/423439597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f5c141500 con 0x7f9f5c0b4490 2024-03-29T04:17:42.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.796+0000 7f9f6135a640 1 -- 172.21.15.22:0/423439597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f5c068890 con 0x7f9f5c0b4490 2024-03-29T04:17:42.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.796+0000 7f9f53fff640 1 -- 172.21.15.22:0/423439597 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9f4401a070 con 0x7f9f5c0b4490 2024-03-29T04:17:42.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.796+0000 7f9f53fff640 1 --2- 172.21.15.22:0/423439597 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9f3c067fc0 0x7f9f3c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:42.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.796+0000 7f9f53fff640 1 -- 172.21.15.22:0/423439597 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9f44086f20 con 0x7f9f5c0b4490 2024-03-29T04:17:42.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.796+0000 7f9f5affd640 1 --2- 172.21.15.22:0/423439597 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9f3c067fc0 0x7f9f3c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:42.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.796+0000 7f9f5affd640 1 --2- 172.21.15.22:0/423439597 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9f3c067fc0 0x7f9f3c06a480 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009bb0 tx=0x7f9f4c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:42.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.800+0000 7f9f53fff640 1 -- 172.21.15.22:0/423439597 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9f44057360 con 0x7f9f5c0b4490 2024-03-29T04:17:42.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.992+0000 7f9f6135a640 1 -- 172.21.15.22:0/423439597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f5c13f6f0 con 0x7f9f5c0b4490 2024-03-29T04:17:42.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.996+0000 7f9f53fff640 1 -- 172.21.15.22:0/423439597 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5183 (secure 0 0 0) 0x7f9f44056c60 con 0x7f9f5c0b4490 2024-03-29T04:17:42.976 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:42.976 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3094991863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3094991863},{"type":"v1","addr":"172.21.15.22:6834","nonce":3094991863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2464050026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2464050026},{"type":"v1","addr":"172.21.15.151:6835","nonce":2464050026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2999052384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2999052384},{"type":"v1","addr":"172.21.15.22:6835","nonce":2999052384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:17:35.918069+0000","modified":"2024-03-29T04:17:41.016073+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6837/1466916671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":1466916671},{"type":"v1","addr":"172.21.15.151:6837","nonce":1466916671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-03-29T04:17:42.977 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 7 2024-03-29T04:17:42.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.996+0000 7f9f6135a640 1 -- 172.21.15.22:0/423439597 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9f3c067fc0 msgr2=0x7f9f3c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:42.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.996+0000 7f9f6135a640 1 --2- 172.21.15.22:0/423439597 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9f3c067fc0 0x7f9f3c06a480 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009bb0 tx=0x7f9f4c008040 comp rx=0 tx=0).stop 2024-03-29T04:17:42.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.996+0000 7f9f6135a640 1 -- 172.21.15.22:0/423439597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f5c0b4490 msgr2=0x7f9f5c139df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:42.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.996+0000 7f9f6135a640 1 --2- 172.21.15.22:0/423439597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f5c0b4490 0x7f9f5c139df0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400e870 tx=0x7f9f4400ed30 comp rx=0 tx=0).stop 2024-03-29T04:17:42.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.996+0000 7f9f6135a640 1 -- 172.21.15.22:0/423439597 shutdown_connections 2024-03-29T04:17:42.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.996+0000 7f9f6135a640 1 --2- 172.21.15.22:0/423439597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f5c144270 0x7f9f5c15a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.996+0000 7f9f6135a640 1 --2- 172.21.15.22:0/423439597 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9f3c067fc0 0x7f9f3c06a480 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.996+0000 7f9f6135a640 1 --2- 172.21.15.22:0/423439597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f5c0b4490 0x7f9f5c139df0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.996+0000 7f9f6135a640 1 --2- 172.21.15.22:0/423439597 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9f5c0b2ef0 0x7f9f5c157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:42.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.996+0000 7f9f6135a640 1 -- 172.21.15.22:0/423439597 >> 172.21.15.22:0/423439597 conn(0x7f9f5c0cfee0 msgr2=0x7f9f5c13cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:42.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.996+0000 7f9f6135a640 1 -- 172.21.15.22:0/423439597 shutdown_connections 2024-03-29T04:17:42.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:42.996+0000 7f9f6135a640 1 -- 172.21.15.22:0/423439597 wait complete. 2024-03-29T04:17:42.994 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:17:43.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.136+0000 7fc603963640 1 Processor -- start 2024-03-29T04:17:43.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.136+0000 7fc603963640 1 -- start start 2024-03-29T04:17:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.136+0000 7fc603963640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc5fc06c670 0x7fc5fc0cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.136+0000 7fc603963640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc5fc0b41f0 0x7fc5fc06e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.136+0000 7fc603963640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5fc06ca50 0x7fc5fc15fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.136+0000 7fc603963640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fc5fc0b34c0 con 0x7fc5fc06c670 2024-03-29T04:17:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.136+0000 7fc603963640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fc5fc06e5c0 con 0x7fc5fc0b41f0 2024-03-29T04:17:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.136+0000 7fc603963640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc5fc0ccef0 con 0x7fc5fc06ca50 2024-03-29T04:17:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.136+0000 7fc6016ff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc5fc06c670 0x7fc5fc0cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.136+0000 7fc6016ff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc5fc06c670 0x7fc5fc0cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46534/0 (socket says 172.21.15.22:46534) 2024-03-29T04:17:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.136+0000 7fc6016ff640 1 -- 172.21.15.22:0/4282069488 learned_addr learned my addr 172.21.15.22:0/4282069488 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.136+0000 7fc600efe640 1 --2- 172.21.15.22:0/4282069488 >> v2:172.21.15.151:3300/0 conn(0x7fc5fc0b41f0 0x7fc5fc06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.136+0000 7fc601f00640 1 --2- 172.21.15.22:0/4282069488 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5fc06ca50 0x7fc5fc15fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.136+0000 7fc6016ff640 1 -- 172.21.15.22:0/4282069488 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5fc06ca50 msgr2=0x7fc5fc15fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:43.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.136+0000 7fc6016ff640 1 --2- 172.21.15.22:0/4282069488 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5fc06ca50 0x7fc5fc15fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.136+0000 7fc6016ff640 1 -- 172.21.15.22:0/4282069488 >> v2:172.21.15.151:3300/0 conn(0x7fc5fc0b41f0 msgr2=0x7fc5fc06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:43.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.136+0000 7fc6016ff640 1 --2- 172.21.15.22:0/4282069488 >> v2:172.21.15.151:3300/0 conn(0x7fc5fc0b41f0 0x7fc5fc06e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.136+0000 7fc6016ff640 1 -- 172.21.15.22:0/4282069488 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5fc0b3710 con 0x7fc5fc06c670 2024-03-29T04:17:43.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc6016ff640 1 --2- 172.21.15.22:0/4282069488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5fc06c670 0x7fc5fc0cc910 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc06ebe0 tx=0x7fc5f801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db7850c0021f99a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:43.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc5f3fff640 1 -- 172.21.15.22:0/4282069488 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f80b0070 con 0x7fc5fc06c670 2024-03-29T04:17:43.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc5f3fff640 1 -- 172.21.15.22:0/4282069488 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f80ab600 con 0x7fc5fc06c670 2024-03-29T04:17:43.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc5f3fff640 1 -- 172.21.15.22:0/4282069488 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f80ab9d0 con 0x7fc5fc06c670 2024-03-29T04:17:43.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc603963640 1 -- 172.21.15.22:0/4282069488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5fc06c670 msgr2=0x7fc5fc0cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:43.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc603963640 1 --2- 172.21.15.22:0/4282069488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5fc06c670 0x7fc5fc0cc910 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc06ebe0 tx=0x7fc5f801c260 comp rx=0 tx=0).stop 2024-03-29T04:17:43.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc603963640 1 -- 172.21.15.22:0/4282069488 shutdown_connections 2024-03-29T04:17:43.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc603963640 1 --2- 172.21.15.22:0/4282069488 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5fc06ca50 0x7fc5fc15fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc603963640 1 --2- 172.21.15.22:0/4282069488 >> v2:172.21.15.151:3300/0 conn(0x7fc5fc0b41f0 0x7fc5fc06e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc603963640 1 --2- 172.21.15.22:0/4282069488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5fc06c670 0x7fc5fc0cc910 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc603963640 1 -- 172.21.15.22:0/4282069488 >> 172.21.15.22:0/4282069488 conn(0x7fc5fc0cfee0 msgr2=0x7fc5fc0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:43.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc603963640 1 -- 172.21.15.22:0/4282069488 shutdown_connections 2024-03-29T04:17:43.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc603963640 1 -- 172.21.15.22:0/4282069488 wait complete. 2024-03-29T04:17:43.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc603963640 1 Processor -- start 2024-03-29T04:17:43.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc603963640 1 -- start start 2024-03-29T04:17:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc603963640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5fc06c670 0x7fc5fc13e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc603963640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5fc06ca50 0x7fc5fc159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc603963640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5fc0b41f0 0x7fc5fc0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc603963640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc06cfe0 con 0x7fc5fc0b41f0 2024-03-29T04:17:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc603963640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc06ce60 con 0x7fc5fc06c670 2024-03-29T04:17:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc603963640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc5fc0ce5b0 con 0x7fc5fc06ca50 2024-03-29T04:17:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc601f00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5fc0b41f0 0x7fc5fc0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc601f00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5fc0b41f0 0x7fc5fc0ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46536/0 (socket says 172.21.15.22:46536) 2024-03-29T04:17:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc601f00640 1 -- 172.21.15.22:0/3978264380 learned_addr learned my addr 172.21.15.22:0/3978264380 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc6016ff640 1 --2- 172.21.15.22:0/3978264380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5fc06c670 0x7fc5fc13e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc600efe640 1 --2- 172.21.15.22:0/3978264380 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5fc06ca50 0x7fc5fc159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc601f00640 1 -- 172.21.15.22:0/3978264380 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5fc06ca50 msgr2=0x7fc5fc159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc601f00640 1 --2- 172.21.15.22:0/3978264380 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5fc06ca50 0x7fc5fc159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc601f00640 1 -- 172.21.15.22:0/3978264380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5fc06c670 msgr2=0x7fc5fc13e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc601f00640 1 --2- 172.21.15.22:0/3978264380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5fc06c670 0x7fc5fc13e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc601f00640 1 -- 172.21.15.22:0/3978264380 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5fc159960 con 0x7fc5fc0b41f0 2024-03-29T04:17:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc600efe640 1 --2- 172.21.15.22:0/3978264380 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5fc06ca50 0x7fc5fc159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:17:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc601f00640 1 --2- 172.21.15.22:0/3978264380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5fc0b41f0 0x7fc5fc0ced10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4007e90 tx=0x7fc5f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc5f1ffb640 1 -- 172.21.15.22:0/3978264380 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4014420 con 0x7fc5fc0b41f0 2024-03-29T04:17:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc5f1ffb640 1 -- 172.21.15.22:0/3978264380 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f4019070 con 0x7fc5fc0b41f0 2024-03-29T04:17:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc603963640 1 -- 172.21.15.22:0/3978264380 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5fc13d7a0 con 0x7fc5fc0b41f0 2024-03-29T04:17:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.140+0000 7fc5f1ffb640 1 -- 172.21.15.22:0/3978264380 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4014820 con 0x7fc5fc0b41f0 2024-03-29T04:17:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.144+0000 7fc603963640 1 -- 172.21.15.22:0/3978264380 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5fc1403d0 con 0x7fc5fc0b41f0 2024-03-29T04:17:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.144+0000 7fc603963640 1 -- 172.21.15.22:0/3978264380 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5fc13fee0 con 0x7fc5fc0b41f0 2024-03-29T04:17:43.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.144+0000 7fc5f1ffb640 1 -- 172.21.15.22:0/3978264380 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fc5f40149c0 con 0x7fc5fc0b41f0 2024-03-29T04:17:43.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.144+0000 7fc5f1ffb640 1 --2- 172.21.15.22:0/3978264380 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc5e0067fc0 0x7fc5e006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.144+0000 7fc5f1ffb640 1 -- 172.21.15.22:0/3978264380 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc5f4087040 con 0x7fc5fc0b41f0 2024-03-29T04:17:43.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.144+0000 7fc6016ff640 1 --2- 172.21.15.22:0/3978264380 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc5e0067fc0 0x7fc5e006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.144+0000 7fc6016ff640 1 --2- 172.21.15.22:0/3978264380 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc5e0067fc0 0x7fc5e006a480 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc5f80ad000 tx=0x7fc5f8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:43.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.148+0000 7fc5f1ffb640 1 -- 172.21.15.22:0/3978264380 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fc5fc159960 con 0x7fc5fc0b41f0 2024-03-29T04:17:43.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.308+0000 7fc603963640 1 -- 172.21.15.22:0/3978264380 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5fc068890 con 0x7fc5fc0b41f0 2024-03-29T04:17:43.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.312+0000 7fc5f1ffb640 1 -- 172.21.15.22:0/3978264380 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18514 (secure 0 0 0) 0x7fc5f40574b0 con 0x7fc5fc0b41f0 2024-03-29T04:17:43.293 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:43.293 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:17:42.645632+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:17:43.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.312+0000 7fc603963640 1 -- 172.21.15.22:0/3978264380 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc5e0067fc0 msgr2=0x7fc5e006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:43.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.312+0000 7fc603963640 1 --2- 172.21.15.22:0/3978264380 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc5e0067fc0 0x7fc5e006a480 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc5f80ad000 tx=0x7fc5f8008040 comp rx=0 tx=0).stop 2024-03-29T04:17:43.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.312+0000 7fc603963640 1 -- 172.21.15.22:0/3978264380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5fc0b41f0 msgr2=0x7fc5fc0ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:43.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.312+0000 7fc603963640 1 --2- 172.21.15.22:0/3978264380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5fc0b41f0 0x7fc5fc0ced10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4007e90 tx=0x7fc5f400e4b0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.312+0000 7fc603963640 1 -- 172.21.15.22:0/3978264380 shutdown_connections 2024-03-29T04:17:43.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.312+0000 7fc603963640 1 --2- 172.21.15.22:0/3978264380 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc5e0067fc0 0x7fc5e006a480 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.312+0000 7fc603963640 1 --2- 172.21.15.22:0/3978264380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5fc0b41f0 0x7fc5fc0ced10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.312+0000 7fc603963640 1 --2- 172.21.15.22:0/3978264380 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5fc06ca50 0x7fc5fc159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.312+0000 7fc603963640 1 --2- 172.21.15.22:0/3978264380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5fc06c670 0x7fc5fc13e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.312+0000 7fc603963640 1 -- 172.21.15.22:0/3978264380 >> 172.21.15.22:0/3978264380 conn(0x7fc5fc0cfee0 msgr2=0x7fc5fc13bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:43.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.312+0000 7fc603963640 1 -- 172.21.15.22:0/3978264380 shutdown_connections 2024-03-29T04:17:43.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.312+0000 7fc603963640 1 -- 172.21.15.22:0/3978264380 wait complete. 2024-03-29T04:17:43.714 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T04:17:35.918069+0000', 'modified': '2024-03-29T04:17:41.016073+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4294}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4294': {'gid': 4294, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.151:6837/1466916671', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 1466916671}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 1466916671}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4294, 'qdb_cluster': [4294]} 2024-03-29T04:17:43.714 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T04:17:43.715 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-29T04:17:43.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.868+0000 7f2ccf7b3640 1 Processor -- start 2024-03-29T04:17:43.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.868+0000 7f2ccf7b3640 1 -- start start 2024-03-29T04:17:43.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.868+0000 7f2ccf7b3640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2cc80b55a0 0x7f2cc80b3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.868+0000 7f2ccf7b3640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cc80b4500 con 0x7f2cc80cef80 2024-03-29T04:17:43.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.868+0000 7f2ccf7b3640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cc80b47e0 con 0x7f2cc80b3d80 2024-03-29T04:17:43.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.868+0000 7f2ccf7b3640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f2cc80b4ac0 con 0x7f2cc80b55a0 2024-03-29T04:17:43.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.868+0000 7f2ccd54f640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f2cc80cef80 0x7f2cc80cf360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:43118/0 (socket says 172.21.15.22:43118) 2024-03-29T04:17:43.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.868+0000 7f2ccd54f640 1 -- 172.21.15.22:0/2039416586 learned_addr learned my addr 172.21.15.22:0/2039416586 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:43.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.868+0000 7f2cccd4e640 1 --2- 172.21.15.22:0/2039416586 >> v2:172.21.15.22:3300/0 conn(0x7f2cc80b55a0 0x7f2cc80b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.868+0000 7f2cbffff640 1 -- 172.21.15.22:0/2039416586 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2684179477 0 0) 0x7f2cc80b4500 con 0x7f2cc80cef80 2024-03-29T04:17:43.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.868+0000 7f2cbffff640 1 -- 172.21.15.22:0/2039416586 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2cb4003530 con 0x7f2cc80cef80 2024-03-29T04:17:43.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.868+0000 7f2cccd4e640 1 -- 172.21.15.22:0/2039416586 >> v1:172.21.15.151:6789/0 conn(0x7f2cc80b3d80 legacy=0x7f2cc8168300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:43.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.868+0000 7f2cccd4e640 1 -- 172.21.15.22:0/2039416586 >> v1:172.21.15.22:6789/0 conn(0x7f2cc80cef80 legacy=0x7f2cc80cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:43.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.868+0000 7f2cccd4e640 1 -- 172.21.15.22:0/2039416586 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc80b4da0 con 0x7f2cc80b55a0 2024-03-29T04:17:43.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.872+0000 7f2cccd4e640 1 --2- 172.21.15.22:0/2039416586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc80b55a0 0x7f2cc80b3840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2cb80097c0 tx=0x7f2cb801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e9dab7c0861bca server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:43.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.872+0000 7f2cbffff640 1 -- 172.21.15.22:0/2039416586 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb80b0070 con 0x7f2cc80b55a0 2024-03-29T04:17:43.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.872+0000 7f2cbffff640 1 -- 172.21.15.22:0/2039416586 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cb80ab600 con 0x7f2cc80b55a0 2024-03-29T04:17:43.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.872+0000 7f2cbffff640 1 -- 172.21.15.22:0/2039416586 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb80ab9b0 con 0x7f2cc80b55a0 2024-03-29T04:17:43.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.872+0000 7f2ccf7b3640 1 -- 172.21.15.22:0/2039416586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc80b55a0 msgr2=0x7f2cc80b3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:43.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.872+0000 7f2ccf7b3640 1 --2- 172.21.15.22:0/2039416586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc80b55a0 0x7f2cc80b3840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2cb80097c0 tx=0x7f2cb801c4a0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.872+0000 7f2ccf7b3640 1 -- 172.21.15.22:0/2039416586 shutdown_connections 2024-03-29T04:17:43.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.872+0000 7f2ccf7b3640 1 --2- 172.21.15.22:0/2039416586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc80b55a0 0x7f2cc80b3840 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.872+0000 7f2ccf7b3640 1 -- 172.21.15.22:0/2039416586 >> 172.21.15.22:0/2039416586 conn(0x7f2cc80d4310 msgr2=0x7f2cc80d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:43.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.872+0000 7f2ccf7b3640 1 -- 172.21.15.22:0/2039416586 shutdown_connections 2024-03-29T04:17:43.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.872+0000 7f2ccf7b3640 1 -- 172.21.15.22:0/2039416586 wait complete. 2024-03-29T04:17:43.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.872+0000 7f2ccf7b3640 1 Processor -- start 2024-03-29T04:17:43.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.872+0000 7f2ccf7b3640 1 -- start start 2024-03-29T04:17:43.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2ccf7b3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc80b3d80 0x7f2cc8150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2ccf7b3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cc80b55a0 0x7f2cc815c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2ccf7b3640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2cc80cef80 0x7f2cc81615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2ccf7b3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc816c7d0 con 0x7f2cc80b3d80 2024-03-29T04:17:43.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2ccf7b3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc816c650 con 0x7f2cc80b55a0 2024-03-29T04:17:43.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2ccf7b3640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc816c950 con 0x7f2cc80cef80 2024-03-29T04:17:43.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2ccd54f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc80b3d80 0x7f2cc8150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2ccd54f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc80b3d80 0x7f2cc8150310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46558/0 (socket says 172.21.15.22:46558) 2024-03-29T04:17:43.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2ccd54f640 1 -- 172.21.15.22:0/617635045 learned_addr learned my addr 172.21.15.22:0/617635045 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:43.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2cccd4e640 1 --2- 172.21.15.22:0/617635045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cc80b55a0 0x7f2cc815c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2ccdd50640 1 --2- 172.21.15.22:0/617635045 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2cc80cef80 0x7f2cc81615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2ccd54f640 1 -- 172.21.15.22:0/617635045 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2cc80cef80 msgr2=0x7f2cc81615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:43.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2ccd54f640 1 --2- 172.21.15.22:0/617635045 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2cc80cef80 0x7f2cc81615e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2ccd54f640 1 -- 172.21.15.22:0/617635045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cc80b55a0 msgr2=0x7f2cc815c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:43.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2ccd54f640 1 --2- 172.21.15.22:0/617635045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cc80b55a0 0x7f2cc815c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2ccd54f640 1 -- 172.21.15.22:0/617635045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc815cd60 con 0x7f2cc80b3d80 2024-03-29T04:17:43.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2ccd54f640 1 --2- 172.21.15.22:0/617635045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc80b3d80 0x7f2cc8150310 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400c420 tx=0x7f2cc400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:43.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2cbdffb640 1 -- 172.21.15.22:0/617635045 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4017070 con 0x7f2cc80b3d80 2024-03-29T04:17:43.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2cbdffb640 1 -- 172.21.15.22:0/617635045 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cc4005cf0 con 0x7f2cc80b3d80 2024-03-29T04:17:43.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2ccf7b3640 1 -- 172.21.15.22:0/617635045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc815d040 con 0x7f2cc80b3d80 2024-03-29T04:17:43.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2cbdffb640 1 -- 172.21.15.22:0/617635045 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40072d0 con 0x7f2cc80b3d80 2024-03-29T04:17:43.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2ccf7b3640 1 -- 172.21.15.22:0/617635045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc815d600 con 0x7f2cc80b3d80 2024-03-29T04:17:43.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.876+0000 7f2cbdffb640 1 -- 172.21.15.22:0/617635045 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f2cc401a070 con 0x7f2cc80b3d80 2024-03-29T04:17:43.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cbdffb640 1 --2- 172.21.15.22:0/617635045 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2cac067fc0 0x7f2cac06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cbdffb640 1 -- 172.21.15.22:0/617635045 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2cc4085bc0 con 0x7f2cc80b3d80 2024-03-29T04:17:43.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cd2935000 1 Processor -- start 2024-03-29T04:17:43.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cd2935000 1 -- start start 2024-03-29T04:17:43.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cccd4e640 1 --2- 172.21.15.22:0/617635045 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2cac067fc0 0x7f2cac06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cd2935000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff07104a0 0x55dff0710880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cd2935000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff0710dc0 0x55dff06c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cd2935000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55dff06c40b0 0x55dff06c4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cd2935000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dff0516950 con 0x55dff07104a0 2024-03-29T04:17:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cd2935000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55dff04b3990 con 0x55dff0710dc0 2024-03-29T04:17:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cd2935000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x55dff0512900 con 0x55dff06c40b0 2024-03-29T04:17:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cccd4e640 1 --2- 172.21.15.22:0/617635045 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2cac067fc0 0x7f2cac06a480 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2cb80ad000 tx=0x7f2cb80a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2ccd54f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff07104a0 0x55dff0710880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2ccd54f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff07104a0 0x55dff0710880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46564/0 (socket says 172.21.15.22:46564) 2024-03-29T04:17:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2ccd54f640 1 -- 172.21.15.22:0/1481626264 learned_addr learned my addr 172.21.15.22:0/1481626264 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cccd4e640 1 --2- 172.21.15.22:0/1481626264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff0710dc0 0x55dff06c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2ccdd50640 1 --2- 172.21.15.22:0/1481626264 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55dff06c40b0 0x55dff06c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2ccd54f640 1 -- 172.21.15.22:0/1481626264 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55dff06c40b0 msgr2=0x55dff06c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2ccd54f640 1 --2- 172.21.15.22:0/1481626264 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55dff06c40b0 0x55dff06c4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2ccd54f640 1 -- 172.21.15.22:0/1481626264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff0710dc0 msgr2=0x55dff06c3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2ccd54f640 1 --2- 172.21.15.22:0/1481626264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff0710dc0 0x55dff06c3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2ccd54f640 1 -- 172.21.15.22:0/1481626264 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff055b100 con 0x55dff07104a0 2024-03-29T04:17:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cccd4e640 1 --2- 172.21.15.22:0/1481626264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff0710dc0 0x55dff06c3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:17:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2ccd54f640 1 --2- 172.21.15.22:0/1481626264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff07104a0 0x55dff0710880 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4083d40 tx=0x7f2cc400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3cf36b3e2e65f9a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cb2ffd640 1 -- 172.21.15.22:0/1481626264 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4085070 con 0x55dff07104a0 2024-03-29T04:17:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cb2ffd640 1 -- 172.21.15.22:0/1481626264 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cc4053cd0 con 0x55dff07104a0 2024-03-29T04:17:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cb2ffd640 1 -- 172.21.15.22:0/1481626264 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc405ae60 con 0x55dff07104a0 2024-03-29T04:17:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cd2935000 1 -- 172.21.15.22:0/1481626264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff07104a0 msgr2=0x55dff0710880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:43.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cd2935000 1 --2- 172.21.15.22:0/1481626264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff07104a0 0x55dff0710880 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4083d40 tx=0x7f2cc400cf90 comp rx=0 tx=0).stop 2024-03-29T04:17:43.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cd2935000 1 -- 172.21.15.22:0/1481626264 shutdown_connections 2024-03-29T04:17:43.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cd2935000 1 --2- 172.21.15.22:0/1481626264 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55dff06c40b0 0x55dff06c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cd2935000 1 --2- 172.21.15.22:0/1481626264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff0710dc0 0x55dff06c3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cd2935000 1 --2- 172.21.15.22:0/1481626264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff07104a0 0x55dff0710880 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cd2935000 1 -- 172.21.15.22:0/1481626264 >> 172.21.15.22:0/1481626264 conn(0x55dff05aeb80 msgr2=0x55dff06f3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:43.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cd2935000 1 -- 172.21.15.22:0/1481626264 shutdown_connections 2024-03-29T04:17:43.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cd2935000 1 -- 172.21.15.22:0/1481626264 wait complete. 2024-03-29T04:17:43.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cd2935000 1 -- start start 2024-03-29T04:17:43.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.880+0000 7f2cd2935000 1 Processor -- start 2024-03-29T04:17:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2cd2935000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff06c40b0 0x55dff06cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2cd2935000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55dff07104a0 0x55dff06f4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2cd2935000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff0710dc0 0x55dff06f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2cd2935000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dff04ae7b0 con 0x55dff0710dc0 2024-03-29T04:17:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2cd2935000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55dff04d43e0 con 0x55dff06c40b0 2024-03-29T04:17:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2cd2935000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x55dff051af80 con 0x55dff07104a0 2024-03-29T04:17:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2cd2935000 10 client.0 _ll_register_callbacks cb 0x55dff05770f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:17:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2ccdd50640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff0710dc0 0x55dff06f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2ccdd50640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff0710dc0 0x55dff06f4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46580/0 (socket says 172.21.15.22:46580) 2024-03-29T04:17:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2ccdd50640 1 -- 172.21.15.22:0/2903376175 learned_addr learned my addr 172.21.15.22:0/2903376175 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2ccd54f640 1 --2- 172.21.15.22:0/2903376175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff06c40b0 0x55dff06cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2cccd4e640 1 --2- 172.21.15.22:0/2903376175 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55dff07104a0 0x55dff06f4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2ccdd50640 1 -- 172.21.15.22:0/2903376175 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55dff07104a0 msgr2=0x55dff06f4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2ccdd50640 1 --2- 172.21.15.22:0/2903376175 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55dff07104a0 0x55dff06f4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2ccdd50640 1 -- 172.21.15.22:0/2903376175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff06c40b0 msgr2=0x55dff06cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2ccdd50640 1 --2- 172.21.15.22:0/2903376175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff06c40b0 0x55dff06cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2ccdd50640 1 -- 172.21.15.22:0/2903376175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff04e08c0 con 0x55dff0710dc0 2024-03-29T04:17:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2ccdd50640 1 --2- 172.21.15.22:0/2903376175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff0710dc0 0x55dff06f4bd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0005920 tx=0x7f2cc00019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b858eba2ca4bd3c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2ca7fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-29T04:17:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2ca7fff640 1 -- 172.21.15.22:0/2903376175 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0013570 con 0x55dff0710dc0 2024-03-29T04:17:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2ca7fff640 1 -- 172.21.15.22:0/2903376175 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2cc0013710 con 0x55dff0710dc0 2024-03-29T04:17:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2cd2935000 1 -- 172.21.15.22:0/2903376175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc4053cd0 con 0x55dff0710dc0 2024-03-29T04:17:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2ca7fff640 1 -- 172.21.15.22:0/2903376175 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f2cc0021050 con 0x55dff0710dc0 2024-03-29T04:17:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2cd2935000 10 client.4407 fetch_fsmap learned FSMap version 7 2024-03-29T04:17:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2cd2935000 1 -- 172.21.15.22:0/2903376175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55dff049ee80 con 0x55dff0710dc0 2024-03-29T04:17:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2ca7fff640 1 -- 172.21.15.22:0/2903376175 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cc4053cd0 con 0x55dff0710dc0 2024-03-29T04:17:43.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2ca7fff640 1 -- 172.21.15.22:0/2903376175 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f2cc0024020 con 0x55dff0710dc0 2024-03-29T04:17:43.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2cd2935000 10 client.4407 fetch_fsmap finished waiting for FSMap version 7 2024-03-29T04:17:43.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2cd2935000 10 client.4407 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-29T04:17:43.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2cd2935000 20 client.4407 populate_metadata read hostname 'smithi022' 2024-03-29T04:17:43.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2cd2935000 1 --2- 172.21.15.22:0/2903376175 >> [v2:172.21.15.151:6836/1466916671,v1:172.21.15.151:6837/1466916671] conn(0x55dff06f3140 0x55dff0722550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2cd2935000 4 client.4407 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:17:43.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2cd2935000 1 -- 172.21.15.22:0/2903376175 --> [v2:172.21.15.151:6836/1466916671,v1:172.21.15.151:6837/1466916671] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dff04b1bc0 con 0x55dff06f3140 2024-03-29T04:17:43.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2ccd54f640 1 --2- 172.21.15.22:0/2903376175 >> [v2:172.21.15.151:6836/1466916671,v1:172.21.15.151:6837/1466916671] conn(0x55dff06f3140 0x55dff0722550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2ca7fff640 10 client.4407 ms_handle_connect on v2:172.21.15.151:6836/1466916671 2024-03-29T04:17:43.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.884+0000 7f2ccd54f640 1 --2- 172.21.15.22:0/2903376175 >> [v2:172.21.15.151:6836/1466916671,v1:172.21.15.151:6837/1466916671] conn(0x55dff06f3140 0x55dff0722550 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dbc2ae7e25b8c67 server_cookie=6cfc46851904297b in_seq=0 out_seq=0 2024-03-29T04:17:43.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.888+0000 7f2ca7fff640 1 -- 172.21.15.22:0/2903376175 <== mds.0 v2:172.21.15.151:6836/1466916671 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55dff04b1bc0 con 0x55dff06f3140 2024-03-29T04:17:43.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.888+0000 7f2ca7fff640 10 client.4407 handle_command_reply: tid=0 2024-03-29T04:17:43.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.888+0000 7f2cd2935000 1 client.4407 shutdown 2024-03-29T04:17:43.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.888+0000 7f2cd2935000 1 -- 172.21.15.22:0/2903376175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff0710dc0 msgr2=0x55dff06f4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:43.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.888+0000 7f2cd2935000 1 --2- 172.21.15.22:0/2903376175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff0710dc0 0x55dff06f4bd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0005920 tx=0x7f2cc00019e0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.888+0000 7f2cd2935000 1 -- 172.21.15.22:0/2903376175 shutdown_connections 2024-03-29T04:17:43.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.888+0000 7f2cd2935000 1 --2- 172.21.15.22:0/2903376175 >> [v2:172.21.15.151:6836/1466916671,v1:172.21.15.151:6837/1466916671] conn(0x55dff06f3140 0x55dff0722550 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.888+0000 7f2cd2935000 1 --2- 172.21.15.22:0/2903376175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff0710dc0 0x55dff06f4bd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.888+0000 7f2cd2935000 1 --2- 172.21.15.22:0/2903376175 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55dff07104a0 0x55dff06f4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.888+0000 7f2cd2935000 1 --2- 172.21.15.22:0/2903376175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff06c40b0 0x55dff06cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.888+0000 7f2cd2935000 1 -- 172.21.15.22:0/2903376175 >> 172.21.15.22:0/2903376175 conn(0x55dff05aeb80 msgr2=0x55dff06c3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:43.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.888+0000 7f2ca7fff640 0 client.4407 ms_handle_reset on v2:172.21.15.151:6836/1466916671 2024-03-29T04:17:43.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.888+0000 7f2cd2935000 1 -- 172.21.15.22:0/2903376175 shutdown_connections 2024-03-29T04:17:43.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.888+0000 7f2cd2935000 1 -- 172.21.15.22:0/2903376175 wait complete. 2024-03-29T04:17:43.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.888+0000 7f2cd2935000 20 client.4407 trim_cache size 0 max 16384 2024-03-29T04:17:43.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2cd2935000 1 Processor -- start 2024-03-29T04:17:43.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2cd2935000 1 -- start start 2024-03-29T04:17:43.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2cd2935000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff06c40b0 0x55dff0722550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2cd2935000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55dff07104a0 0x55dff0722a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2cd2935000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff0710dc0 0x55dff0722fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2cd2935000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc8204fe0 con 0x55dff0710dc0 2024-03-29T04:17:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2cd2935000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc8204040 con 0x55dff06c40b0 2024-03-29T04:17:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2cd2935000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc82033a0 con 0x55dff07104a0 2024-03-29T04:17:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2ccdd50640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff0710dc0 0x55dff0722fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2ccdd50640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff0710dc0 0x55dff0722fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46588/0 (socket says 172.21.15.22:46588) 2024-03-29T04:17:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2ccdd50640 1 -- 172.21.15.22:0/304592399 learned_addr learned my addr 172.21.15.22:0/304592399 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2ccd54f640 1 --2- 172.21.15.22:0/304592399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff06c40b0 0x55dff0722550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2cccd4e640 1 --2- 172.21.15.22:0/304592399 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55dff07104a0 0x55dff0722a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2ccdd50640 1 -- 172.21.15.22:0/304592399 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55dff07104a0 msgr2=0x55dff0722a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2ccdd50640 1 --2- 172.21.15.22:0/304592399 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55dff07104a0 0x55dff0722a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2ccdd50640 1 -- 172.21.15.22:0/304592399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff06c40b0 msgr2=0x55dff0722550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:43.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2ccdd50640 1 --2- 172.21.15.22:0/304592399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff06c40b0 0x55dff0722550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2ccdd50640 1 -- 172.21.15.22:0/304592399 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff04d1490 con 0x55dff0710dc0 2024-03-29T04:17:43.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2ccdd50640 1 --2- 172.21.15.22:0/304592399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff0710dc0 0x55dff0722fd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0002240 tx=0x7f2cc000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f925c64459642f00 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:43.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2cb27fc640 1 -- 172.21.15.22:0/304592399 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0001ee0 con 0x55dff0710dc0 2024-03-29T04:17:43.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2cb27fc640 1 -- 172.21.15.22:0/304592399 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cc002a070 con 0x55dff0710dc0 2024-03-29T04:17:43.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.904+0000 7f2cb27fc640 1 -- 172.21.15.22:0/304592399 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0013570 con 0x55dff0710dc0 2024-03-29T04:17:43.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 1 -- 172.21.15.22:0/304592399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff0710dc0 msgr2=0x55dff0722fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:43.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 1 --2- 172.21.15.22:0/304592399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff0710dc0 0x55dff0722fd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0002240 tx=0x7f2cc000e000 comp rx=0 tx=0).stop 2024-03-29T04:17:43.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 1 -- 172.21.15.22:0/304592399 shutdown_connections 2024-03-29T04:17:43.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 1 --2- 172.21.15.22:0/304592399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff0710dc0 0x55dff0722fd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 1 --2- 172.21.15.22:0/304592399 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55dff07104a0 0x55dff0722a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 1 --2- 172.21.15.22:0/304592399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff06c40b0 0x55dff0722550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 1 -- 172.21.15.22:0/304592399 >> 172.21.15.22:0/304592399 conn(0x55dff05aeb80 msgr2=0x55dff071d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 1 -- 172.21.15.22:0/304592399 shutdown_connections 2024-03-29T04:17:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 1 -- 172.21.15.22:0/304592399 wait complete. 2024-03-29T04:17:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 1 -- start start 2024-03-29T04:17:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 1 Processor -- start 2024-03-29T04:17:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55dff06c40b0 0x55dff0783520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff07104a0 0x55dff0785a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff0710dc0 0x55dff0787fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dff05189a0 con 0x55dff0710dc0 2024-03-29T04:17:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc82039f0 con 0x55dff07104a0 2024-03-29T04:17:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x55dff05ab350 con 0x55dff06c40b0 2024-03-29T04:17:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 10 client.0 _ll_register_callbacks cb 0x7f2cc0003ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:17:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2ccdd50640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff0710dc0 0x55dff0787fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2ccdd50640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff0710dc0 0x55dff0787fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46596/0 (socket says 172.21.15.22:46596) 2024-03-29T04:17:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2ccdd50640 1 -- 172.21.15.22:0/14015574 learned_addr learned my addr 172.21.15.22:0/14015574 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cccd4e640 1 --2- 172.21.15.22:0/14015574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff07104a0 0x55dff0785a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2ccd54f640 1 --2- 172.21.15.22:0/14015574 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55dff06c40b0 0x55dff0783520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2ccdd50640 1 -- 172.21.15.22:0/14015574 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55dff06c40b0 msgr2=0x55dff0783520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2ccdd50640 1 --2- 172.21.15.22:0/14015574 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55dff06c40b0 0x55dff0783520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2ccdd50640 1 -- 172.21.15.22:0/14015574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff07104a0 msgr2=0x55dff0785a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2ccdd50640 1 --2- 172.21.15.22:0/14015574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff07104a0 0x55dff0785a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2ccdd50640 1 -- 172.21.15.22:0/14015574 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff04cfe80 con 0x55dff0710dc0 2024-03-29T04:17:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cccd4e640 1 --2- 172.21.15.22:0/14015574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff07104a0 0x55dff0785a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:17:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2ccdd50640 1 --2- 172.21.15.22:0/14015574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff0710dc0 0x55dff0787fc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0003e00 tx=0x7f2cc00014b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a788c9136fbf4f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2ca7fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-29T04:17:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2ca7fff640 1 -- 172.21.15.22:0/14015574 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0033070 con 0x55dff0710dc0 2024-03-29T04:17:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2ca7fff640 1 -- 172.21.15.22:0/14015574 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2cc0013980 con 0x55dff0710dc0 2024-03-29T04:17:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 1 -- 172.21.15.22:0/14015574 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc002a070 con 0x55dff0710dc0 2024-03-29T04:17:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2ca7fff640 1 -- 172.21.15.22:0/14015574 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f2cc003c050 con 0x55dff0710dc0 2024-03-29T04:17:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 10 client.4413 fetch_fsmap learned FSMap version 7 2024-03-29T04:17:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 1 -- 172.21.15.22:0/14015574 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55dff04bd270 con 0x55dff0710dc0 2024-03-29T04:17:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2ca7fff640 1 -- 172.21.15.22:0/14015574 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cc002a070 con 0x55dff0710dc0 2024-03-29T04:17:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2ca7fff640 1 -- 172.21.15.22:0/14015574 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f2cc003f020 con 0x55dff0710dc0 2024-03-29T04:17:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 10 client.4413 fetch_fsmap finished waiting for FSMap version 7 2024-03-29T04:17:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 10 client.4413 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-29T04:17:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 20 client.4413 populate_metadata read hostname 'smithi022' 2024-03-29T04:17:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 1 --2- 172.21.15.22:0/14015574 >> [v2:172.21.15.151:6836/1466916671,v1:172.21.15.151:6837/1466916671] conn(0x55dff0788500 0x55dff078a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 4 client.4413 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T04:17:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2cd2935000 1 -- 172.21.15.22:0/14015574 --> [v2:172.21.15.151:6836/1466916671,v1:172.21.15.151:6837/1466916671] -- command(tid 0: {"prefix": "status"}) -- 0x55dff0523e90 con 0x55dff0788500 2024-03-29T04:17:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2ccd54f640 1 --2- 172.21.15.22:0/14015574 >> [v2:172.21.15.151:6836/1466916671,v1:172.21.15.151:6837/1466916671] conn(0x55dff0788500 0x55dff078a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2ccd54f640 1 --2- 172.21.15.22:0/14015574 >> [v2:172.21.15.151:6836/1466916671,v1:172.21.15.151:6837/1466916671] conn(0x55dff0788500 0x55dff078a8f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56de93d401fd96fc server_cookie=36c0ba1b72f299e9 in_seq=0 out_seq=0 2024-03-29T04:17:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.908+0000 7f2ca7fff640 10 client.4413 ms_handle_connect on v2:172.21.15.151:6836/1466916671 2024-03-29T04:17:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2ca7fff640 1 -- 172.21.15.22:0/14015574 <== mds.0 v2:172.21.15.151:6836/1466916671 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55dff0523e90 con 0x55dff0788500 2024-03-29T04:17:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2ca7fff640 10 client.4413 handle_command_reply: tid=0 2024-03-29T04:17:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2cd2935000 1 client.4413 shutdown 2024-03-29T04:17:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2cd2935000 1 -- 172.21.15.22:0/14015574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff0710dc0 msgr2=0x55dff0787fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2cd2935000 1 --2- 172.21.15.22:0/14015574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff0710dc0 0x55dff0787fc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0003e00 tx=0x7f2cc00014b0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2cd2935000 1 -- 172.21.15.22:0/14015574 shutdown_connections 2024-03-29T04:17:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2cd2935000 1 --2- 172.21.15.22:0/14015574 >> [v2:172.21.15.151:6836/1466916671,v1:172.21.15.151:6837/1466916671] conn(0x55dff0788500 0x55dff078a8f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2cd2935000 1 --2- 172.21.15.22:0/14015574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dff0710dc0 0x55dff0787fc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2cd2935000 1 --2- 172.21.15.22:0/14015574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dff07104a0 0x55dff0785a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2ca7fff640 0 client.4413 ms_handle_reset on v2:172.21.15.151:6836/1466916671 2024-03-29T04:17:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2cd2935000 1 --2- 172.21.15.22:0/14015574 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55dff06c40b0 0x55dff0783520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2cd2935000 1 -- 172.21.15.22:0/14015574 >> 172.21.15.22:0/14015574 conn(0x55dff05aeb80 msgr2=0x55dff06c8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2cd2935000 1 -- 172.21.15.22:0/14015574 shutdown_connections 2024-03-29T04:17:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2cd2935000 1 -- 172.21.15.22:0/14015574 wait complete. 2024-03-29T04:17:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2cd2935000 20 client.4413 trim_cache size 0 max 16384 2024-03-29T04:17:43.899 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-29T04:17:43.899 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "9f2f9779-fe00-4763-9aa1-fd3768e2f135", 2024-03-29T04:17:43.900 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-03-29T04:17:43.900 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 4294, 2024-03-29T04:17:43.900 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-03-29T04:17:43.900 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-03-29T04:17:43.900 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-03-29T04:17:43.900 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 7.3394232500000003, 2024-03-29T04:17:43.900 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 7, 2024-03-29T04:17:43.900 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 26, 2024-03-29T04:17:43.900 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 24, 2024-03-29T04:17:43.900 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 14.413778647999999 2024-03-29T04:17:43.900 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-29T04:17:43.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2ccf7b3640 1 -- 172.21.15.22:0/617635045 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2cac067fc0 msgr2=0x7f2cac06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:43.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2ccf7b3640 1 --2- 172.21.15.22:0/617635045 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2cac067fc0 0x7f2cac06a480 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2cb80ad000 tx=0x7f2cb80a6000 comp rx=0 tx=0).stop 2024-03-29T04:17:43.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2ccf7b3640 1 -- 172.21.15.22:0/617635045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc80b3d80 msgr2=0x7f2cc8150310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2ccf7b3640 1 --2- 172.21.15.22:0/617635045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc80b3d80 0x7f2cc8150310 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400c420 tx=0x7f2cc400c8e0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2ccf7b3640 1 -- 172.21.15.22:0/617635045 shutdown_connections 2024-03-29T04:17:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2ccf7b3640 1 --2- 172.21.15.22:0/617635045 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2cc80cef80 0x7f2cc81615e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2ccf7b3640 1 --2- 172.21.15.22:0/617635045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cc80b55a0 0x7f2cc815c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2ccf7b3640 1 --2- 172.21.15.22:0/617635045 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2cac067fc0 0x7f2cac06a480 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2ccf7b3640 1 --2- 172.21.15.22:0/617635045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cc80b3d80 0x7f2cc8150310 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2ccf7b3640 1 -- 172.21.15.22:0/617635045 >> 172.21.15.22:0/617635045 conn(0x7f2cc80d4310 msgr2=0x7f2cc8150f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2ccf7b3640 1 -- 172.21.15.22:0/617635045 shutdown_connections 2024-03-29T04:17:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:43.912+0000 7f2ccf7b3640 1 -- 172.21.15.22:0/617635045 wait complete. 2024-03-29T04:17:43.908 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T04:17:43.908 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:17:43.908 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:17:44.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.036+0000 7fb10dd16640 1 Processor -- start 2024-03-29T04:17:44.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.036+0000 7fb10dd16640 1 -- start start 2024-03-29T04:17:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.036+0000 7fb10dd16640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fb1080d18e0 0x7fb108160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.036+0000 7fb10dd16640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb1080c6ef0 0x7fb1080c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.036+0000 7fb10dd16640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1080cfce0 con 0x7fb108145670 2024-03-29T04:17:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.036+0000 7fb10dd16640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fb1080d1e30 con 0x7fb1080d18e0 2024-03-29T04:17:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.036+0000 7fb10dd16640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb10806f430 con 0x7fb1080c6ef0 2024-03-29T04:17:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.036+0000 7fb0fffff640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fb1080d18e0 0x7fb108160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.036+0000 7fb1077fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb108145670 0x7fb10815fc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:43128/0 (socket says 172.21.15.22:43128) 2024-03-29T04:17:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.036+0000 7fb1077fe640 1 -- 172.21.15.22:0/285032819 learned_addr learned my addr 172.21.15.22:0/285032819 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.036+0000 7fb107fff640 1 --2- 172.21.15.22:0/285032819 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb1080c6ef0 0x7fb1080c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.036+0000 7fb106ffd640 1 -- 172.21.15.22:0/285032819 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1601024736 0 0) 0x7fb1080cfce0 con 0x7fb108145670 2024-03-29T04:17:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.036+0000 7fb106ffd640 1 -- 172.21.15.22:0/285032819 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0f4003530 con 0x7fb108145670 2024-03-29T04:17:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.036+0000 7fb0fffff640 1 -- 172.21.15.22:0/285032819 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb1080c6ef0 msgr2=0x7fb1080c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:44.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.036+0000 7fb0fffff640 1 --2- 172.21.15.22:0/285032819 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb1080c6ef0 0x7fb1080c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:44.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.036+0000 7fb0fffff640 1 -- 172.21.15.22:0/285032819 >> v1:172.21.15.22:6789/0 conn(0x7fb108145670 legacy=0x7fb10815fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:44.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.036+0000 7fb0fffff640 1 -- 172.21.15.22:0/285032819 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10806f710 con 0x7fb1080d18e0 2024-03-29T04:17:44.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb0fffff640 1 --2- 172.21.15.22:0/285032819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1080d18e0 0x7fb108160380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00097c0 tx=0x7fb0f001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=709e891b2dfe2e91 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:44.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb106ffd640 1 -- 172.21.15.22:0/285032819 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f00b1070 con 0x7fb1080d18e0 2024-03-29T04:17:44.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb106ffd640 1 -- 172.21.15.22:0/285032819 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f00ac410 con 0x7fb1080d18e0 2024-03-29T04:17:44.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb106ffd640 1 -- 172.21.15.22:0/285032819 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f00ac7c0 con 0x7fb1080d18e0 2024-03-29T04:17:44.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb10dd16640 1 -- 172.21.15.22:0/285032819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1080d18e0 msgr2=0x7fb108160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:44.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb10dd16640 1 --2- 172.21.15.22:0/285032819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1080d18e0 0x7fb108160380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00097c0 tx=0x7fb0f001c4a0 comp rx=0 tx=0).stop 2024-03-29T04:17:44.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb10dd16640 1 -- 172.21.15.22:0/285032819 shutdown_connections 2024-03-29T04:17:44.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb10dd16640 1 --2- 172.21.15.22:0/285032819 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb1080c6ef0 0x7fb1080c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:44.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb10dd16640 1 --2- 172.21.15.22:0/285032819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1080d18e0 0x7fb108160380 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:44.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb10dd16640 1 -- 172.21.15.22:0/285032819 >> 172.21.15.22:0/285032819 conn(0x7fb1080b3c80 msgr2=0x7fb1080b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:44.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb10dd16640 1 -- 172.21.15.22:0/285032819 shutdown_connections 2024-03-29T04:17:44.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb10dd16640 1 -- 172.21.15.22:0/285032819 wait complete. 2024-03-29T04:17:44.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb10dd16640 1 Processor -- start 2024-03-29T04:17:44.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb10dd16640 1 -- start start 2024-03-29T04:17:44.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb10dd16640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1080c6ef0 0x7fb10815ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:44.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb10dd16640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1080d18e0 0x7fb108142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:44.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb10dd16640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb108145670 0x7fb10815e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb10dd16640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb1080d0340 con 0x7fb1080c6ef0 2024-03-29T04:17:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb10dd16640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb1080d01c0 con 0x7fb1080d18e0 2024-03-29T04:17:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb10dd16640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb1081608c0 con 0x7fb108145670 2024-03-29T04:17:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb1077fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1080c6ef0 0x7fb10815ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb1077fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1080c6ef0 0x7fb10815ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46606/0 (socket says 172.21.15.22:46606) 2024-03-29T04:17:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb1077fe640 1 -- 172.21.15.22:0/1128380137 learned_addr learned my addr 172.21.15.22:0/1128380137 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb0fffff640 1 --2- 172.21.15.22:0/1128380137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1080d18e0 0x7fb108142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb107fff640 1 --2- 172.21.15.22:0/1128380137 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb108145670 0x7fb10815e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb1077fe640 1 -- 172.21.15.22:0/1128380137 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb108145670 msgr2=0x7fb10815e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb1077fe640 1 --2- 172.21.15.22:0/1128380137 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb108145670 0x7fb10815e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb1077fe640 1 -- 172.21.15.22:0/1128380137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1080d18e0 msgr2=0x7fb108142670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb1077fe640 1 --2- 172.21.15.22:0/1128380137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1080d18e0 0x7fb108142670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb1077fe640 1 -- 172.21.15.22:0/1128380137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10815d760 con 0x7fb1080c6ef0 2024-03-29T04:17:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb107fff640 1 --2- 172.21.15.22:0/1128380137 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb108145670 0x7fb10815e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.040+0000 7fb0fffff640 1 --2- 172.21.15.22:0/1128380137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1080d18e0 0x7fb108142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:17:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.044+0000 7fb1077fe640 1 --2- 172.21.15.22:0/1128380137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1080c6ef0 0x7fb10815ba80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800c450 tx=0x7fb0f800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.044+0000 7fb104ff9640 1 -- 172.21.15.22:0/1128380137 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8017070 con 0x7fb1080c6ef0 2024-03-29T04:17:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.044+0000 7fb104ff9640 1 -- 172.21.15.22:0/1128380137 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f8005cf0 con 0x7fb1080c6ef0 2024-03-29T04:17:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.044+0000 7fb10dd16640 1 -- 172.21.15.22:0/1128380137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb10815da40 con 0x7fb1080c6ef0 2024-03-29T04:17:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.044+0000 7fb104ff9640 1 -- 172.21.15.22:0/1128380137 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8007290 con 0x7fb1080c6ef0 2024-03-29T04:17:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.044+0000 7fb10dd16640 1 -- 172.21.15.22:0/1128380137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb10815d450 con 0x7fb1080c6ef0 2024-03-29T04:17:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.044+0000 7fb10dd16640 1 -- 172.21.15.22:0/1128380137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb108068890 con 0x7fb1080c6ef0 2024-03-29T04:17:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.044+0000 7fb104ff9640 1 -- 172.21.15.22:0/1128380137 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb0f801a070 con 0x7fb1080c6ef0 2024-03-29T04:17:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.044+0000 7fb104ff9640 1 --2- 172.21.15.22:0/1128380137 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb0e8067fc0 0x7fb0e806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.044+0000 7fb104ff9640 1 -- 172.21.15.22:0/1128380137 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb0f8085c70 con 0x7fb1080c6ef0 2024-03-29T04:17:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.044+0000 7fb0fffff640 1 --2- 172.21.15.22:0/1128380137 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb0e8067fc0 0x7fb0e806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.044+0000 7fb0fffff640 1 --2- 172.21.15.22:0/1128380137 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb0e8067fc0 0x7fb0e806a480 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00ae000 tx=0x7fb0f00a6040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:44.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.048+0000 7fb104ff9640 1 -- 172.21.15.22:0/1128380137 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fb0f80560e0 con 0x7fb1080c6ef0 2024-03-29T04:17:44.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.244+0000 7fb10dd16640 1 -- 172.21.15.22:0/1128380137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb1080c8730 con 0x7fb1080c6ef0 2024-03-29T04:17:44.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.244+0000 7fb104ff9640 1 -- 172.21.15.22:0/1128380137 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7fb0f80559e0 con 0x7fb1080c6ef0 2024-03-29T04:17:44.227 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-29T04:17:44.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.248+0000 7fb10dd16640 1 -- 172.21.15.22:0/1128380137 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb0e8067fc0 msgr2=0x7fb0e806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:44.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.248+0000 7fb10dd16640 1 --2- 172.21.15.22:0/1128380137 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb0e8067fc0 0x7fb0e806a480 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00ae000 tx=0x7fb0f00a6040 comp rx=0 tx=0).stop 2024-03-29T04:17:44.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.248+0000 7fb10dd16640 1 -- 172.21.15.22:0/1128380137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1080c6ef0 msgr2=0x7fb10815ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:44.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.248+0000 7fb10dd16640 1 --2- 172.21.15.22:0/1128380137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1080c6ef0 0x7fb10815ba80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800c450 tx=0x7fb0f800c910 comp rx=0 tx=0).stop 2024-03-29T04:17:44.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.248+0000 7fb10dd16640 1 -- 172.21.15.22:0/1128380137 shutdown_connections 2024-03-29T04:17:44.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.248+0000 7fb10dd16640 1 --2- 172.21.15.22:0/1128380137 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb108145670 0x7fb10815e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:44.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.248+0000 7fb10dd16640 1 --2- 172.21.15.22:0/1128380137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1080d18e0 0x7fb108142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:44.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.248+0000 7fb10dd16640 1 --2- 172.21.15.22:0/1128380137 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb0e8067fc0 0x7fb0e806a480 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:44.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.248+0000 7fb10dd16640 1 --2- 172.21.15.22:0/1128380137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1080c6ef0 0x7fb10815ba80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:44.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.248+0000 7fb10dd16640 1 -- 172.21.15.22:0/1128380137 >> 172.21.15.22:0/1128380137 conn(0x7fb1080b3c80 msgr2=0x7fb108141db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:44.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.248+0000 7fb10dd16640 1 -- 172.21.15.22:0/1128380137 shutdown_connections 2024-03-29T04:17:44.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44.248+0000 7fb10dd16640 1 -- 172.21.15.22:0/1128380137 wait complete. 2024-03-29T04:17:44.245 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-29T04:17:44.245 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:17:44.245 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-03-29T04:17:44.245 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-29T04:17:44.245 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:17:44.245 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-29T04:17:44.245 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:17:44.245 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:17:44.245 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-29T04:17:44.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:17:44.246 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-03-29T04:17:44.252 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:17:44.252 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:17:44.252 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:17:44.252 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:17:44.252 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-03-29T04:17:44.252 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:17:44.252 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T04:17:44.252 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-03-29T04:17:44.252 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-03-29T04:17:44.253 INFO:teuthology.orchestra.run.smithi022.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:17:44.253 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:17:44.253 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-03-29T04:17:44.253 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-29T04:17:44.253 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42480sec preferred_lft 42480sec 2024-03-29T04:17:44.253 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-03-29T04:17:44.253 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:17:44.253 INFO:teuthology.orchestra.run.smithi022.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T04:17:44.254 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-03-29T04:17:44.254 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-03-29T04:17:44.254 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T04:17:44.254 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:17:44.254 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-03-29T04:17:44.254 INFO:teuthology.orchestra.run.smithi022.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T04:17:44.254 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 02:42:df:ac:77:fb brd ff:ff:ff:ff:ff:ff 2024-03-29T04:17:44.254 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T04:17:44.255 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:17:44.255 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-29T04:17:44.255 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:17:44.255 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-29T04:17:44.255 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add name ceph-brx type bridge 2024-03-29T04:17:44.255 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr flush dev ceph-brx 2024-03-29T04:17:44.256 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set ceph-brx up 2024-03-29T04:17:44.256 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-29T04:17:44.256 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-29T04:17:44.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:17:44.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:17:44.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:17:44.376 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:17:44.386 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-03-29T04:17:44.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:17:44.388 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-03-29T04:17:44.456 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-03-29T04:17:44.456 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:17:44.456 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-29T04:17:44.456 INFO:teuthology.orchestra.run.smithi022.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T04:17:44.456 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-29T04:17:44.456 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:17:44.457 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:17:44.457 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-29T04:17:44.457 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T04:17:44.457 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T04:17:44.457 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T04:17:44.457 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-29T04:17:44.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:17:44.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:17:44.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:17:44.579 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-03-29T04:17:44.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:17:44.585 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list-id 2024-03-29T04:17:44.636 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:17:44.636 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-29T04:17:44.636 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T04:17:44.636 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-29T04:17:44.636 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-29T04:17:44.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:17:44.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:17:44.763 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-29T04:17:44.763 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:17:44.763 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-29T04:17:44.763 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-29T04:17:44.763 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-29T04:17:44.763 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-29T04:17:44.763 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-29T04:17:44.764 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-29T04:17:44.764 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-29T04:17:44.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:17:44.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:17:44.883 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:17:44.883 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-29T04:17:44.883 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-03-29T04:17:44.883 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set dev brx.0 master ceph-brx 2024-03-29T04:17:44.883 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-29T04:17:44.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:17:44.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:17:44.987 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-29T04:17:44.987 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:17:44.987 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:17:45.031 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-29T04:17:45.032 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:17:45.032 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:17:45.080 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-03-29T04:17:45.138 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:17:45.186 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-03-29T04:17:45.186 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-03-29T04:17:45.186 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-03-29T04:17:45.186 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-03-29T04:17:45.187 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-03-29T04:17:45.187 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-03-29T04:17:45.187 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-03-29T04:17:45.187 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-03-29T04:17:45.187 INFO:teuthology.orchestra.run.smithi022.stdout:/run/lock 2024-03-29T04:17:45.187 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-03-29T04:17:45.187 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-03-29T04:17:45.187 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-03-29T04:17:45.187 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:17:45.187 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-03-29T04:17:45.187 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-03-29T04:17:45.187 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-03-29T04:17:45.187 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-03-29T04:17:45.187 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/fs/nfsd 2024-03-29T04:17:45.188 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-03-29T04:17:45.188 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-03-29T04:17:45.188 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysusers.service 2024-03-29T04:17:45.188 INFO:teuthology.orchestra.run.smithi022.stdout:/run/rpc_pipefs 2024-03-29T04:17:45.188 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-03-29T04:17:45.188 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-03-29T04:17:45.188 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:17:45.188 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:17:45.188 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:17:45.188 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:17:45.188 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-03-29T04:17:45.188 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T04:17:45.188 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T04:17:45.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:17:45.189 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-03-29T04:17:45.236 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T04:17:45.236 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-29T04:17:45.281 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-03-29T04:17:45.295 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:17:45.344 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-03-29T04:17:45.344 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-03-29T04:17:45.344 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-03-29T04:17:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-03-29T04:17:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-03-29T04:17:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-03-29T04:17:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-03-29T04:17:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-03-29T04:17:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/run/lock 2024-03-29T04:17:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-03-29T04:17:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-03-29T04:17:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-03-29T04:17:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:17:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-03-29T04:17:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-03-29T04:17:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-03-29T04:17:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-03-29T04:17:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/fs/nfsd 2024-03-29T04:17:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-03-29T04:17:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-03-29T04:17:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysusers.service 2024-03-29T04:17:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/run/rpc_pipefs 2024-03-29T04:17:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-03-29T04:17:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-03-29T04:17:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:17:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:17:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:17:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:17:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-03-29T04:17:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T04:17:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T04:17:45.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:17:45.347 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-03-29T04:17:45.359 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi022.stderr:2024-03-29T04:17:45.376+0000 7f2497246580 -1 init, newargv = 0x561a4e1d8560 newargc=15 2024-03-29T04:17:45.359 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi022.stderr:ceph-fuse[33235]: starting ceph client 2024-03-29T04:17:45.368 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi022.stderr:ceph-fuse[33235]: starting fuse 2024-03-29T04:17:46.464 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-03-29T04:17:46.481 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:17:46.533 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-03-29T04:17:46.533 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-03-29T04:17:46.533 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-03-29T04:17:46.534 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-03-29T04:17:46.534 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-03-29T04:17:46.534 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-03-29T04:17:46.534 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-03-29T04:17:46.534 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-03-29T04:17:46.534 INFO:teuthology.orchestra.run.smithi022.stdout:/run/lock 2024-03-29T04:17:46.534 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-03-29T04:17:46.534 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-03-29T04:17:46.534 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-03-29T04:17:46.534 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:17:46.534 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-03-29T04:17:46.534 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-03-29T04:17:46.534 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-03-29T04:17:46.535 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-03-29T04:17:46.535 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/fs/nfsd 2024-03-29T04:17:46.535 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-03-29T04:17:46.535 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-03-29T04:17:46.535 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysusers.service 2024-03-29T04:17:46.535 INFO:teuthology.orchestra.run.smithi022.stdout:/run/rpc_pipefs 2024-03-29T04:17:46.535 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-03-29T04:17:46.535 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-03-29T04:17:46.535 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:17:46.535 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:17:46.535 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:17:46.535 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:17:46.535 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-03-29T04:17:46.535 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T04:17:46.536 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T04:17:46.536 INFO:teuthology.orchestra.run.smithi022.stdout:/home/ubuntu/cephtest/mnt.admin 2024-03-29T04:17:46.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:17:46.537 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-03-29T04:17:46.584 INFO:teuthology.orchestra.run.smithi022.stdout:45 2024-03-29T04:17:46.585 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-03-29T04:17:46.585 DEBUG:teuthology.orchestra.run.smithi022:> sudo stdin-killer -- python3 -c ' 2024-03-29T04:17:46.585 DEBUG:teuthology.orchestra.run.smithi022:> import glob 2024-03-29T04:17:46.585 DEBUG:teuthology.orchestra.run.smithi022:> import re 2024-03-29T04:17:46.585 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-03-29T04:17:46.585 DEBUG:teuthology.orchestra.run.smithi022:> import subprocess 2024-03-29T04:17:46.585 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-29T04:17:46.585 DEBUG:teuthology.orchestra.run.smithi022:> def _find_admin_socket(client_name): 2024-03-29T04:17:46.585 DEBUG:teuthology.orchestra.run.smithi022:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-29T04:17:46.586 DEBUG:teuthology.orchestra.run.smithi022:> files = glob.glob(asok_path) 2024-03-29T04:17:46.586 DEBUG:teuthology.orchestra.run.smithi022:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-29T04:17:46.586 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-29T04:17:46.586 DEBUG:teuthology.orchestra.run.smithi022:> # Given a non-glob path, it better be there 2024-03-29T04:17:46.586 DEBUG:teuthology.orchestra.run.smithi022:> if "*" not in asok_path: 2024-03-29T04:17:46.586 DEBUG:teuthology.orchestra.run.smithi022:> assert(len(files) == 1) 2024-03-29T04:17:46.586 DEBUG:teuthology.orchestra.run.smithi022:> return files[0] 2024-03-29T04:17:46.586 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-29T04:17:46.586 DEBUG:teuthology.orchestra.run.smithi022:> for f in files: 2024-03-29T04:17:46.586 DEBUG:teuthology.orchestra.run.smithi022:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T04:17:46.586 DEBUG:teuthology.orchestra.run.smithi022:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T04:17:46.586 DEBUG:teuthology.orchestra.run.smithi022:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T04:17:46.586 DEBUG:teuthology.orchestra.run.smithi022:> contents = proc_f.read() 2024-03-29T04:17:46.586 DEBUG:teuthology.orchestra.run.smithi022:> if mountpoint in contents: 2024-03-29T04:17:46.586 DEBUG:teuthology.orchestra.run.smithi022:> return f 2024-03-29T04:17:46.587 DEBUG:teuthology.orchestra.run.smithi022:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T04:17:46.587 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-29T04:17:46.587 DEBUG:teuthology.orchestra.run.smithi022:> print(_find_admin_socket("client.admin")) 2024-03-29T04:17:46.587 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-03-29T04:17:46.716 INFO:teuthology.orchestra.run.smithi022.stdout:/var/run/ceph/ceph-client.admin.33235.asok 2024-03-29T04:17:46.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:17:46.729 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.33235.asok 2024-03-29T04:17:46.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:17:46.729 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.33235.asok status 2024-03-29T04:17:46.889 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-29T04:17:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: "metadata": { 2024-03-29T04:17:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_sha1": "69e2cd73c52284dfd5096c66b60c6e6b26a3c447", 2024-03-29T04:17:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 19.0.0-2520-g69e2cd73 (69e2cd73c52284dfd5096c66b60c6e6b26a3c447) squid (dev)", 2024-03-29T04:17:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: "entity_id": "admin", 2024-03-29T04:17:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": "smithi022", 2024-03-29T04:17:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-29T04:17:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: "pid": "33235", 2024-03-29T04:17:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: "root": "/" 2024-03-29T04:17:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:17:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_count": 0, 2024-03-29T04:17:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_pinned_count": 0, 2024-03-29T04:17:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 4425, 2024-03-29T04:17:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: "inst": { 2024-03-29T04:17:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: "name": { 2024-03-29T04:17:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "client", 2024-03-29T04:17:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: "num": 4425 2024-03-29T04:17:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:17:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2024-03-29T04:17:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-03-29T04:17:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:17:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3271745786 2024-03-29T04:17:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:17:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:17:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2024-03-29T04:17:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-03-29T04:17:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:17:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3271745786 2024-03-29T04:17:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:17:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: "inst_str": "client.4425 192.168.0.1:0/3271745786", 2024-03-29T04:17:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: "addr_str": "192.168.0.1:0/3271745786", 2024-03-29T04:17:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: "inode_count": 1, 2024-03-29T04:17:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_epoch": 7, 2024-03-29T04:17:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch": 26, 2024-03-29T04:17:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch_barrier": 0, 2024-03-29T04:17:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklisted": false, 2024-03-29T04:17:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs" 2024-03-29T04:17:46.891 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-29T04:17:46.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:17:46.903 DEBUG:teuthology.orchestra.run.smithi022:> stat --file-system '--printf=%T 2024-03-29T04:17:46.903 DEBUG:teuthology.orchestra.run.smithi022:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:17:46.910 INFO:teuthology.orchestra.run.smithi022.stdout:fuseblk 2024-03-29T04:17:46.910 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:17:46.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:17:46.910 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:17:46.963 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-29T04:17:47.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:17:47.097 INFO:teuthology.orchestra.run.smithi022.stdout:# file: . 2024-03-29T04:17:47.097 INFO:teuthology.orchestra.run.smithi022.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-29T04:17:47.097 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:47.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:17:47.106 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:17:47.106 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:17:47.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:17:47.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.304+0000 7fbc42a0a640 1 Processor -- start 2024-03-29T04:17:47.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.304+0000 7fbc42a0a640 1 -- start start 2024-03-29T04:17:47.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.304+0000 7fbc42a0a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fbc3c06c670 0x7fbc3c0cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:47.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.304+0000 7fbc42a0a640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fbc3c0b41f0 0x7fbc3c06e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:47.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.304+0000 7fbc42a0a640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc3c06ca50 0x7fbc3c15fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:47.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.304+0000 7fbc42a0a640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fbc3c0b34c0 con 0x7fbc3c06c670 2024-03-29T04:17:47.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.304+0000 7fbc42a0a640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fbc3c06e5c0 con 0x7fbc3c0b41f0 2024-03-29T04:17:47.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.304+0000 7fbc42a0a640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fbc3c0ccef0 con 0x7fbc3c06ca50 2024-03-29T04:17:47.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.304+0000 7fbc3bfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fbc3c06c670 0x7fbc3c0cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:47.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.304+0000 7fbc3bfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fbc3c06c670 0x7fbc3c0cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49202/0 (socket says 172.21.15.22:49202) 2024-03-29T04:17:47.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.304+0000 7fbc3bfff640 1 -- 172.21.15.22:0/371955140 learned_addr learned my addr 172.21.15.22:0/371955140 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:47.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.304+0000 7fbc3b7fe640 1 --2- 172.21.15.22:0/371955140 >> v2:172.21.15.151:3300/0 conn(0x7fbc3c0b41f0 0x7fbc3c06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:47.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.304+0000 7fbc40fa7640 1 --2- 172.21.15.22:0/371955140 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc3c06ca50 0x7fbc3c15fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:47.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.304+0000 7fbc3bfff640 1 -- 172.21.15.22:0/371955140 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc3c06ca50 msgr2=0x7fbc3c15fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:47.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.304+0000 7fbc3bfff640 1 --2- 172.21.15.22:0/371955140 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc3c06ca50 0x7fbc3c15fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.304+0000 7fbc3bfff640 1 -- 172.21.15.22:0/371955140 >> v2:172.21.15.151:3300/0 conn(0x7fbc3c0b41f0 msgr2=0x7fbc3c06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:47.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.304+0000 7fbc3bfff640 1 --2- 172.21.15.22:0/371955140 >> v2:172.21.15.151:3300/0 conn(0x7fbc3c0b41f0 0x7fbc3c06e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.304+0000 7fbc3bfff640 1 -- 172.21.15.22:0/371955140 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3c0b3710 con 0x7fbc3c06c670 2024-03-29T04:17:47.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.308+0000 7fbc3bfff640 1 --2- 172.21.15.22:0/371955140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc3c06c670 0x7fbc3c0cc910 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c06ebe0 tx=0x7fbc2c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28595c58e416e96a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:47.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.308+0000 7fbc3affd640 1 -- 172.21.15.22:0/371955140 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c0b0070 con 0x7fbc3c06c670 2024-03-29T04:17:47.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.308+0000 7fbc3affd640 1 -- 172.21.15.22:0/371955140 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc2c0ab600 con 0x7fbc3c06c670 2024-03-29T04:17:47.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.308+0000 7fbc3affd640 1 -- 172.21.15.22:0/371955140 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c0ab9d0 con 0x7fbc3c06c670 2024-03-29T04:17:47.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.308+0000 7fbc42a0a640 1 -- 172.21.15.22:0/371955140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc3c06c670 msgr2=0x7fbc3c0cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:47.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.308+0000 7fbc42a0a640 1 --2- 172.21.15.22:0/371955140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc3c06c670 0x7fbc3c0cc910 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c06ebe0 tx=0x7fbc2c01c260 comp rx=0 tx=0).stop 2024-03-29T04:17:47.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.308+0000 7fbc42a0a640 1 -- 172.21.15.22:0/371955140 shutdown_connections 2024-03-29T04:17:47.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.308+0000 7fbc42a0a640 1 --2- 172.21.15.22:0/371955140 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc3c06ca50 0x7fbc3c15fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.308+0000 7fbc42a0a640 1 --2- 172.21.15.22:0/371955140 >> v2:172.21.15.151:3300/0 conn(0x7fbc3c0b41f0 0x7fbc3c06e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.308+0000 7fbc42a0a640 1 --2- 172.21.15.22:0/371955140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc3c06c670 0x7fbc3c0cc910 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.308+0000 7fbc42a0a640 1 -- 172.21.15.22:0/371955140 >> 172.21.15.22:0/371955140 conn(0x7fbc3c0cfee0 msgr2=0x7fbc3c0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:47.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.308+0000 7fbc42a0a640 1 -- 172.21.15.22:0/371955140 shutdown_connections 2024-03-29T04:17:47.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.308+0000 7fbc42a0a640 1 -- 172.21.15.22:0/371955140 wait complete. 2024-03-29T04:17:47.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.308+0000 7fbc42a0a640 1 Processor -- start 2024-03-29T04:17:47.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.308+0000 7fbc42a0a640 1 -- start start 2024-03-29T04:17:47.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc42a0a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc3c06c670 0x7fbc3c15bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:47.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc42a0a640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc3c06ca50 0x7fbc3c13f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:47.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc42a0a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc3c0b41f0 0x7fbc3c1f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:47.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc42a0a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c06cfe0 con 0x7fbc3c0b41f0 2024-03-29T04:17:47.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc42a0a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c06ce60 con 0x7fbc3c06c670 2024-03-29T04:17:47.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc42a0a640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fbc3c0ce5b0 con 0x7fbc3c06ca50 2024-03-29T04:17:47.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc40fa7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc3c0b41f0 0x7fbc3c1f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:47.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc40fa7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc3c0b41f0 0x7fbc3c1f7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49218/0 (socket says 172.21.15.22:49218) 2024-03-29T04:17:47.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc40fa7640 1 -- 172.21.15.22:0/1976362098 learned_addr learned my addr 172.21.15.22:0/1976362098 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:47.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc3bfff640 1 --2- 172.21.15.22:0/1976362098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc3c06c670 0x7fbc3c15bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:47.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc3b7fe640 1 --2- 172.21.15.22:0/1976362098 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc3c06ca50 0x7fbc3c13f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:47.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc40fa7640 1 -- 172.21.15.22:0/1976362098 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc3c06ca50 msgr2=0x7fbc3c13f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:47.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc40fa7640 1 --2- 172.21.15.22:0/1976362098 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc3c06ca50 0x7fbc3c13f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc40fa7640 1 -- 172.21.15.22:0/1976362098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc3c06c670 msgr2=0x7fbc3c15bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:47.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc40fa7640 1 --2- 172.21.15.22:0/1976362098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc3c06c670 0x7fbc3c15bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc40fa7640 1 -- 172.21.15.22:0/1976362098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3c0719a0 con 0x7fbc3c0b41f0 2024-03-29T04:17:47.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc40fa7640 1 --2- 172.21.15.22:0/1976362098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc3c0b41f0 0x7fbc3c1f7170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400eed0 tx=0x7fbc3400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:47.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc38ff9640 1 -- 172.21.15.22:0/1976362098 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34018020 con 0x7fbc3c0b41f0 2024-03-29T04:17:47.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc38ff9640 1 -- 172.21.15.22:0/1976362098 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc34014400 con 0x7fbc3c0b41f0 2024-03-29T04:17:47.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc38ff9640 1 -- 172.21.15.22:0/1976362098 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc340147f0 con 0x7fbc3c0b41f0 2024-03-29T04:17:47.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc42a0a640 1 -- 172.21.15.22:0/1976362098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc3c071c80 con 0x7fbc3c0b41f0 2024-03-29T04:17:47.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc42a0a640 1 -- 172.21.15.22:0/1976362098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc3c072240 con 0x7fbc3c0b41f0 2024-03-29T04:17:47.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc42a0a640 1 -- 172.21.15.22:0/1976362098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc3c068890 con 0x7fbc3c0b41f0 2024-03-29T04:17:47.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc38ff9640 1 -- 172.21.15.22:0/1976362098 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fbc3401b020 con 0x7fbc3c0b41f0 2024-03-29T04:17:47.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.312+0000 7fbc38ff9640 1 --2- 172.21.15.22:0/1976362098 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbc28067ff0 0x7fbc2806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:47.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.316+0000 7fbc38ff9640 1 -- 172.21.15.22:0/1976362098 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbc34086ee0 con 0x7fbc3c0b41f0 2024-03-29T04:17:47.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.316+0000 7fbc3bfff640 1 --2- 172.21.15.22:0/1976362098 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbc28067ff0 0x7fbc2806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:47.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.316+0000 7fbc3bfff640 1 --2- 172.21.15.22:0/1976362098 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbc28067ff0 0x7fbc2806a4b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c0ad000 tx=0x7fbc2c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:47.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.320+0000 7fbc38ff9640 1 -- 172.21.15.22:0/1976362098 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fbc34022030 con 0x7fbc3c0b41f0 2024-03-29T04:17:47.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.484+0000 7fbc42a0a640 1 -- 172.21.15.22:0/1976362098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc3c13c500 con 0x7fbc3c0b41f0 2024-03-29T04:17:47.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.484+0000 7fbc38ff9640 1 -- 172.21.15.22:0/1976362098 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18514 (secure 0 0 0) 0x7fbc34057350 con 0x7fbc3c0b41f0 2024-03-29T04:17:47.467 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:47.468 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:17:42.645632+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:17:47.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.488+0000 7fbc42a0a640 1 -- 172.21.15.22:0/1976362098 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbc28067ff0 msgr2=0x7fbc2806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:47.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.488+0000 7fbc42a0a640 1 --2- 172.21.15.22:0/1976362098 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbc28067ff0 0x7fbc2806a4b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c0ad000 tx=0x7fbc2c008040 comp rx=0 tx=0).stop 2024-03-29T04:17:47.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.488+0000 7fbc42a0a640 1 -- 172.21.15.22:0/1976362098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc3c0b41f0 msgr2=0x7fbc3c1f7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:47.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.488+0000 7fbc42a0a640 1 --2- 172.21.15.22:0/1976362098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc3c0b41f0 0x7fbc3c1f7170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400eed0 tx=0x7fbc3400b480 comp rx=0 tx=0).stop 2024-03-29T04:17:47.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.488+0000 7fbc42a0a640 1 -- 172.21.15.22:0/1976362098 shutdown_connections 2024-03-29T04:17:47.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.488+0000 7fbc42a0a640 1 --2- 172.21.15.22:0/1976362098 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbc28067ff0 0x7fbc2806a4b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.488+0000 7fbc42a0a640 1 --2- 172.21.15.22:0/1976362098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc3c0b41f0 0x7fbc3c1f7170 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.488+0000 7fbc42a0a640 1 --2- 172.21.15.22:0/1976362098 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc3c06ca50 0x7fbc3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.488+0000 7fbc42a0a640 1 --2- 172.21.15.22:0/1976362098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc3c06c670 0x7fbc3c15bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.488+0000 7fbc42a0a640 1 -- 172.21.15.22:0/1976362098 >> 172.21.15.22:0/1976362098 conn(0x7fbc3c0cfee0 msgr2=0x7fbc3c15db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:47.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.488+0000 7fbc42a0a640 1 -- 172.21.15.22:0/1976362098 shutdown_connections 2024-03-29T04:17:47.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.488+0000 7fbc42a0a640 1 -- 172.21.15.22:0/1976362098 wait complete. 2024-03-29T04:17:47.487 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:17:34.034142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:17:34.886659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:17:38.628632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:17:47.487 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:17:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.624+0000 7fafa9b53640 1 Processor -- start 2024-03-29T04:17:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.624+0000 7fafa9b53640 1 -- start start 2024-03-29T04:17:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.628+0000 7fafa9b53640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fafa406c790 0x7fafa406b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.628+0000 7fafa9b53640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fafa406f9e0 0x7fafa406fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.628+0000 7fafa9b53640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fafa4161fd0 0x7fafa4162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.628+0000 7fafa9b53640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fafa415f8b0 con 0x7fafa4161fd0 2024-03-29T04:17:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.628+0000 7fafa9b53640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fafa4070340 con 0x7fafa406f9e0 2024-03-29T04:17:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.628+0000 7fafa9b53640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fafa415e9e0 con 0x7fafa406c790 2024-03-29T04:17:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.628+0000 7fafa3fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fafa4161fd0 0x7fafa4162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.628+0000 7fafa3fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fafa4161fd0 0x7fafa4162440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49226/0 (socket says 172.21.15.22:49226) 2024-03-29T04:17:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.628+0000 7fafa3fff640 1 -- 172.21.15.22:0/2525202984 learned_addr learned my addr 172.21.15.22:0/2525202984 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.628+0000 7fafa37fe640 1 --2- 172.21.15.22:0/2525202984 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fafa406c790 0x7fafa406b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.628+0000 7faf9bfff640 1 --2- 172.21.15.22:0/2525202984 >> v2:172.21.15.151:3300/0 conn(0x7fafa406f9e0 0x7fafa406fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.628+0000 7fafa3fff640 1 -- 172.21.15.22:0/2525202984 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fafa406c790 msgr2=0x7fafa406b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.628+0000 7fafa3fff640 1 --2- 172.21.15.22:0/2525202984 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fafa406c790 0x7fafa406b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.628+0000 7fafa3fff640 1 -- 172.21.15.22:0/2525202984 >> v2:172.21.15.151:3300/0 conn(0x7fafa406f9e0 msgr2=0x7fafa406fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.628+0000 7fafa3fff640 1 --2- 172.21.15.22:0/2525202984 >> v2:172.21.15.151:3300/0 conn(0x7fafa406f9e0 0x7fafa406fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.628+0000 7fafa3fff640 1 -- 172.21.15.22:0/2525202984 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa415eca0 con 0x7fafa4161fd0 2024-03-29T04:17:47.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.628+0000 7faf9bfff640 1 --2- 172.21.15.22:0/2525202984 >> v2:172.21.15.151:3300/0 conn(0x7fafa406f9e0 0x7fafa406fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:17:47.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.628+0000 7fafa3fff640 1 --2- 172.21.15.22:0/2525202984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafa4161fd0 0x7fafa4162440 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fafa406f7a0 tx=0x7faf9c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4dde4cef43d6b43 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:47.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa2ffd640 1 -- 172.21.15.22:0/2525202984 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c0a5840 con 0x7fafa4161fd0 2024-03-29T04:17:47.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa2ffd640 1 -- 172.21.15.22:0/2525202984 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf9c0a59e0 con 0x7fafa4161fd0 2024-03-29T04:17:47.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa2ffd640 1 -- 172.21.15.22:0/2525202984 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c0a5db0 con 0x7fafa4161fd0 2024-03-29T04:17:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa9b53640 1 -- 172.21.15.22:0/2525202984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafa4161fd0 msgr2=0x7fafa4162440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa9b53640 1 --2- 172.21.15.22:0/2525202984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafa4161fd0 0x7fafa4162440 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fafa406f7a0 tx=0x7faf9c01c9c0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa9b53640 1 -- 172.21.15.22:0/2525202984 shutdown_connections 2024-03-29T04:17:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa9b53640 1 --2- 172.21.15.22:0/2525202984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafa4161fd0 0x7fafa4162440 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa9b53640 1 --2- 172.21.15.22:0/2525202984 >> v2:172.21.15.151:3300/0 conn(0x7fafa406f9e0 0x7fafa406fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa9b53640 1 --2- 172.21.15.22:0/2525202984 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fafa406c790 0x7fafa406b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa9b53640 1 -- 172.21.15.22:0/2525202984 >> 172.21.15.22:0/2525202984 conn(0x7fafa40d1070 msgr2=0x7fafa40d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa9b53640 1 -- 172.21.15.22:0/2525202984 shutdown_connections 2024-03-29T04:17:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa9b53640 1 -- 172.21.15.22:0/2525202984 wait complete. 2024-03-29T04:17:47.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa9b53640 1 Processor -- start 2024-03-29T04:17:47.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa9b53640 1 -- start start 2024-03-29T04:17:47.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa9b53640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafa406c790 0x7fafa4149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:47.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa9b53640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fafa406f9e0 0x7fafa414d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:47.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa9b53640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafa4161fd0 0x7fafa4153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:47.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa9b53640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fafa4070b20 con 0x7fafa406c790 2024-03-29T04:17:47.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa9b53640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fafa4162d90 con 0x7fafa4161fd0 2024-03-29T04:17:47.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa9b53640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fafa406cb70 con 0x7fafa406f9e0 2024-03-29T04:17:47.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafa406c790 0x7fafa4149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:47.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafa406c790 0x7fafa4149f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49232/0 (socket says 172.21.15.22:49232) 2024-03-29T04:17:47.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa37fe640 1 -- 172.21.15.22:0/571049131 learned_addr learned my addr 172.21.15.22:0/571049131 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:47.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa3fff640 1 --2- 172.21.15.22:0/571049131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafa4161fd0 0x7fafa4153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:47.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7faf9bfff640 1 --2- 172.21.15.22:0/571049131 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fafa406f9e0 0x7fafa414d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:47.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa37fe640 1 -- 172.21.15.22:0/571049131 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fafa406f9e0 msgr2=0x7fafa414d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:47.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa37fe640 1 --2- 172.21.15.22:0/571049131 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fafa406f9e0 0x7fafa414d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa37fe640 1 -- 172.21.15.22:0/571049131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafa4161fd0 msgr2=0x7fafa4153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:47.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa37fe640 1 --2- 172.21.15.22:0/571049131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafa4161fd0 0x7fafa4153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa37fe640 1 -- 172.21.15.22:0/571049131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa40adba0 con 0x7fafa406c790 2024-03-29T04:17:47.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7faf9bfff640 1 --2- 172.21.15.22:0/571049131 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fafa406f9e0 0x7fafa414d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:47.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.632+0000 7fafa37fe640 1 --2- 172.21.15.22:0/571049131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafa406c790 0x7fafa4149f20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faf9400e870 tx=0x7faf9400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:47.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.636+0000 7fafa0ff9640 1 -- 172.21.15.22:0/571049131 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf94017070 con 0x7fafa406c790 2024-03-29T04:17:47.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.636+0000 7fafa0ff9640 1 -- 172.21.15.22:0/571049131 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf9400ee60 con 0x7fafa406c790 2024-03-29T04:17:47.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.636+0000 7fafa9b53640 1 -- 172.21.15.22:0/571049131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa414df80 con 0x7fafa406c790 2024-03-29T04:17:47.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.636+0000 7fafa0ff9640 1 -- 172.21.15.22:0/571049131 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9400cb20 con 0x7fafa406c790 2024-03-29T04:17:47.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.636+0000 7fafa9b53640 1 -- 172.21.15.22:0/571049131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa414e820 con 0x7fafa406c790 2024-03-29T04:17:47.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.636+0000 7fafa9b53640 1 -- 172.21.15.22:0/571049131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa4068900 con 0x7fafa406c790 2024-03-29T04:17:47.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.636+0000 7fafa0ff9640 1 -- 172.21.15.22:0/571049131 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7faf9401a070 con 0x7fafa406c790 2024-03-29T04:17:47.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.636+0000 7fafa0ff9640 1 --2- 172.21.15.22:0/571049131 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faf84067ff0 0x7faf8406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:47.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.636+0000 7fafa0ff9640 1 -- 172.21.15.22:0/571049131 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7faf94086c90 con 0x7fafa406c790 2024-03-29T04:17:47.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.636+0000 7faf9bfff640 1 --2- 172.21.15.22:0/571049131 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faf84067ff0 0x7faf8406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:47.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.636+0000 7faf9bfff640 1 --2- 172.21.15.22:0/571049131 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faf84067ff0 0x7faf8406a4b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fafa4160160 tx=0x7faf8c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:47.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.640+0000 7fafa0ff9640 1 -- 172.21.15.22:0/571049131 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fafa40adba0 con 0x7fafa406c790 2024-03-29T04:17:47.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.800+0000 7fafa9b53640 1 -- 172.21.15.22:0/571049131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fafa40706c0 con 0x7fafa406c790 2024-03-29T04:17:47.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.800+0000 7fafa0ff9640 1 -- 172.21.15.22:0/571049131 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7faf94057270 con 0x7fafa406c790 2024-03-29T04:17:47.783 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:17:47.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.804+0000 7fafa9b53640 1 -- 172.21.15.22:0/571049131 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faf84067ff0 msgr2=0x7faf8406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:47.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.804+0000 7fafa9b53640 1 --2- 172.21.15.22:0/571049131 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faf84067ff0 0x7faf8406a4b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fafa4160160 tx=0x7faf8c008040 comp rx=0 tx=0).stop 2024-03-29T04:17:47.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.804+0000 7fafa9b53640 1 -- 172.21.15.22:0/571049131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafa406c790 msgr2=0x7fafa4149f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:47.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.804+0000 7fafa9b53640 1 --2- 172.21.15.22:0/571049131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafa406c790 0x7fafa4149f20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faf9400e870 tx=0x7faf9400ed30 comp rx=0 tx=0).stop 2024-03-29T04:17:47.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.804+0000 7fafa9b53640 1 -- 172.21.15.22:0/571049131 shutdown_connections 2024-03-29T04:17:47.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.804+0000 7fafa9b53640 1 --2- 172.21.15.22:0/571049131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafa4161fd0 0x7fafa4153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.804+0000 7fafa9b53640 1 --2- 172.21.15.22:0/571049131 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fafa406f9e0 0x7fafa414d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.804+0000 7fafa9b53640 1 --2- 172.21.15.22:0/571049131 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faf84067ff0 0x7faf8406a4b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.804+0000 7fafa9b53640 1 --2- 172.21.15.22:0/571049131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafa406c790 0x7fafa4149f20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.804+0000 7fafa9b53640 1 -- 172.21.15.22:0/571049131 >> 172.21.15.22:0/571049131 conn(0x7fafa40d1070 msgr2=0x7fafa414f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:47.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.804+0000 7fafa9b53640 1 -- 172.21.15.22:0/571049131 shutdown_connections 2024-03-29T04:17:47.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.804+0000 7fafa9b53640 1 -- 172.21.15.22:0/571049131 wait complete. 2024-03-29T04:17:47.802 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:17:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.940+0000 7f0d0bb57640 1 Processor -- start 2024-03-29T04:17:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.940+0000 7f0d0bb57640 1 -- start start 2024-03-29T04:17:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.940+0000 7f0d0bb57640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0d0406cd80 0x7f0d04166fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.940+0000 7f0d0bb57640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d0406d710 con 0x7f0d0413f140 2024-03-29T04:17:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.940+0000 7f0d0bb57640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d0406d9f0 con 0x7f0d0415ff40 2024-03-29T04:17:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.940+0000 7f0d0bb57640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f0d0406e320 con 0x7f0d0406cd80 2024-03-29T04:17:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.940+0000 7f0d098f3640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f0d0413f140 0x7f0d041606d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51036/0 (socket says 172.21.15.22:51036) 2024-03-29T04:17:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.940+0000 7f0d098f3640 1 -- 172.21.15.22:0/1732814825 learned_addr learned my addr 172.21.15.22:0/1732814825 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.940+0000 7f0d0a0f4640 1 --2- 172.21.15.22:0/1732814825 >> v2:172.21.15.22:3300/0 conn(0x7f0d0406cd80 0x7f0d04166fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:47.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.940+0000 7f0d088f1640 1 -- 172.21.15.22:0/1732814825 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 76643366 0 0) 0x7f0d0406d710 con 0x7f0d0413f140 2024-03-29T04:17:47.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.940+0000 7f0d088f1640 1 -- 172.21.15.22:0/1732814825 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cf0003530 con 0x7f0d0413f140 2024-03-29T04:17:47.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.940+0000 7f0d0a0f4640 1 -- 172.21.15.22:0/1732814825 >> v1:172.21.15.151:6789/0 conn(0x7f0d0415ff40 legacy=0x7f0d0406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:47.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.940+0000 7f0d0a0f4640 1 -- 172.21.15.22:0/1732814825 >> v1:172.21.15.22:6789/0 conn(0x7f0d0413f140 legacy=0x7f0d041606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:47.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.940+0000 7f0d0a0f4640 1 -- 172.21.15.22:0/1732814825 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0406dcd0 con 0x7f0d0406cd80 2024-03-29T04:17:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d0a0f4640 1 --2- 172.21.15.22:0/1732814825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d0406cd80 0x7f0d04166fe0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009d90 tx=0x7f0cfc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f147c2f17af03b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d088f1640 1 -- 172.21.15.22:0/1732814825 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0b2070 con 0x7f0d0406cd80 2024-03-29T04:17:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d088f1640 1 -- 172.21.15.22:0/1732814825 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cfc0a5940 con 0x7f0d0406cd80 2024-03-29T04:17:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d088f1640 1 -- 172.21.15.22:0/1732814825 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0a5d10 con 0x7f0d0406cd80 2024-03-29T04:17:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d0bb57640 1 -- 172.21.15.22:0/1732814825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d0406cd80 msgr2=0x7f0d04166fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d0bb57640 1 --2- 172.21.15.22:0/1732814825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d0406cd80 0x7f0d04166fe0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009d90 tx=0x7f0cfc01c970 comp rx=0 tx=0).stop 2024-03-29T04:17:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d0bb57640 1 -- 172.21.15.22:0/1732814825 shutdown_connections 2024-03-29T04:17:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d0bb57640 1 --2- 172.21.15.22:0/1732814825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d0406cd80 0x7f0d04166fe0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d0bb57640 1 -- 172.21.15.22:0/1732814825 >> 172.21.15.22:0/1732814825 conn(0x7f0d040d00a0 msgr2=0x7f0d040d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d0bb57640 1 -- 172.21.15.22:0/1732814825 shutdown_connections 2024-03-29T04:17:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d0bb57640 1 -- 172.21.15.22:0/1732814825 wait complete. 2024-03-29T04:17:47.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d0bb57640 1 Processor -- start 2024-03-29T04:17:47.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d0bb57640 1 -- start start 2024-03-29T04:17:47.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d0bb57640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d0406cd80 0x7f0d04139b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:47.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d0bb57640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d0413f140 0x7f0d04135e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:47.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d0bb57640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0d0415ff40 0x7f0d04071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:47.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d0bb57640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0416a0d0 con 0x7f0d0406cd80 2024-03-29T04:17:47.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d0bb57640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0d04169f50 con 0x7f0d0413f140 2024-03-29T04:17:47.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d0bb57640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0d0416a250 con 0x7f0d0415ff40 2024-03-29T04:17:47.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d098f3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d0406cd80 0x7f0d04139b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:47.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d098f3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d0406cd80 0x7f0d04139b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49250/0 (socket says 172.21.15.22:49250) 2024-03-29T04:17:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d098f3640 1 -- 172.21.15.22:0/1368472094 learned_addr learned my addr 172.21.15.22:0/1368472094 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d090f2640 1 --2- 172.21.15.22:0/1368472094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d0413f140 0x7f0d04135e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d0a0f4640 1 --2- 172.21.15.22:0/1368472094 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0d0415ff40 0x7f0d04071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d098f3640 1 -- 172.21.15.22:0/1368472094 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0d0415ff40 msgr2=0x7f0d04071b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d098f3640 1 --2- 172.21.15.22:0/1368472094 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0d0415ff40 0x7f0d04071b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d098f3640 1 -- 172.21.15.22:0/1368472094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d0413f140 msgr2=0x7f0d04135e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d098f3640 1 --2- 172.21.15.22:0/1368472094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d0413f140 0x7f0d04135e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d098f3640 1 -- 172.21.15.22:0/1368472094 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0413a200 con 0x7f0d0406cd80 2024-03-29T04:17:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d0a0f4640 1 --2- 172.21.15.22:0/1368472094 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0d0415ff40 0x7f0d04071b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.944+0000 7f0d098f3640 1 --2- 172.21.15.22:0/1368472094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d0406cd80 0x7f0d04139b90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000c4d0 tx=0x7f0d0000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.948+0000 7f0cfa7fc640 1 -- 172.21.15.22:0/1368472094 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00017070 con 0x7f0d0406cd80 2024-03-29T04:17:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.948+0000 7f0cfa7fc640 1 -- 172.21.15.22:0/1368472094 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d00005cf0 con 0x7f0d0406cd80 2024-03-29T04:17:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.948+0000 7f0d0bb57640 1 -- 172.21.15.22:0/1368472094 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0413a4e0 con 0x7f0d0406cd80 2024-03-29T04:17:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.948+0000 7f0cfa7fc640 1 -- 172.21.15.22:0/1368472094 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00007270 con 0x7f0d0406cd80 2024-03-29T04:17:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.948+0000 7f0d0bb57640 1 -- 172.21.15.22:0/1368472094 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d0413aaa0 con 0x7f0d0406cd80 2024-03-29T04:17:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.948+0000 7f0d0bb57640 1 -- 172.21.15.22:0/1368472094 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d04068900 con 0x7f0d0406cd80 2024-03-29T04:17:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.948+0000 7f0cfa7fc640 1 -- 172.21.15.22:0/1368472094 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f0d0001a070 con 0x7f0d0406cd80 2024-03-29T04:17:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.948+0000 7f0cfa7fc640 1 --2- 172.21.15.22:0/1368472094 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0ce8067fc0 0x7f0ce806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:47.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.948+0000 7f0cfa7fc640 1 -- 172.21.15.22:0/1368472094 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0d00085b20 con 0x7f0d0406cd80 2024-03-29T04:17:47.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.948+0000 7f0d090f2640 1 --2- 172.21.15.22:0/1368472094 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0ce8067fc0 0x7f0ce806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:47.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.948+0000 7f0d090f2640 1 --2- 172.21.15.22:0/1368472094 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0ce8067fc0 0x7f0ce806a480 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d0412f0b0 tx=0x7f0cf4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:47.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:47.952+0000 7f0cfa7fc640 1 -- 172.21.15.22:0/1368472094 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f0d00055940 con 0x7f0d0406cd80 2024-03-29T04:17:48.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.112+0000 7f0d0bb57640 1 -- 172.21.15.22:0/1368472094 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0d0415dc60 con 0x7f0d0406cd80 2024-03-29T04:17:48.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.112+0000 7f0cfa7fc640 1 -- 172.21.15.22:0/1368472094 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f0d00059c40 con 0x7f0d0406cd80 2024-03-29T04:17:48.094 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:17:48.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.116+0000 7f0d0bb57640 1 -- 172.21.15.22:0/1368472094 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0ce8067fc0 msgr2=0x7f0ce806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:48.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.116+0000 7f0d0bb57640 1 --2- 172.21.15.22:0/1368472094 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0ce8067fc0 0x7f0ce806a480 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d0412f0b0 tx=0x7f0cf4009040 comp rx=0 tx=0).stop 2024-03-29T04:17:48.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.116+0000 7f0d0bb57640 1 -- 172.21.15.22:0/1368472094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d0406cd80 msgr2=0x7f0d04139b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:48.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.116+0000 7f0d0bb57640 1 --2- 172.21.15.22:0/1368472094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d0406cd80 0x7f0d04139b90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000c4d0 tx=0x7f0d0000c990 comp rx=0 tx=0).stop 2024-03-29T04:17:48.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.116+0000 7f0d0bb57640 1 -- 172.21.15.22:0/1368472094 shutdown_connections 2024-03-29T04:17:48.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.116+0000 7f0d0bb57640 1 --2- 172.21.15.22:0/1368472094 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0d0415ff40 0x7f0d04071b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.116+0000 7f0d0bb57640 1 --2- 172.21.15.22:0/1368472094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d0413f140 0x7f0d04135e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.116+0000 7f0d0bb57640 1 --2- 172.21.15.22:0/1368472094 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0ce8067fc0 0x7f0ce806a480 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.116+0000 7f0d0bb57640 1 --2- 172.21.15.22:0/1368472094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d0406cd80 0x7f0d04139b90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.116+0000 7f0d0bb57640 1 -- 172.21.15.22:0/1368472094 >> 172.21.15.22:0/1368472094 conn(0x7f0d040d00a0 msgr2=0x7f0d040b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:48.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.116+0000 7f0d0bb57640 1 -- 172.21.15.22:0/1368472094 shutdown_connections 2024-03-29T04:17:48.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.116+0000 7f0d0bb57640 1 -- 172.21.15.22:0/1368472094 wait complete. 2024-03-29T04:17:48.111 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:17:48.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06b0cc1640 1 Processor -- start 2024-03-29T04:17:48.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06b0cc1640 1 -- start start 2024-03-29T04:17:48.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06b0cc1640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f06ac15ff40 0x7f06ac06c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:48.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06b0cc1640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06ac06d710 con 0x7f06ac13f140 2024-03-29T04:17:48.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06b0cc1640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06ac06d9f0 con 0x7f06ac06cbb0 2024-03-29T04:17:48.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06b0cc1640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f06ac06d000 con 0x7f06ac15ff40 2024-03-29T04:17:48.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06aa59c640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f06ac13f140 0x7f06ac1606d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51050/0 (socket says 172.21.15.22:51050) 2024-03-29T04:17:48.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06aa59c640 1 -- 172.21.15.22:0/2264464879 learned_addr learned my addr 172.21.15.22:0/2264464879 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:48.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a9d9b640 1 --2- 172.21.15.22:0/2264464879 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f06ac15ff40 0x7f06ac06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:48.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a959a640 1 -- 172.21.15.22:0/2264464879 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3766248315 0 0) 0x7f06ac06d710 con 0x7f06ac13f140 2024-03-29T04:17:48.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a959a640 1 -- 172.21.15.22:0/2264464879 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0694003530 con 0x7f06ac13f140 2024-03-29T04:17:48.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a959a640 1 -- 172.21.15.22:0/2264464879 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3173153132 0 0) 0x7f0694003530 con 0x7f06ac13f140 2024-03-29T04:17:48.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a959a640 1 -- 172.21.15.22:0/2264464879 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f06ac15ff40 msgr2=0x7f06ac06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:48.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a959a640 1 --2- 172.21.15.22:0/2264464879 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f06ac15ff40 0x7f06ac06c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a959a640 1 -- 172.21.15.22:0/2264464879 >> v1:172.21.15.151:6789/0 conn(0x7f06ac06cbb0 legacy=0x7f06ac0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:48.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a959a640 1 -- 172.21.15.22:0/2264464879 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ac06dcd0 con 0x7f06ac13f140 2024-03-29T04:17:48.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a959a640 1 -- 172.21.15.22:0/2264464879 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f069c001680 con 0x7f06ac13f140 2024-03-29T04:17:48.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a9d9b640 1 --2- 172.21.15.22:0/2264464879 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f06ac15ff40 0x7f06ac06c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:17:48.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a959a640 1 -- 172.21.15.22:0/2264464879 >> v1:172.21.15.22:6789/0 conn(0x7f06ac13f140 legacy=0x7f06ac1606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:48.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a959a640 1 --2- 172.21.15.22:0/2264464879 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f06ac15ff40 0x7f06940039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:48.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a959a640 1 --2- 172.21.15.22:0/2264464879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0694003fe0 0x7f0694004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:48.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a959a640 1 --2- 172.21.15.22:0/2264464879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0694004d50 0x7f06940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:48.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a959a640 1 -- 172.21.15.22:0/2264464879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac06d000 con 0x7f0694003fe0 2024-03-29T04:17:48.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a959a640 1 -- 172.21.15.22:0/2264464879 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f069400bb30 con 0x7f0694004d50 2024-03-29T04:17:48.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a959a640 1 -- 172.21.15.22:0/2264464879 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f069400bd30 con 0x7f06ac15ff40 2024-03-29T04:17:48.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a9d9b640 1 --2- 172.21.15.22:0/2264464879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0694003fe0 0x7f0694004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:48.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06aad9d640 1 --2- 172.21.15.22:0/2264464879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0694004d50 0x7f06940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:48.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06aa59c640 1 --2- 172.21.15.22:0/2264464879 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f06ac15ff40 0x7f06940039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:48.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a9d9b640 1 -- 172.21.15.22:0/2264464879 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f06ac15ff40 msgr2=0x7f06940039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:48.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a9d9b640 1 --2- 172.21.15.22:0/2264464879 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f06ac15ff40 0x7f06940039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a9d9b640 1 -- 172.21.15.22:0/2264464879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0694004d50 msgr2=0x7f06940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:48.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a9d9b640 1 --2- 172.21.15.22:0/2264464879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0694004d50 0x7f06940051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06a9d9b640 1 -- 172.21.15.22:0/2264464879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0694003530 con 0x7f0694003fe0 2024-03-29T04:17:48.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.256+0000 7f06aa59c640 1 --2- 172.21.15.22:0/2264464879 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f06ac15ff40 0x7f06940039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:48.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06a9d9b640 1 --2- 172.21.15.22:0/2264464879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0694003fe0 0x7f0694004810 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f06a00098d0 tx=0x7f06a001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b270beb2462f3b53 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:48.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06a959a640 1 -- 172.21.15.22:0/2264464879 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a000f040 con 0x7f0694003fe0 2024-03-29T04:17:48.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06a959a640 1 -- 172.21.15.22:0/2264464879 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06a00a9620 con 0x7f0694003fe0 2024-03-29T04:17:48.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06a959a640 1 -- 172.21.15.22:0/2264464879 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c001d50 con 0x7f0694003fe0 2024-03-29T04:17:48.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06b0cc1640 1 -- 172.21.15.22:0/2264464879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0694003fe0 msgr2=0x7f0694004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:48.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06b0cc1640 1 --2- 172.21.15.22:0/2264464879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0694003fe0 0x7f0694004810 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f06a00098d0 tx=0x7f06a001c510 comp rx=0 tx=0).stop 2024-03-29T04:17:48.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06aad9d640 1 -- 172.21.15.22:0/2264464879 reap_dead start 2024-03-29T04:17:48.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06b0cc1640 1 -- 172.21.15.22:0/2264464879 shutdown_connections 2024-03-29T04:17:48.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06b0cc1640 1 -- 172.21.15.22:0/2264464879 >> 172.21.15.22:0/2264464879 conn(0x7f06ac0d00a0 msgr2=0x7f06ac0d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:48.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06b0cc1640 1 -- 172.21.15.22:0/2264464879 shutdown_connections 2024-03-29T04:17:48.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06b0cc1640 1 -- 172.21.15.22:0/2264464879 wait complete. 2024-03-29T04:17:48.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06b0cc1640 1 Processor -- start 2024-03-29T04:17:48.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06b0cc1640 1 -- start start 2024-03-29T04:17:48.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06b0cc1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0694003fe0 0x7f06ac145520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:48.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06b0cc1640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f06ac141d10 0x7f06ac140420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:48.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06b0cc1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06ac13a6f0 0x7f06ac0af820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:48.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06b0cc1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac160de0 con 0x7f06ac13a6f0 2024-03-29T04:17:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06b0cc1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac06e320 con 0x7f0694003fe0 2024-03-29T04:17:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06b0cc1640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f06ac13f520 con 0x7f06ac141d10 2024-03-29T04:17:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06aad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06ac13a6f0 0x7f06ac0af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06aad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06ac13a6f0 0x7f06ac0af820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49276/0 (socket says 172.21.15.22:49276) 2024-03-29T04:17:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06aad9d640 1 -- 172.21.15.22:0/444302948 learned_addr learned my addr 172.21.15.22:0/444302948 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06aa59c640 1 --2- 172.21.15.22:0/444302948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0694003fe0 0x7f06ac145520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06a9d9b640 1 --2- 172.21.15.22:0/444302948 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f06ac141d10 0x7f06ac140420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06aad9d640 1 -- 172.21.15.22:0/444302948 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f06ac141d10 msgr2=0x7f06ac140420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06aad9d640 1 --2- 172.21.15.22:0/444302948 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f06ac141d10 0x7f06ac140420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06aad9d640 1 -- 172.21.15.22:0/444302948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0694003fe0 msgr2=0x7f06ac145520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06aad9d640 1 --2- 172.21.15.22:0/444302948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0694003fe0 0x7f06ac145520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.260+0000 7f06aad9d640 1 -- 172.21.15.22:0/444302948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ac140c70 con 0x7f06ac13a6f0 2024-03-29T04:17:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.264+0000 7f06aad9d640 1 --2- 172.21.15.22:0/444302948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06ac13a6f0 0x7f06ac0af820 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f06ac06d230 tx=0x7f06a400e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.264+0000 7f069affd640 1 -- 172.21.15.22:0/444302948 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a4003b90 con 0x7f06ac13a6f0 2024-03-29T04:17:48.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.264+0000 7f069affd640 1 -- 172.21.15.22:0/444302948 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06a4018070 con 0x7f06ac13a6f0 2024-03-29T04:17:48.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.264+0000 7f06b0cc1640 1 -- 172.21.15.22:0/444302948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06ac140f50 con 0x7f06ac13a6f0 2024-03-29T04:17:48.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.264+0000 7f069affd640 1 -- 172.21.15.22:0/444302948 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a40050d0 con 0x7f06ac13a6f0 2024-03-29T04:17:48.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.264+0000 7f06b0cc1640 1 -- 172.21.15.22:0/444302948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06ac141510 con 0x7f06ac13a6f0 2024-03-29T04:17:48.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.264+0000 7f06b0cc1640 1 -- 172.21.15.22:0/444302948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06ac068900 con 0x7f06ac13a6f0 2024-03-29T04:17:48.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.264+0000 7f069affd640 1 -- 172.21.15.22:0/444302948 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f06a4005270 con 0x7f06ac13a6f0 2024-03-29T04:17:48.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.264+0000 7f069affd640 1 --2- 172.21.15.22:0/444302948 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f068c067ff0 0x7f068c06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:48.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.264+0000 7f069affd640 1 -- 172.21.15.22:0/444302948 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f06a4086440 con 0x7f06ac13a6f0 2024-03-29T04:17:48.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.264+0000 7f06aa59c640 1 --2- 172.21.15.22:0/444302948 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f068c067ff0 0x7f068c06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:48.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.264+0000 7f06aa59c640 1 --2- 172.21.15.22:0/444302948 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f068c067ff0 0x7f068c06a4b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f06ac153900 tx=0x7f069c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:48.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.268+0000 7f069affd640 1 -- 172.21.15.22:0/444302948 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f06a4056810 con 0x7f06ac13a6f0 2024-03-29T04:17:48.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.432+0000 7f06b0cc1640 1 -- 172.21.15.22:0/444302948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f06ac140960 con 0x7f06ac13a6f0 2024-03-29T04:17:48.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.432+0000 7f069affd640 1 -- 172.21.15.22:0/444302948 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f06a4056110 con 0x7f06ac13a6f0 2024-03-29T04:17:48.414 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-29T04:17:48.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.436+0000 7f06b0cc1640 1 -- 172.21.15.22:0/444302948 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f068c067ff0 msgr2=0x7f068c06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:48.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.436+0000 7f06b0cc1640 1 --2- 172.21.15.22:0/444302948 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f068c067ff0 0x7f068c06a4b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f06ac153900 tx=0x7f069c009040 comp rx=0 tx=0).stop 2024-03-29T04:17:48.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.436+0000 7f06b0cc1640 1 -- 172.21.15.22:0/444302948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06ac13a6f0 msgr2=0x7f06ac0af820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:48.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.436+0000 7f06b0cc1640 1 --2- 172.21.15.22:0/444302948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06ac13a6f0 0x7f06ac0af820 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f06ac06d230 tx=0x7f06a400e9e0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.436+0000 7f06b0cc1640 1 -- 172.21.15.22:0/444302948 shutdown_connections 2024-03-29T04:17:48.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.436+0000 7f06b0cc1640 1 --2- 172.21.15.22:0/444302948 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f068c067ff0 0x7f068c06a4b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.436+0000 7f06b0cc1640 1 --2- 172.21.15.22:0/444302948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06ac13a6f0 0x7f06ac0af820 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.436+0000 7f06b0cc1640 1 --2- 172.21.15.22:0/444302948 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f06ac141d10 0x7f06ac140420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.436+0000 7f06b0cc1640 1 --2- 172.21.15.22:0/444302948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0694003fe0 0x7f06ac145520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.436+0000 7f06b0cc1640 1 -- 172.21.15.22:0/444302948 >> 172.21.15.22:0/444302948 conn(0x7f06ac0d00a0 msgr2=0x7f06ac15c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:48.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.436+0000 7f06b0cc1640 1 -- 172.21.15.22:0/444302948 shutdown_connections 2024-03-29T04:17:48.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.436+0000 7f06b0cc1640 1 -- 172.21.15.22:0/444302948 wait complete. 2024-03-29T04:17:48.434 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:17:48.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.580+0000 7fb351454640 1 Processor -- start 2024-03-29T04:17:48.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.580+0000 7fb351454640 1 -- start start 2024-03-29T04:17:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.584+0000 7fb351454640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb34c06dee0 0x7fb34c15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.584+0000 7fb351454640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb34c06f5d0 con 0x7fb34c06db00 2024-03-29T04:17:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.584+0000 7fb351454640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb34c06f8b0 con 0x7fb34c06f060 2024-03-29T04:17:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.584+0000 7fb351454640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fb34c06fb90 con 0x7fb34c06dee0 2024-03-29T04:17:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.584+0000 7fb34a7fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb34c06dee0 0x7fb34c15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.584+0000 7fb34affd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb34c06db00 0x7fb34c0cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51062/0 (socket says 172.21.15.22:51062) 2024-03-29T04:17:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.584+0000 7fb34a7fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb34c06dee0 0x7fb34c15ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49288/0 (socket says 172.21.15.22:49288) 2024-03-29T04:17:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.584+0000 7fb34affd640 1 -- 172.21.15.22:0/4198390079 learned_addr learned my addr 172.21.15.22:0/4198390079 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:48.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.584+0000 7fb34a7fc640 1 -- 172.21.15.22:0/4198390079 >> v1:172.21.15.151:6789/0 conn(0x7fb34c06f060 legacy=0x7fb34c0cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:48.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.584+0000 7fb34a7fc640 1 -- 172.21.15.22:0/4198390079 >> v1:172.21.15.22:6789/0 conn(0x7fb34c06db00 legacy=0x7fb34c0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:48.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.584+0000 7fb34a7fc640 1 -- 172.21.15.22:0/4198390079 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34c06fe70 con 0x7fb34c06dee0 2024-03-29T04:17:48.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.588+0000 7fb34a7fc640 1 --2- 172.21.15.22:0/4198390079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb34c06dee0 0x7fb34c15ecc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb3400097c0 tx=0x7fb34001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6c9044053540ef server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:48.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.588+0000 7fb349ffb640 1 -- 172.21.15.22:0/4198390079 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3400b0070 con 0x7fb34c06dee0 2024-03-29T04:17:48.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.588+0000 7fb349ffb640 1 -- 172.21.15.22:0/4198390079 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3400ab600 con 0x7fb34c06dee0 2024-03-29T04:17:48.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.588+0000 7fb349ffb640 1 -- 172.21.15.22:0/4198390079 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3400ab9d0 con 0x7fb34c06dee0 2024-03-29T04:17:48.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.588+0000 7fb351454640 1 -- 172.21.15.22:0/4198390079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb34c06dee0 msgr2=0x7fb34c15ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:48.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.588+0000 7fb351454640 1 --2- 172.21.15.22:0/4198390079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb34c06dee0 0x7fb34c15ecc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb3400097c0 tx=0x7fb34001c4a0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.588+0000 7fb351454640 1 -- 172.21.15.22:0/4198390079 shutdown_connections 2024-03-29T04:17:48.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.588+0000 7fb351454640 1 --2- 172.21.15.22:0/4198390079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb34c06dee0 0x7fb34c15ecc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.588+0000 7fb351454640 1 -- 172.21.15.22:0/4198390079 >> 172.21.15.22:0/4198390079 conn(0x7fb34c0b2c70 msgr2=0x7fb34c0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:48.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.588+0000 7fb351454640 1 -- 172.21.15.22:0/4198390079 shutdown_connections 2024-03-29T04:17:48.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.588+0000 7fb351454640 1 -- 172.21.15.22:0/4198390079 wait complete. 2024-03-29T04:17:48.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.588+0000 7fb351454640 1 Processor -- start 2024-03-29T04:17:48.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.588+0000 7fb351454640 1 -- start start 2024-03-29T04:17:48.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.588+0000 7fb351454640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb34c06db00 0x7fb34c14dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:48.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.588+0000 7fb351454640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb34c06dee0 0x7fb34c06d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.588+0000 7fb351454640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb34c06f060 0x7fb34c06b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.588+0000 7fb351454640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c169070 con 0x7fb34c06dee0 2024-03-29T04:17:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.588+0000 7fb351454640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c0cd4a0 con 0x7fb34c06db00 2024-03-29T04:17:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.588+0000 7fb351454640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb34c1691f0 con 0x7fb34c06f060 2024-03-29T04:17:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.592+0000 7fb34a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb34c06dee0 0x7fb34c06d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.592+0000 7fb34a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb34c06dee0 0x7fb34c06d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49300/0 (socket says 172.21.15.22:49300) 2024-03-29T04:17:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.592+0000 7fb34a7fc640 1 -- 172.21.15.22:0/2215274261 learned_addr learned my addr 172.21.15.22:0/2215274261 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.592+0000 7fb34affd640 1 --2- 172.21.15.22:0/2215274261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb34c06db00 0x7fb34c14dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.592+0000 7fb34b7fe640 1 --2- 172.21.15.22:0/2215274261 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb34c06f060 0x7fb34c06b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.592+0000 7fb34a7fc640 1 -- 172.21.15.22:0/2215274261 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb34c06f060 msgr2=0x7fb34c06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:48.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.592+0000 7fb34a7fc640 1 --2- 172.21.15.22:0/2215274261 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb34c06f060 0x7fb34c06b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.592+0000 7fb34a7fc640 1 -- 172.21.15.22:0/2215274261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb34c06db00 msgr2=0x7fb34c14dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:48.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.592+0000 7fb34a7fc640 1 --2- 172.21.15.22:0/2215274261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb34c06db00 0x7fb34c14dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.592+0000 7fb34a7fc640 1 -- 172.21.15.22:0/2215274261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34c0c8070 con 0x7fb34c06dee0 2024-03-29T04:17:48.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.592+0000 7fb34a7fc640 1 --2- 172.21.15.22:0/2215274261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb34c06dee0 0x7fb34c06d1f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb3400190b0 tx=0x7fb340019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:48.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.592+0000 7fb3337fe640 1 -- 172.21.15.22:0/2215274261 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3400b8020 con 0x7fb34c06dee0 2024-03-29T04:17:48.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.592+0000 7fb3337fe640 1 -- 172.21.15.22:0/2215274261 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3400aa620 con 0x7fb34c06dee0 2024-03-29T04:17:48.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.592+0000 7fb3337fe640 1 -- 172.21.15.22:0/2215274261 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3400b0040 con 0x7fb34c06dee0 2024-03-29T04:17:48.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.592+0000 7fb351454640 1 -- 172.21.15.22:0/2215274261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb34c0c8350 con 0x7fb34c06dee0 2024-03-29T04:17:48.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.592+0000 7fb351454640 1 -- 172.21.15.22:0/2215274261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb34c0c8910 con 0x7fb34c06dee0 2024-03-29T04:17:48.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.592+0000 7fb351454640 1 -- 172.21.15.22:0/2215274261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb34c068900 con 0x7fb34c06dee0 2024-03-29T04:17:48.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.592+0000 7fb3337fe640 1 -- 172.21.15.22:0/2215274261 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb3400bb020 con 0x7fb34c06dee0 2024-03-29T04:17:48.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.592+0000 7fb3337fe640 1 --2- 172.21.15.22:0/2215274261 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb338067ff0 0x7fb33806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:48.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.592+0000 7fb3337fe640 1 -- 172.21.15.22:0/2215274261 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb340120640 con 0x7fb34c06dee0 2024-03-29T04:17:48.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.592+0000 7fb34affd640 1 --2- 172.21.15.22:0/2215274261 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb338067ff0 0x7fb33806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:48.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.596+0000 7fb34affd640 1 --2- 172.21.15.22:0/2215274261 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb338067ff0 0x7fb33806a4b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb33c005890 tx=0x7fb33c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:48.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.596+0000 7fb3337fe640 1 -- 172.21.15.22:0/2215274261 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fb3400c2030 con 0x7fb34c06dee0 2024-03-29T04:17:48.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.764+0000 7fb351454640 1 -- 172.21.15.22:0/2215274261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb34c06ec10 con 0x7fb34c06dee0 2024-03-29T04:17:48.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.764+0000 7fb3337fe640 1 -- 172.21.15.22:0/2215274261 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fb340016090 con 0x7fb34c06dee0 2024-03-29T04:17:48.747 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:17:48.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.768+0000 7fb351454640 1 -- 172.21.15.22:0/2215274261 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb338067ff0 msgr2=0x7fb33806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:48.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.768+0000 7fb351454640 1 --2- 172.21.15.22:0/2215274261 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb338067ff0 0x7fb33806a4b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb33c005890 tx=0x7fb33c009040 comp rx=0 tx=0).stop 2024-03-29T04:17:48.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.768+0000 7fb351454640 1 -- 172.21.15.22:0/2215274261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb34c06dee0 msgr2=0x7fb34c06d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:48.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.768+0000 7fb351454640 1 --2- 172.21.15.22:0/2215274261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb34c06dee0 0x7fb34c06d1f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb3400190b0 tx=0x7fb340019240 comp rx=0 tx=0).stop 2024-03-29T04:17:48.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.768+0000 7fb351454640 1 -- 172.21.15.22:0/2215274261 shutdown_connections 2024-03-29T04:17:48.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.768+0000 7fb351454640 1 --2- 172.21.15.22:0/2215274261 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb34c06f060 0x7fb34c06b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.768+0000 7fb351454640 1 --2- 172.21.15.22:0/2215274261 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb338067ff0 0x7fb33806a4b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.768+0000 7fb351454640 1 --2- 172.21.15.22:0/2215274261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb34c06dee0 0x7fb34c06d1f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.768+0000 7fb351454640 1 --2- 172.21.15.22:0/2215274261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb34c06db00 0x7fb34c14dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.768+0000 7fb351454640 1 -- 172.21.15.22:0/2215274261 >> 172.21.15.22:0/2215274261 conn(0x7fb34c0b2c70 msgr2=0x7fb34c0b5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:48.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.768+0000 7fb351454640 1 -- 172.21.15.22:0/2215274261 shutdown_connections 2024-03-29T04:17:48.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.768+0000 7fb351454640 1 -- 172.21.15.22:0/2215274261 wait complete. 2024-03-29T04:17:48.767 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:17:48.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.908+0000 7f18c4d42640 1 Processor -- start 2024-03-29T04:17:48.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.908+0000 7f18c4d42640 1 -- start start 2024-03-29T04:17:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.912+0000 7f18c4d42640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f18c00b9170 0x7f18c00b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.912+0000 7f18c4d42640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18c00d3e60 con 0x7f18c015e600 2024-03-29T04:17:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.912+0000 7f18c4d42640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18c00b7680 con 0x7f18c00d1810 2024-03-29T04:17:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.912+0000 7f18c4d42640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f18c00b7450 con 0x7f18c00b9170 2024-03-29T04:17:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.912+0000 7f18bed9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f18c00b9170 0x7f18c00b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.912+0000 7f18be59c640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f18c015e600 0x7f18c00d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51068/0 (socket says 172.21.15.22:51068) 2024-03-29T04:17:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.912+0000 7f18be59c640 1 -- 172.21.15.22:0/11861089 learned_addr learned my addr 172.21.15.22:0/11861089 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.912+0000 7f18bed9d640 1 -- 172.21.15.22:0/11861089 >> v1:172.21.15.151:6789/0 conn(0x7f18c00d1810 legacy=0x7f18c00d2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.912+0000 7f18bed9d640 1 -- 172.21.15.22:0/11861089 >> v1:172.21.15.22:6789/0 conn(0x7f18c015e600 legacy=0x7f18c00d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.912+0000 7f18bd59a640 1 -- 172.21.15.22:0/11861089 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3431759039 0 0) 0x7f18c00d3e60 con 0x7f18c015e600 2024-03-29T04:17:48.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.912+0000 7f18bed9d640 1 -- 172.21.15.22:0/11861089 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c00b7960 con 0x7f18c00b9170 2024-03-29T04:17:48.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.912+0000 7f18bed9d640 1 --2- 172.21.15.22:0/11861089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c00b9170 0x7f18c00b9f00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009d90 tx=0x7f18b801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963b78eedc15a7bb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:48.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.912+0000 7f18bd59a640 1 -- 172.21.15.22:0/11861089 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b80b2070 con 0x7f18c00b9170 2024-03-29T04:17:48.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.912+0000 7f18bd59a640 1 -- 172.21.15.22:0/11861089 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b80a58b0 con 0x7f18c00b9170 2024-03-29T04:17:48.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.912+0000 7f18bd59a640 1 -- 172.21.15.22:0/11861089 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b80a5c80 con 0x7f18c00b9170 2024-03-29T04:17:48.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18c4d42640 1 -- 172.21.15.22:0/11861089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c00b9170 msgr2=0x7f18c00b9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:48.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18c4d42640 1 --2- 172.21.15.22:0/11861089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c00b9170 0x7f18c00b9f00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009d90 tx=0x7f18b801c8e0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18c4d42640 1 -- 172.21.15.22:0/11861089 shutdown_connections 2024-03-29T04:17:48.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18c4d42640 1 --2- 172.21.15.22:0/11861089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c00b9170 0x7f18c00b9f00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18c4d42640 1 -- 172.21.15.22:0/11861089 >> 172.21.15.22:0/11861089 conn(0x7f18c00b6230 msgr2=0x7f18c00b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:48.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18c4d42640 1 -- 172.21.15.22:0/11861089 shutdown_connections 2024-03-29T04:17:48.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18c4d42640 1 -- 172.21.15.22:0/11861089 wait complete. 2024-03-29T04:17:48.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18c4d42640 1 Processor -- start 2024-03-29T04:17:48.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18c4d42640 1 -- start start 2024-03-29T04:17:48.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18c4d42640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18c00b9170 0x7f18c014e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:48.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18c4d42640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c00d1810 0x7f18c014fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:48.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18c4d42640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18c015e600 0x7f18c0150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:48.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18c4d42640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f18c016bb70 con 0x7f18c00d1810 2024-03-29T04:17:48.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18c4d42640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f18c016b9f0 con 0x7f18c00b9170 2024-03-29T04:17:48.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18c4d42640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f18c016bcf0 con 0x7f18c015e600 2024-03-29T04:17:48.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18bdd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c00d1810 0x7f18c014fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:48.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18bdd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c00d1810 0x7f18c014fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49318/0 (socket says 172.21.15.22:49318) 2024-03-29T04:17:48.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18bdd9b640 1 -- 172.21.15.22:0/2619551331 learned_addr learned my addr 172.21.15.22:0/2619551331 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:48.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18be59c640 1 --2- 172.21.15.22:0/2619551331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18c00b9170 0x7f18c014e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:48.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18bdd9b640 1 -- 172.21.15.22:0/2619551331 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18c015e600 msgr2=0x7f18c0150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:48.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18bdd9b640 1 --2- 172.21.15.22:0/2619551331 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18c015e600 0x7f18c0150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18bdd9b640 1 -- 172.21.15.22:0/2619551331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18c00b9170 msgr2=0x7f18c014e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18bdd9b640 1 --2- 172.21.15.22:0/2619551331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18c00b9170 0x7f18c014e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18bdd9b640 1 -- 172.21.15.22:0/2619551331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c014eaa0 con 0x7f18c00d1810 2024-03-29T04:17:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18be59c640 1 --2- 172.21.15.22:0/2619551331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18c00b9170 0x7f18c014e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18bdd9b640 1 --2- 172.21.15.22:0/2619551331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c00d1810 0x7f18c014fbe0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f18b400c3c0 tx=0x7f18b400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18aeffd640 1 -- 172.21.15.22:0/2619551331 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4017070 con 0x7f18c00d1810 2024-03-29T04:17:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18aeffd640 1 -- 172.21.15.22:0/2619551331 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b40052d0 con 0x7f18c00d1810 2024-03-29T04:17:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18aeffd640 1 -- 172.21.15.22:0/2619551331 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4003840 con 0x7f18c00d1810 2024-03-29T04:17:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18c4d42640 1 -- 172.21.15.22:0/2619551331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c014ed80 con 0x7f18c00d1810 2024-03-29T04:17:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.916+0000 7f18c4d42640 1 -- 172.21.15.22:0/2619551331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c0150f00 con 0x7f18c00d1810 2024-03-29T04:17:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.920+0000 7f18c4d42640 1 -- 172.21.15.22:0/2619551331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c0068890 con 0x7f18c00d1810 2024-03-29T04:17:48.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.920+0000 7f18aeffd640 1 -- 172.21.15.22:0/2619551331 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f18b401a070 con 0x7f18c00d1810 2024-03-29T04:17:48.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.920+0000 7f18aeffd640 1 --2- 172.21.15.22:0/2619551331 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f18a0067fc0 0x7f18a006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:48.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.920+0000 7f18aeffd640 1 -- 172.21.15.22:0/2619551331 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f18b4085ad0 con 0x7f18c00d1810 2024-03-29T04:17:48.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.920+0000 7f18be59c640 1 --2- 172.21.15.22:0/2619551331 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f18a0067fc0 0x7f18a006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:48.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.920+0000 7f18be59c640 1 --2- 172.21.15.22:0/2619551331 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f18a0067fc0 0x7f18a006a480 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f18b0000f70 tx=0x7f18b0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:48.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:48.924+0000 7f18aeffd640 1 -- 172.21.15.22:0/2619551331 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f18b4005480 con 0x7f18c00d1810 2024-03-29T04:17:49.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.100+0000 7f18c4d42640 1 -- 172.21.15.22:0/2619551331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f18c015ce60 con 0x7f18c00d1810 2024-03-29T04:17:49.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.100+0000 7f18aeffd640 1 -- 172.21.15.22:0/2619551331 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f18b4055fd0 con 0x7f18c00d1810 2024-03-29T04:17:49.082 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-03-29T04:17:49.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.100+0000 7f18c4d42640 1 -- 172.21.15.22:0/2619551331 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f18a0067fc0 msgr2=0x7f18a006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:49.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.100+0000 7f18c4d42640 1 --2- 172.21.15.22:0/2619551331 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f18a0067fc0 0x7f18a006a480 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f18b0000f70 tx=0x7f18b0009040 comp rx=0 tx=0).stop 2024-03-29T04:17:49.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.100+0000 7f18c4d42640 1 -- 172.21.15.22:0/2619551331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c00d1810 msgr2=0x7f18c014fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:49.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.100+0000 7f18c4d42640 1 --2- 172.21.15.22:0/2619551331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c00d1810 0x7f18c014fbe0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f18b400c3c0 tx=0x7f18b400c880 comp rx=0 tx=0).stop 2024-03-29T04:17:49.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.100+0000 7f18c4d42640 1 -- 172.21.15.22:0/2619551331 shutdown_connections 2024-03-29T04:17:49.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.100+0000 7f18c4d42640 1 --2- 172.21.15.22:0/2619551331 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18c015e600 0x7f18c0150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.100+0000 7f18c4d42640 1 --2- 172.21.15.22:0/2619551331 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f18a0067fc0 0x7f18a006a480 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.100+0000 7f18c4d42640 1 --2- 172.21.15.22:0/2619551331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c00d1810 0x7f18c014fbe0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.100+0000 7f18c4d42640 1 --2- 172.21.15.22:0/2619551331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18c00b9170 0x7f18c014e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.100+0000 7f18c4d42640 1 -- 172.21.15.22:0/2619551331 >> 172.21.15.22:0/2619551331 conn(0x7f18c00b6230 msgr2=0x7f18c015d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:49.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.100+0000 7f18c4d42640 1 -- 172.21.15.22:0/2619551331 shutdown_connections 2024-03-29T04:17:49.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.104+0000 7f18c4d42640 1 -- 172.21.15.22:0/2619551331 wait complete. 2024-03-29T04:17:49.101 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:17:49.101 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:17:49.108 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:17:49.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.292+0000 7f342f099640 1 Processor -- start 2024-03-29T04:17:49.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.292+0000 7f342f099640 1 -- start start 2024-03-29T04:17:49.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.292+0000 7f342f099640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f3428147e10 0x7f3428071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:49.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.292+0000 7f342f099640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3428162b90 0x7f342806e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:49.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.292+0000 7f342f099640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3428072070 con 0x7f342806ec50 2024-03-29T04:17:49.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.292+0000 7f342f099640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f342806e280 con 0x7f3428162b90 2024-03-29T04:17:49.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.292+0000 7f342f099640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f342806dc30 con 0x7f3428147e10 2024-03-29T04:17:49.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.292+0000 7f3427fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3428162b90 0x7f342806e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:49.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.292+0000 7f3427fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3428162b90 0x7f342806e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49328/0 (socket says 172.21.15.22:49328) 2024-03-29T04:17:49.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.292+0000 7f3427fff640 1 -- 172.21.15.22:0/2632545356 learned_addr learned my addr 172.21.15.22:0/2632545356 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:49.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.292+0000 7f342ce35640 1 --2- 172.21.15.22:0/2632545356 >> v2:172.21.15.151:3300/0 conn(0x7f3428147e10 0x7f3428071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:49.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.292+0000 7f3427fff640 1 -- 172.21.15.22:0/2632545356 >> v2:172.21.15.151:3300/0 conn(0x7f3428147e10 msgr2=0x7f3428071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:49.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.292+0000 7f3427fff640 1 --2- 172.21.15.22:0/2632545356 >> v2:172.21.15.151:3300/0 conn(0x7f3428147e10 0x7f3428071740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.292+0000 7f3427fff640 1 -- 172.21.15.22:0/2632545356 >> v1:172.21.15.151:6789/0 conn(0x7f342806ec50 legacy=0x7f342806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:49.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.292+0000 7f3427fff640 1 -- 172.21.15.22:0/2632545356 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342806df10 con 0x7f3428162b90 2024-03-29T04:17:49.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f3427fff640 1 --2- 172.21.15.22:0/2632545356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3428162b90 0x7f342806e710 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f34280d0f70 tx=0x7f341001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ddaa0e9818b66dd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:49.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f34277fe640 1 -- 172.21.15.22:0/2632545356 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34100a5730 con 0x7f3428162b90 2024-03-29T04:17:49.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f34277fe640 1 -- 172.21.15.22:0/2632545356 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34100a58d0 con 0x7f3428162b90 2024-03-29T04:17:49.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f34277fe640 1 -- 172.21.15.22:0/2632545356 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34100a5ca0 con 0x7f3428162b90 2024-03-29T04:17:49.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f342f099640 1 -- 172.21.15.22:0/2632545356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3428162b90 msgr2=0x7f342806e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:49.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f342f099640 1 --2- 172.21.15.22:0/2632545356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3428162b90 0x7f342806e710 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f34280d0f70 tx=0x7f341001c960 comp rx=0 tx=0).stop 2024-03-29T04:17:49.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f342f099640 1 -- 172.21.15.22:0/2632545356 shutdown_connections 2024-03-29T04:17:49.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f342f099640 1 --2- 172.21.15.22:0/2632545356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3428162b90 0x7f342806e710 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f342f099640 1 --2- 172.21.15.22:0/2632545356 >> v2:172.21.15.151:3300/0 conn(0x7f3428147e10 0x7f3428071740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f342f099640 1 -- 172.21.15.22:0/2632545356 >> 172.21.15.22:0/2632545356 conn(0x7f34280cfee0 msgr2=0x7f34280d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:49.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f342f099640 1 -- 172.21.15.22:0/2632545356 shutdown_connections 2024-03-29T04:17:49.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f342f099640 1 -- 172.21.15.22:0/2632545356 wait complete. 2024-03-29T04:17:49.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f342f099640 1 Processor -- start 2024-03-29T04:17:49.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f342f099640 1 -- start start 2024-03-29T04:17:49.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f342f099640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f342806ec50 0x7f342815d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:49.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f342f099640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3428147e10 0x7f34281402a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:49.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f342f099640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3428162b90 0x7f34281410e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:49.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f342f099640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f342806cf00 con 0x7f342806ec50 2024-03-29T04:17:49.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f342f099640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f342806cd80 con 0x7f3428147e10 2024-03-29T04:17:49.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f342f099640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f342816b0d0 con 0x7f3428162b90 2024-03-29T04:17:49.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f342ce35640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f342806ec50 0x7f342815d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:49.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f342ce35640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f342806ec50 0x7f342815d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49334/0 (socket says 172.21.15.22:49334) 2024-03-29T04:17:49.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.296+0000 7f342ce35640 1 -- 172.21.15.22:0/3903944742 learned_addr learned my addr 172.21.15.22:0/3903944742 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.300+0000 7f3427fff640 1 --2- 172.21.15.22:0/3903944742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3428147e10 0x7f34281402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.300+0000 7f342d636640 1 --2- 172.21.15.22:0/3903944742 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3428162b90 0x7f34281410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.300+0000 7f342ce35640 1 -- 172.21.15.22:0/3903944742 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3428162b90 msgr2=0x7f34281410e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.300+0000 7f342ce35640 1 --2- 172.21.15.22:0/3903944742 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3428162b90 0x7f34281410e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.300+0000 7f342ce35640 1 -- 172.21.15.22:0/3903944742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3428147e10 msgr2=0x7f34281402a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.300+0000 7f342ce35640 1 --2- 172.21.15.22:0/3903944742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3428147e10 0x7f34281402a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.300+0000 7f342ce35640 1 -- 172.21.15.22:0/3903944742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3428140bf0 con 0x7f342806ec50 2024-03-29T04:17:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.300+0000 7f342ce35640 1 --2- 172.21.15.22:0/3903944742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f342806ec50 0x7f342815d510 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f341800d800 tx=0x7f341800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.300+0000 7f34257fa640 1 -- 172.21.15.22:0/3903944742 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341800b6c0 con 0x7f342806ec50 2024-03-29T04:17:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.300+0000 7f34257fa640 1 -- 172.21.15.22:0/3903944742 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3418019070 con 0x7f342806ec50 2024-03-29T04:17:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.300+0000 7f342f099640 1 -- 172.21.15.22:0/3903944742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342813ca40 con 0x7f342806ec50 2024-03-29T04:17:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.300+0000 7f34257fa640 1 -- 172.21.15.22:0/3903944742 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341800bae0 con 0x7f342806ec50 2024-03-29T04:17:49.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.300+0000 7f342f099640 1 -- 172.21.15.22:0/3903944742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342813d000 con 0x7f342806ec50 2024-03-29T04:17:49.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.300+0000 7f342f099640 1 -- 172.21.15.22:0/3903944742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3428068890 con 0x7f342806ec50 2024-03-29T04:17:49.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.300+0000 7f34257fa640 1 -- 172.21.15.22:0/3903944742 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f341800bc80 con 0x7f342806ec50 2024-03-29T04:17:49.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.300+0000 7f34257fa640 1 --2- 172.21.15.22:0/3903944742 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3408067fc0 0x7f340806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:49.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.300+0000 7f34257fa640 1 -- 172.21.15.22:0/3903944742 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3418086e00 con 0x7f342806ec50 2024-03-29T04:17:49.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.300+0000 7f3427fff640 1 --2- 172.21.15.22:0/3903944742 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3408067fc0 0x7f340806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:49.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.300+0000 7f3427fff640 1 --2- 172.21.15.22:0/3903944742 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3408067fc0 0x7f340806a480 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f34100ae000 tx=0x7f34100a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:49.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.304+0000 7f34257fa640 1 -- 172.21.15.22:0/3903944742 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f3418057270 con 0x7f342806ec50 2024-03-29T04:17:49.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.468+0000 7f342f099640 1 -- 172.21.15.22:0/3903944742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34280b0840 con 0x7f342806ec50 2024-03-29T04:17:49.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.472+0000 7f34257fa640 1 -- 172.21.15.22:0/3903944742 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18514 (secure 0 0 0) 0x7f3418056b70 con 0x7f342806ec50 2024-03-29T04:17:49.454 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:49.454 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:17:42.645632+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:17:49.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.472+0000 7f342f099640 1 -- 172.21.15.22:0/3903944742 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3408067fc0 msgr2=0x7f340806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:49.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.472+0000 7f342f099640 1 --2- 172.21.15.22:0/3903944742 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3408067fc0 0x7f340806a480 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f34100ae000 tx=0x7f34100a7070 comp rx=0 tx=0).stop 2024-03-29T04:17:49.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.472+0000 7f342f099640 1 -- 172.21.15.22:0/3903944742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f342806ec50 msgr2=0x7f342815d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:49.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.472+0000 7f342f099640 1 --2- 172.21.15.22:0/3903944742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f342806ec50 0x7f342815d510 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f341800d800 tx=0x7f341800dcc0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.472+0000 7f342f099640 1 -- 172.21.15.22:0/3903944742 shutdown_connections 2024-03-29T04:17:49.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.472+0000 7f342f099640 1 --2- 172.21.15.22:0/3903944742 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3428162b90 0x7f34281410e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.472+0000 7f342f099640 1 --2- 172.21.15.22:0/3903944742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3428147e10 0x7f34281402a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.472+0000 7f342f099640 1 --2- 172.21.15.22:0/3903944742 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3408067fc0 0x7f340806a480 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.472+0000 7f342f099640 1 --2- 172.21.15.22:0/3903944742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f342806ec50 0x7f342815d510 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.472+0000 7f342f099640 1 -- 172.21.15.22:0/3903944742 >> 172.21.15.22:0/3903944742 conn(0x7f34280cfee0 msgr2=0x7f34280aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:49.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.472+0000 7f342f099640 1 -- 172.21.15.22:0/3903944742 shutdown_connections 2024-03-29T04:17:49.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.472+0000 7f342f099640 1 -- 172.21.15.22:0/3903944742 wait complete. 2024-03-29T04:17:49.473 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:17:34.034142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:17:34.886659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:17:38.628632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:17:49.474 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:17:49.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb7bba640 1 Processor -- start 2024-03-29T04:17:49.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb7bba640 1 -- start start 2024-03-29T04:17:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb7bba640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f2cb0158b40 0x7f2cb00b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb7bba640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cb0158f20 con 0x7f2cb00cce60 2024-03-29T04:17:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb7bba640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cb01620f0 con 0x7f2cb00b5530 2024-03-29T04:17:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb7bba640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f2cb0162d00 con 0x7f2cb0158b40 2024-03-29T04:17:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb6157640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f2cb00cce60 0x7f2cb00b6c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51074/0 (socket says 172.21.15.22:51074) 2024-03-29T04:17:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb6157640 1 -- 172.21.15.22:0/2920511319 learned_addr learned my addr 172.21.15.22:0/2920511319 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb5956640 1 --2- 172.21.15.22:0/2920511319 >> v2:172.21.15.151:3300/0 conn(0x7f2cb0158b40 0x7f2cb00b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb4954640 1 -- 172.21.15.22:0/2920511319 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2986516732 0 0) 0x7f2cb0158f20 con 0x7f2cb00cce60 2024-03-29T04:17:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb4954640 1 -- 172.21.15.22:0/2920511319 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c9c003530 con 0x7f2cb00cce60 2024-03-29T04:17:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb4954640 1 -- 172.21.15.22:0/2920511319 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4242336915 0 0) 0x7f2cb01620f0 con 0x7f2cb00b5530 2024-03-29T04:17:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb4954640 1 -- 172.21.15.22:0/2920511319 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2cb0158f20 con 0x7f2cb00b5530 2024-03-29T04:17:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb4954640 1 -- 172.21.15.22:0/2920511319 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1341134829 0 0) 0x7f2c9c003530 con 0x7f2cb00cce60 2024-03-29T04:17:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb4954640 1 -- 172.21.15.22:0/2920511319 >> v2:172.21.15.151:3300/0 conn(0x7f2cb0158b40 msgr2=0x7f2cb00b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb4954640 1 --2- 172.21.15.22:0/2920511319 >> v2:172.21.15.151:3300/0 conn(0x7f2cb0158b40 0x7f2cb00b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb4954640 1 -- 172.21.15.22:0/2920511319 >> v1:172.21.15.151:6789/0 conn(0x7f2cb00b5530 legacy=0x7f2cb00cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb4954640 1 -- 172.21.15.22:0/2920511319 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb01623d0 con 0x7f2cb00cce60 2024-03-29T04:17:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb4954640 1 -- 172.21.15.22:0/2920511319 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f2ca8001a20 con 0x7f2cb00cce60 2024-03-29T04:17:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb4954640 1 -- 172.21.15.22:0/2920511319 >> v1:172.21.15.22:6789/0 conn(0x7f2cb00cce60 legacy=0x7f2cb00b6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb4954640 1 --2- 172.21.15.22:0/2920511319 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2c9c003ba0 0x7f2c9c003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb4954640 1 --2- 172.21.15.22:0/2920511319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c9c004810 0x7f2c9c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:49.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb4954640 1 --2- 172.21.15.22:0/2920511319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c9c005220 0x7f2c9c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:49.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb4954640 1 -- 172.21.15.22:0/2920511319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb0162d00 con 0x7f2c9c005220 2024-03-29T04:17:49.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb4954640 1 -- 172.21.15.22:0/2920511319 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c00c150 con 0x7f2c9c004810 2024-03-29T04:17:49.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb4954640 1 -- 172.21.15.22:0/2920511319 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f2c9c00c350 con 0x7f2c9c003ba0 2024-03-29T04:17:49.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb6157640 1 --2- 172.21.15.22:0/2920511319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c9c005220 0x7f2c9c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:49.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb5155640 1 --2- 172.21.15.22:0/2920511319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c9c004810 0x7f2c9c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:49.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb5956640 1 --2- 172.21.15.22:0/2920511319 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2c9c003ba0 0x7f2c9c003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:49.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb6157640 1 -- 172.21.15.22:0/2920511319 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2c9c003ba0 msgr2=0x7f2c9c003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:49.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb6157640 1 --2- 172.21.15.22:0/2920511319 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2c9c003ba0 0x7f2c9c003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb6157640 1 -- 172.21.15.22:0/2920511319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c9c004810 msgr2=0x7f2c9c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:49.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb6157640 1 --2- 172.21.15.22:0/2920511319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c9c004810 0x7f2c9c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.624+0000 7f2cb6157640 1 -- 172.21.15.22:0/2920511319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9c003530 con 0x7f2c9c005220 2024-03-29T04:17:49.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.628+0000 7f2cb6157640 1 -- 172.21.15.22:0/2920511319 reap_dead start 2024-03-29T04:17:49.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.628+0000 7f2cb6157640 1 --2- 172.21.15.22:0/2920511319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c9c005220 0x7f2c9c00b9d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800a720 tx=0x7f2ca801d200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9dc375669fc0e57 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:49.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.628+0000 7f2cb4954640 1 -- 172.21.15.22:0/2920511319 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800a370 con 0x7f2c9c005220 2024-03-29T04:17:49.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.628+0000 7f2cb4954640 1 -- 172.21.15.22:0/2920511319 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca80ad630 con 0x7f2c9c005220 2024-03-29T04:17:49.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.628+0000 7f2cb4954640 1 -- 172.21.15.22:0/2920511319 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca80ada20 con 0x7f2c9c005220 2024-03-29T04:17:49.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.628+0000 7f2cb7bba640 1 -- 172.21.15.22:0/2920511319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c9c005220 msgr2=0x7f2c9c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:49.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.628+0000 7f2cb7bba640 1 --2- 172.21.15.22:0/2920511319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c9c005220 0x7f2c9c00b9d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800a720 tx=0x7f2ca801d200 comp rx=0 tx=0).stop 2024-03-29T04:17:49.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.628+0000 7f2cb7bba640 1 -- 172.21.15.22:0/2920511319 shutdown_connections 2024-03-29T04:17:49.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.628+0000 7f2cb7bba640 1 --2- 172.21.15.22:0/2920511319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c9c005220 0x7f2c9c00b9d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.628+0000 7f2cb7bba640 1 -- 172.21.15.22:0/2920511319 >> 172.21.15.22:0/2920511319 conn(0x7f2cb00b3cf0 msgr2=0x7f2cb00b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:49.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.628+0000 7f2cb7bba640 1 -- 172.21.15.22:0/2920511319 shutdown_connections 2024-03-29T04:17:49.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.628+0000 7f2cb7bba640 1 -- 172.21.15.22:0/2920511319 wait complete. 2024-03-29T04:17:49.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.628+0000 7f2cb7bba640 1 Processor -- start 2024-03-29T04:17:49.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.628+0000 7f2cb7bba640 1 -- start start 2024-03-29T04:17:49.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2cb7bba640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2c9c005220 0x7f2cb01f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:49.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2cb7bba640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cb00c83d0 0x7f2cb00ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:49.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2cb7bba640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cb00caea0 0x7f2cb00c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:49.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2cb7bba640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb00d1d30 con 0x7f2cb00caea0 2024-03-29T04:17:49.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2cb7bba640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb0162990 con 0x7f2cb00c83d0 2024-03-29T04:17:49.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2cb7bba640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb00d1eb0 con 0x7f2c9c005220 2024-03-29T04:17:49.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2cb6157640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cb00caea0 0x7f2cb00c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:49.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2cb6157640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cb00caea0 0x7f2cb00c92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49346/0 (socket says 172.21.15.22:49346) 2024-03-29T04:17:49.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2cb6157640 1 -- 172.21.15.22:0/3411840332 learned_addr learned my addr 172.21.15.22:0/3411840332 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:49.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2cb5155640 1 --2- 172.21.15.22:0/3411840332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cb00c83d0 0x7f2cb00ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:49.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2cb5956640 1 --2- 172.21.15.22:0/3411840332 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2c9c005220 0x7f2cb01f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:49.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2cb6157640 1 -- 172.21.15.22:0/3411840332 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2c9c005220 msgr2=0x7f2cb01f4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:49.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2cb6157640 1 --2- 172.21.15.22:0/3411840332 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2c9c005220 0x7f2cb01f4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2cb6157640 1 -- 172.21.15.22:0/3411840332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cb00c83d0 msgr2=0x7f2cb00ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2cb6157640 1 --2- 172.21.15.22:0/3411840332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cb00c83d0 0x7f2cb00ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2cb6157640 1 -- 172.21.15.22:0/3411840332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb01f46d0 con 0x7f2cb00caea0 2024-03-29T04:17:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2cb5956640 1 --2- 172.21.15.22:0/3411840332 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2c9c005220 0x7f2cb01f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:17:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2cb6157640 1 --2- 172.21.15.22:0/3411840332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cb00caea0 0x7f2cb00c92f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2ca801d030 tx=0x7f2c9c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2ca27fc640 1 -- 172.21.15.22:0/3411840332 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800a370 con 0x7f2cb00caea0 2024-03-29T04:17:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2ca27fc640 1 -- 172.21.15.22:0/3411840332 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca8004b70 con 0x7f2cb00caea0 2024-03-29T04:17:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2ca27fc640 1 -- 172.21.15.22:0/3411840332 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca80ad7e0 con 0x7f2cb00caea0 2024-03-29T04:17:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2cb7bba640 1 -- 172.21.15.22:0/3411840332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb00ca090 con 0x7f2cb00caea0 2024-03-29T04:17:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2cb7bba640 1 -- 172.21.15.22:0/3411840332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb00c5eb0 con 0x7f2cb00caea0 2024-03-29T04:17:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2cb7bba640 1 -- 172.21.15.22:0/3411840332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb0068900 con 0x7f2cb00caea0 2024-03-29T04:17:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.632+0000 7f2ca27fc640 1 -- 172.21.15.22:0/3411840332 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f2ca80ad980 con 0x7f2cb00caea0 2024-03-29T04:17:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.636+0000 7f2ca27fc640 1 --2- 172.21.15.22:0/3411840332 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2c94067fc0 0x7f2c9406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.636+0000 7f2ca27fc640 1 -- 172.21.15.22:0/3411840332 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2ca8120c10 con 0x7f2cb00caea0 2024-03-29T04:17:49.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.636+0000 7f2cb5956640 1 --2- 172.21.15.22:0/3411840332 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2c94067fc0 0x7f2c9406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:49.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.636+0000 7f2cb5956640 1 --2- 172.21.15.22:0/3411840332 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2c94067fc0 0x7f2c9406a480 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009a70 tx=0x7f2ca4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:49.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.640+0000 7f2ca27fc640 1 -- 172.21.15.22:0/3411840332 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f2ca80b4070 con 0x7f2cb00caea0 2024-03-29T04:17:49.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.800+0000 7f2cb7bba640 1 -- 172.21.15.22:0/3411840332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2cb00cd540 con 0x7f2cb00caea0 2024-03-29T04:17:49.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.800+0000 7f2ca27fc640 1 -- 172.21.15.22:0/3411840332 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f2ca8016f10 con 0x7f2cb00caea0 2024-03-29T04:17:49.783 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:17:49.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.804+0000 7f2cb7bba640 1 -- 172.21.15.22:0/3411840332 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2c94067fc0 msgr2=0x7f2c9406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:49.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.804+0000 7f2cb7bba640 1 --2- 172.21.15.22:0/3411840332 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2c94067fc0 0x7f2c9406a480 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009a70 tx=0x7f2ca4008040 comp rx=0 tx=0).stop 2024-03-29T04:17:49.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.804+0000 7f2cb7bba640 1 -- 172.21.15.22:0/3411840332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cb00caea0 msgr2=0x7f2cb00c92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:49.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.804+0000 7f2cb7bba640 1 --2- 172.21.15.22:0/3411840332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cb00caea0 0x7f2cb00c92f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2ca801d030 tx=0x7f2c9c005840 comp rx=0 tx=0).stop 2024-03-29T04:17:49.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.804+0000 7f2cb7bba640 1 -- 172.21.15.22:0/3411840332 shutdown_connections 2024-03-29T04:17:49.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.804+0000 7f2cb7bba640 1 --2- 172.21.15.22:0/3411840332 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2c94067fc0 0x7f2c9406a480 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.804+0000 7f2cb7bba640 1 --2- 172.21.15.22:0/3411840332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cb00caea0 0x7f2cb00c92f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.804+0000 7f2cb7bba640 1 --2- 172.21.15.22:0/3411840332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cb00c83d0 0x7f2cb00ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.804+0000 7f2cb7bba640 1 --2- 172.21.15.22:0/3411840332 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f2c9c005220 0x7f2cb01f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.804+0000 7f2cb7bba640 1 -- 172.21.15.22:0/3411840332 >> 172.21.15.22:0/3411840332 conn(0x7f2cb00b3cf0 msgr2=0x7f2cb00cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:49.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.804+0000 7f2cb7bba640 1 -- 172.21.15.22:0/3411840332 shutdown_connections 2024-03-29T04:17:49.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.804+0000 7f2cb7bba640 1 -- 172.21.15.22:0/3411840332 wait complete. 2024-03-29T04:17:49.801 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:17:49.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.936+0000 7feb0ceda640 1 Processor -- start 2024-03-29T04:17:49.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.936+0000 7feb0ceda640 1 -- start start 2024-03-29T04:17:49.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.940+0000 7feb0ceda640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7feb08144930 0x7feb080b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:49.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.940+0000 7feb0ceda640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb080b4870 0x7feb0806c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:49.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.940+0000 7feb0ceda640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7feb0806cbb0 0x7feb080713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:49.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.940+0000 7feb0ceda640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7feb080cdb90 con 0x7feb0806cbb0 2024-03-29T04:17:49.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.940+0000 7feb0ceda640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7feb080b33c0 con 0x7feb08144930 2024-03-29T04:17:49.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.940+0000 7feb0ceda640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7feb080b3660 con 0x7feb080b4870 2024-03-29T04:17:49.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.940+0000 7feb06d9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7feb0806cbb0 0x7feb080713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:49.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.940+0000 7feb06d9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7feb0806cbb0 0x7feb080713f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49352/0 (socket says 172.21.15.22:49352) 2024-03-29T04:17:49.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.940+0000 7feb06d9d640 1 -- 172.21.15.22:0/3682943426 learned_addr learned my addr 172.21.15.22:0/3682943426 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:49.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.940+0000 7feb0659c640 1 --2- 172.21.15.22:0/3682943426 >> v2:172.21.15.151:3300/0 conn(0x7feb08144930 0x7feb080b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:49.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.940+0000 7feb05d9b640 1 --2- 172.21.15.22:0/3682943426 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb080b4870 0x7feb0806c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:49.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.940+0000 7feb06d9d640 1 -- 172.21.15.22:0/3682943426 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb080b4870 msgr2=0x7feb0806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:49.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.940+0000 7feb06d9d640 1 --2- 172.21.15.22:0/3682943426 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb080b4870 0x7feb0806c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.940+0000 7feb06d9d640 1 -- 172.21.15.22:0/3682943426 >> v2:172.21.15.151:3300/0 conn(0x7feb08144930 msgr2=0x7feb080b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:49.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.940+0000 7feb06d9d640 1 --2- 172.21.15.22:0/3682943426 >> v2:172.21.15.151:3300/0 conn(0x7feb08144930 0x7feb080b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.940+0000 7feb06d9d640 1 -- 172.21.15.22:0/3682943426 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb080b3940 con 0x7feb0806cbb0 2024-03-29T04:17:49.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.940+0000 7feb06d9d640 1 --2- 172.21.15.22:0/3682943426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb0806cbb0 0x7feb080713f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feb00009e50 tx=0x7feb0001c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ea49d34182a900 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:49.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.940+0000 7feb0559a640 1 -- 172.21.15.22:0/3682943426 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb000ac070 con 0x7feb0806cbb0 2024-03-29T04:17:49.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.940+0000 7feb0559a640 1 -- 172.21.15.22:0/3682943426 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb000a8440 con 0x7feb0806cbb0 2024-03-29T04:17:49.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.940+0000 7feb0559a640 1 -- 172.21.15.22:0/3682943426 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb000a8810 con 0x7feb0806cbb0 2024-03-29T04:17:49.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb0ceda640 1 -- 172.21.15.22:0/3682943426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb0806cbb0 msgr2=0x7feb080713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:49.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb0ceda640 1 --2- 172.21.15.22:0/3682943426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb0806cbb0 0x7feb080713f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feb00009e50 tx=0x7feb0001c6a0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb0ceda640 1 -- 172.21.15.22:0/3682943426 shutdown_connections 2024-03-29T04:17:49.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb0ceda640 1 --2- 172.21.15.22:0/3682943426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb0806cbb0 0x7feb080713f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb0ceda640 1 --2- 172.21.15.22:0/3682943426 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb080b4870 0x7feb0806c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb0ceda640 1 --2- 172.21.15.22:0/3682943426 >> v2:172.21.15.151:3300/0 conn(0x7feb08144930 0x7feb080b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb0ceda640 1 -- 172.21.15.22:0/3682943426 >> 172.21.15.22:0/3682943426 conn(0x7feb080d00a0 msgr2=0x7feb080d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:49.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb0ceda640 1 -- 172.21.15.22:0/3682943426 shutdown_connections 2024-03-29T04:17:49.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb0ceda640 1 -- 172.21.15.22:0/3682943426 wait complete. 2024-03-29T04:17:49.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb0ceda640 1 Processor -- start 2024-03-29T04:17:49.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb0ceda640 1 -- start start 2024-03-29T04:17:49.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb0ceda640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb0806cbb0 0x7feb081f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:49.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb0ceda640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb080b4870 0x7feb0813cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:49.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb0ceda640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb08144930 0x7feb0815a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:49.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb0ceda640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7feb080ce180 con 0x7feb08144930 2024-03-29T04:17:49.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb0ceda640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feb080ce000 con 0x7feb0806cbb0 2024-03-29T04:17:49.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb0ceda640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7feb080ce300 con 0x7feb080b4870 2024-03-29T04:17:49.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb06d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb08144930 0x7feb0815a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:49.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb06d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb08144930 0x7feb0815a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49368/0 (socket says 172.21.15.22:49368) 2024-03-29T04:17:49.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb06d9d640 1 -- 172.21.15.22:0/3162720067 learned_addr learned my addr 172.21.15.22:0/3162720067 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:49.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb0659c640 1 --2- 172.21.15.22:0/3162720067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb0806cbb0 0x7feb081f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:49.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb05d9b640 1 --2- 172.21.15.22:0/3162720067 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb080b4870 0x7feb0813cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:49.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb06d9d640 1 -- 172.21.15.22:0/3162720067 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb080b4870 msgr2=0x7feb0813cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:49.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb06d9d640 1 --2- 172.21.15.22:0/3162720067 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb080b4870 0x7feb0813cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb06d9d640 1 -- 172.21.15.22:0/3162720067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb0806cbb0 msgr2=0x7feb081f6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:49.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb06d9d640 1 --2- 172.21.15.22:0/3162720067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb0806cbb0 0x7feb081f6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:49.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb06d9d640 1 -- 172.21.15.22:0/3162720067 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0813d120 con 0x7feb08144930 2024-03-29T04:17:49.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb05d9b640 1 --2- 172.21.15.22:0/3162720067 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb080b4870 0x7feb0813cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:49.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb06d9d640 1 --2- 172.21.15.22:0/3162720067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb08144930 0x7feb0815a0d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feb0001ad50 tx=0x7feb00019440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:49.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feaf6ffd640 1 -- 172.21.15.22:0/3162720067 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb000ac040 con 0x7feb08144930 2024-03-29T04:17:49.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feaf6ffd640 1 -- 172.21.15.22:0/3162720067 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb000bd450 con 0x7feb08144930 2024-03-29T04:17:49.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb0ceda640 1 -- 172.21.15.22:0/3162720067 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb080af870 con 0x7feb08144930 2024-03-29T04:17:49.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feaf6ffd640 1 -- 172.21.15.22:0/3162720067 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00009510 con 0x7feb08144930 2024-03-29T04:17:49.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.944+0000 7feb0ceda640 1 -- 172.21.15.22:0/3162720067 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb080afe30 con 0x7feb08144930 2024-03-29T04:17:49.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.948+0000 7feb0ceda640 1 -- 172.21.15.22:0/3162720067 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb08068900 con 0x7feb08144930 2024-03-29T04:17:49.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.948+0000 7feaf6ffd640 1 -- 172.21.15.22:0/3162720067 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7feb000af070 con 0x7feb08144930 2024-03-29T04:17:49.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.948+0000 7feaf6ffd640 1 --2- 172.21.15.22:0/3162720067 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7feae8067fc0 0x7feae806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:49.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.948+0000 7feaf6ffd640 1 -- 172.21.15.22:0/3162720067 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7feb00121150 con 0x7feb08144930 2024-03-29T04:17:49.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.948+0000 7feb0659c640 1 --2- 172.21.15.22:0/3162720067 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7feae8067fc0 0x7feae806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:49.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.948+0000 7feb0659c640 1 --2- 172.21.15.22:0/3162720067 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7feae8067fc0 0x7feae806a480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feb081fa3f0 tx=0x7feaf8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:49.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:49.952+0000 7feaf6ffd640 1 -- 172.21.15.22:0/3162720067 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7feb000165e0 con 0x7feb08144930 2024-03-29T04:17:50.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.108+0000 7feb0ceda640 1 -- 172.21.15.22:0/3162720067 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feb0815a610 con 0x7feb08144930 2024-03-29T04:17:50.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.108+0000 7feaf6ffd640 1 -- 172.21.15.22:0/3162720067 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7feb000f1650 con 0x7feb08144930 2024-03-29T04:17:50.090 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:17:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.112+0000 7feb0ceda640 1 -- 172.21.15.22:0/3162720067 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7feae8067fc0 msgr2=0x7feae806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.112+0000 7feb0ceda640 1 --2- 172.21.15.22:0/3162720067 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7feae8067fc0 0x7feae806a480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feb081fa3f0 tx=0x7feaf8008040 comp rx=0 tx=0).stop 2024-03-29T04:17:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.112+0000 7feb0ceda640 1 -- 172.21.15.22:0/3162720067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb08144930 msgr2=0x7feb0815a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.112+0000 7feb0ceda640 1 --2- 172.21.15.22:0/3162720067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb08144930 0x7feb0815a0d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feb0001ad50 tx=0x7feb00019440 comp rx=0 tx=0).stop 2024-03-29T04:17:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.112+0000 7feb0ceda640 1 -- 172.21.15.22:0/3162720067 shutdown_connections 2024-03-29T04:17:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.112+0000 7feb0ceda640 1 --2- 172.21.15.22:0/3162720067 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7feae8067fc0 0x7feae806a480 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.112+0000 7feb0ceda640 1 --2- 172.21.15.22:0/3162720067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb08144930 0x7feb0815a0d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.112+0000 7feb0ceda640 1 --2- 172.21.15.22:0/3162720067 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb080b4870 0x7feb0813cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.112+0000 7feb0ceda640 1 --2- 172.21.15.22:0/3162720067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb0806cbb0 0x7feb081f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.112+0000 7feb0ceda640 1 -- 172.21.15.22:0/3162720067 >> 172.21.15.22:0/3162720067 conn(0x7feb080d00a0 msgr2=0x7feb0806eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:50.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.112+0000 7feb0ceda640 1 -- 172.21.15.22:0/3162720067 shutdown_connections 2024-03-29T04:17:50.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.112+0000 7feb0ceda640 1 -- 172.21.15.22:0/3162720067 wait complete. 2024-03-29T04:17:50.108 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:17:50.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.252+0000 7f5307a87640 1 Processor -- start 2024-03-29T04:17:50.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.252+0000 7f5307a87640 1 -- start start 2024-03-29T04:17:50.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.252+0000 7f5307a87640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53000cc8c0 0x7f53000d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:50.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.252+0000 7f5307a87640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f53000ce3c0 0x7f53000d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:50.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.252+0000 7f5307a87640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f530006eab0 con 0x7f53001531a0 2024-03-29T04:17:50.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.252+0000 7f5307a87640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f53000d0de0 con 0x7f53000ce3c0 2024-03-29T04:17:50.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.252+0000 7f5307a87640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5300070700 con 0x7f53000cc8c0 2024-03-29T04:17:50.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.252+0000 7f5305823640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f53001531a0 0x7f530006fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51090/0 (socket says 172.21.15.22:51090) 2024-03-29T04:17:50.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.252+0000 7f5305823640 1 -- 172.21.15.22:0/1751419730 learned_addr learned my addr 172.21.15.22:0/1751419730 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:50.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.252+0000 7f5306024640 1 --2- 172.21.15.22:0/1751419730 >> v2:172.21.15.22:3300/0 conn(0x7f53000ce3c0 0x7f53000d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:50.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.252+0000 7f5305022640 1 --2- 172.21.15.22:0/1751419730 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53000cc8c0 0x7f53000d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:50.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.252+0000 7f5304821640 1 -- 172.21.15.22:0/1751419730 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1911355940 0 0) 0x7f530006eab0 con 0x7f53001531a0 2024-03-29T04:17:50.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.252+0000 7f5304821640 1 -- 172.21.15.22:0/1751419730 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52ec003530 con 0x7f53001531a0 2024-03-29T04:17:50.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.252+0000 7f5306024640 1 -- 172.21.15.22:0/1751419730 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53000cc8c0 msgr2=0x7f53000d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:50.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.252+0000 7f5306024640 1 --2- 172.21.15.22:0/1751419730 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53000cc8c0 0x7f53000d1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.252+0000 7f5306024640 1 -- 172.21.15.22:0/1751419730 >> v1:172.21.15.22:6789/0 conn(0x7f53001531a0 legacy=0x7f530006fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:50.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.252+0000 7f5306024640 1 -- 172.21.15.22:0/1751419730 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530006ed90 con 0x7f53000ce3c0 2024-03-29T04:17:50.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.256+0000 7f5306024640 1 --2- 172.21.15.22:0/1751419730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53000ce3c0 0x7f53000d16c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0097c0 tx=0x7f52fc01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8956193841cd4a9a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:50.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.256+0000 7f5304821640 1 -- 172.21.15.22:0/1751419730 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc0b1070 con 0x7f53000ce3c0 2024-03-29T04:17:50.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.256+0000 7f5304821640 1 -- 172.21.15.22:0/1751419730 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52fc0a8050 con 0x7f53000ce3c0 2024-03-29T04:17:50.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.256+0000 7f5304821640 1 -- 172.21.15.22:0/1751419730 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc0ac860 con 0x7f53000ce3c0 2024-03-29T04:17:50.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.256+0000 7f5307a87640 1 -- 172.21.15.22:0/1751419730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53000ce3c0 msgr2=0x7f53000d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:50.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.256+0000 7f5307a87640 1 --2- 172.21.15.22:0/1751419730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53000ce3c0 0x7f53000d16c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0097c0 tx=0x7f52fc01c3e0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.256+0000 7f5307a87640 1 -- 172.21.15.22:0/1751419730 shutdown_connections 2024-03-29T04:17:50.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.256+0000 7f5307a87640 1 --2- 172.21.15.22:0/1751419730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53000ce3c0 0x7f53000d16c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.256+0000 7f5307a87640 1 --2- 172.21.15.22:0/1751419730 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53000cc8c0 0x7f53000d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.256+0000 7f5307a87640 1 -- 172.21.15.22:0/1751419730 >> 172.21.15.22:0/1751419730 conn(0x7f53000b3c80 msgr2=0x7f53000b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:50.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.256+0000 7f5307a87640 1 -- 172.21.15.22:0/1751419730 shutdown_connections 2024-03-29T04:17:50.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.256+0000 7f5307a87640 1 -- 172.21.15.22:0/1751419730 wait complete. 2024-03-29T04:17:50.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.256+0000 7f5307a87640 1 Processor -- start 2024-03-29T04:17:50.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.256+0000 7f5307a87640 1 -- start start 2024-03-29T04:17:50.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f5307a87640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53000cc8c0 0x7f5300155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:50.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f5307a87640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53000ce3c0 0x7f53000c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:50.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f5307a87640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53001531a0 0x7f53000c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:50.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f5307a87640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5300153700 con 0x7f53000ce3c0 2024-03-29T04:17:50.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f5307a87640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5300153580 con 0x7f53000cc8c0 2024-03-29T04:17:50.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f5307a87640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f53000d05e0 con 0x7f53001531a0 2024-03-29T04:17:50.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f5305022640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53000ce3c0 0x7f53000c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:50.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f5305022640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53000ce3c0 0x7f53000c5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49382/0 (socket says 172.21.15.22:49382) 2024-03-29T04:17:50.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f5305022640 1 -- 172.21.15.22:0/4024765458 learned_addr learned my addr 172.21.15.22:0/4024765458 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:50.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f5305823640 1 --2- 172.21.15.22:0/4024765458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53000cc8c0 0x7f5300155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f5306024640 1 --2- 172.21.15.22:0/4024765458 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53001531a0 0x7f53000c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f5305022640 1 -- 172.21.15.22:0/4024765458 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53001531a0 msgr2=0x7f53000c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f5305022640 1 --2- 172.21.15.22:0/4024765458 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53001531a0 0x7f53000c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f5305022640 1 -- 172.21.15.22:0/4024765458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53000cc8c0 msgr2=0x7f5300155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f5305022640 1 --2- 172.21.15.22:0/4024765458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53000cc8c0 0x7f5300155ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f5305022640 1 -- 172.21.15.22:0/4024765458 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5300156af0 con 0x7f53000ce3c0 2024-03-29T04:17:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f5305022640 1 --2- 172.21.15.22:0/4024765458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53000ce3c0 0x7f53000c5ca0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f52f4007f20 tx=0x7f52f400e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f52f27fc640 1 -- 172.21.15.22:0/4024765458 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4014620 con 0x7f53000ce3c0 2024-03-29T04:17:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f52f27fc640 1 -- 172.21.15.22:0/4024765458 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52f4019070 con 0x7f53000ce3c0 2024-03-29T04:17:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f5307a87640 1 -- 172.21.15.22:0/4024765458 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53000b70b0 con 0x7f53000ce3c0 2024-03-29T04:17:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f52f27fc640 1 -- 172.21.15.22:0/4024765458 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4014a00 con 0x7f53000ce3c0 2024-03-29T04:17:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f5307a87640 1 -- 172.21.15.22:0/4024765458 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53000b7670 con 0x7f53000ce3c0 2024-03-29T04:17:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f5307a87640 1 -- 172.21.15.22:0/4024765458 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5300075980 con 0x7f53000ce3c0 2024-03-29T04:17:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f52f27fc640 1 -- 172.21.15.22:0/4024765458 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f52f4014ba0 con 0x7f53000ce3c0 2024-03-29T04:17:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f52f27fc640 1 --2- 172.21.15.22:0/4024765458 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f52e4067ff0 0x7f52e406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.260+0000 7f52f27fc640 1 -- 172.21.15.22:0/4024765458 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f52f4087000 con 0x7f53000ce3c0 2024-03-29T04:17:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.264+0000 7f5305823640 1 --2- 172.21.15.22:0/4024765458 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f52e4067ff0 0x7f52e406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:50.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.264+0000 7f5305823640 1 --2- 172.21.15.22:0/4024765458 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f52e4067ff0 0x7f52e406a4b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f52f8007fd0 tx=0x7f52f8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:50.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.268+0000 7f52f27fc640 1 -- 172.21.15.22:0/4024765458 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f52f4057500 con 0x7f53000ce3c0 2024-03-29T04:17:50.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.428+0000 7f5307a87640 1 -- 172.21.15.22:0/4024765458 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5300068900 con 0x7f53000ce3c0 2024-03-29T04:17:50.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.428+0000 7f52f27fc640 1 -- 172.21.15.22:0/4024765458 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f52f4056e00 con 0x7f53000ce3c0 2024-03-29T04:17:50.409 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-29T04:17:50.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.432+0000 7f5307a87640 1 -- 172.21.15.22:0/4024765458 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f52e4067ff0 msgr2=0x7f52e406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:50.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.432+0000 7f5307a87640 1 --2- 172.21.15.22:0/4024765458 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f52e4067ff0 0x7f52e406a4b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f52f8007fd0 tx=0x7f52f8009040 comp rx=0 tx=0).stop 2024-03-29T04:17:50.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.432+0000 7f5307a87640 1 -- 172.21.15.22:0/4024765458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53000ce3c0 msgr2=0x7f53000c5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:50.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.432+0000 7f5307a87640 1 --2- 172.21.15.22:0/4024765458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53000ce3c0 0x7f53000c5ca0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f52f4007f20 tx=0x7f52f400e570 comp rx=0 tx=0).stop 2024-03-29T04:17:50.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.432+0000 7f5307a87640 1 -- 172.21.15.22:0/4024765458 shutdown_connections 2024-03-29T04:17:50.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.432+0000 7f5307a87640 1 --2- 172.21.15.22:0/4024765458 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53001531a0 0x7f53000c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.432+0000 7f5307a87640 1 --2- 172.21.15.22:0/4024765458 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f52e4067ff0 0x7f52e406a4b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.432+0000 7f5307a87640 1 --2- 172.21.15.22:0/4024765458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53000ce3c0 0x7f53000c5ca0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.432+0000 7f5307a87640 1 --2- 172.21.15.22:0/4024765458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53000cc8c0 0x7f5300155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.432+0000 7f5307a87640 1 -- 172.21.15.22:0/4024765458 >> 172.21.15.22:0/4024765458 conn(0x7f53000b3c80 msgr2=0x7f530015a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:50.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.432+0000 7f5307a87640 1 -- 172.21.15.22:0/4024765458 shutdown_connections 2024-03-29T04:17:50.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.432+0000 7f5307a87640 1 -- 172.21.15.22:0/4024765458 wait complete. 2024-03-29T04:17:50.427 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:17:50.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.576+0000 7f61fcbef640 1 Processor -- start 2024-03-29T04:17:50.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.576+0000 7f61fcbef640 1 -- start start 2024-03-29T04:17:50.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61fcbef640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61f806cd80 0x7f61f8166fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:50.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61fcbef640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61f806d710 con 0x7f61f815ff40 2024-03-29T04:17:50.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61fcbef640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61f806d9f0 con 0x7f61f813f140 2024-03-29T04:17:50.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61fcbef640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f61f806e320 con 0x7f61f806cd80 2024-03-29T04:17:50.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f5d9b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f61f815ff40 0x7f61f806c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51094/0 (socket says 172.21.15.22:51094) 2024-03-29T04:17:50.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f659c640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f61f813f140 0x7f61f81606d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.22:40862/0 (socket says 172.21.15.22:40862) 2024-03-29T04:17:50.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f5d9b640 1 -- 172.21.15.22:0/1131644476 learned_addr learned my addr 172.21.15.22:0/1131644476 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:50.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f6d9d640 1 --2- 172.21.15.22:0/1131644476 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61f806cd80 0x7f61f8166fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:50.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f559a640 1 -- 172.21.15.22:0/1131644476 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2901473943 0 0) 0x7f61f806d710 con 0x7f61f815ff40 2024-03-29T04:17:50.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f559a640 1 -- 172.21.15.22:0/1131644476 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61e0003530 con 0x7f61f815ff40 2024-03-29T04:17:50.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f559a640 1 -- 172.21.15.22:0/1131644476 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3876096842 0 0) 0x7f61f806d9f0 con 0x7f61f813f140 2024-03-29T04:17:50.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f559a640 1 -- 172.21.15.22:0/1131644476 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61f806d710 con 0x7f61f813f140 2024-03-29T04:17:50.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f559a640 1 -- 172.21.15.22:0/1131644476 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2634560220 0 0) 0x7f61e0003530 con 0x7f61f815ff40 2024-03-29T04:17:50.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f559a640 1 -- 172.21.15.22:0/1131644476 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61f806cd80 msgr2=0x7f61f8166fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:50.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f559a640 1 --2- 172.21.15.22:0/1131644476 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61f806cd80 0x7f61f8166fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f559a640 1 -- 172.21.15.22:0/1131644476 >> v1:172.21.15.151:6789/0 conn(0x7f61f813f140 legacy=0x7f61f81606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:50.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f559a640 1 -- 172.21.15.22:0/1131644476 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f806dcd0 con 0x7f61f815ff40 2024-03-29T04:17:50.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f559a640 1 -- 172.21.15.22:0/1131644476 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f61ec001680 con 0x7f61f815ff40 2024-03-29T04:17:50.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f6d9d640 1 --2- 172.21.15.22:0/1131644476 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61f806cd80 0x7f61f8166fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:17:50.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f559a640 1 -- 172.21.15.22:0/1131644476 >> v1:172.21.15.22:6789/0 conn(0x7f61f815ff40 legacy=0x7f61f806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:50.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f559a640 1 --2- 172.21.15.22:0/1131644476 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61f806cd80 0x7f61e0003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:50.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f559a640 1 --2- 172.21.15.22:0/1131644476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e00040c0 0x7f61e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:50.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f559a640 1 --2- 172.21.15.22:0/1131644476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61e0004d50 0x7f61e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:50.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f559a640 1 -- 172.21.15.22:0/1131644476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f61f806e320 con 0x7f61e0004d50 2024-03-29T04:17:50.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f559a640 1 -- 172.21.15.22:0/1131644476 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f61e000ba80 con 0x7f61e00040c0 2024-03-29T04:17:50.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f559a640 1 -- 172.21.15.22:0/1131644476 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f61e000bc80 con 0x7f61f806cd80 2024-03-29T04:17:50.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f6d9d640 1 --2- 172.21.15.22:0/1131644476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61e0004d50 0x7f61e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:50.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f5d9b640 1 --2- 172.21.15.22:0/1131644476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e00040c0 0x7f61e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:50.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f659c640 1 --2- 172.21.15.22:0/1131644476 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61f806cd80 0x7f61e0003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:50.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f6d9d640 1 -- 172.21.15.22:0/1131644476 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61f806cd80 msgr2=0x7f61e0003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:50.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f6d9d640 1 --2- 172.21.15.22:0/1131644476 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61f806cd80 0x7f61e0003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f6d9d640 1 -- 172.21.15.22:0/1131644476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e00040c0 msgr2=0x7f61e0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:50.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f6d9d640 1 --2- 172.21.15.22:0/1131644476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e00040c0 0x7f61e0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.580+0000 7f61f6d9d640 1 -- 172.21.15.22:0/1131644476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e0003530 con 0x7f61e0004d50 2024-03-29T04:17:50.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61f6d9d640 1 --2- 172.21.15.22:0/1131644476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61e0004d50 0x7f61e00051d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f61f000e320 tx=0x7f61f001cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0781f8b5adff0f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:50.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61f559a640 1 -- 172.21.15.22:0/1131644476 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f0009dc0 con 0x7f61e0004d50 2024-03-29T04:17:50.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61f559a640 1 -- 172.21.15.22:0/1131644476 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61f00a5cc0 con 0x7f61e0004d50 2024-03-29T04:17:50.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61f559a640 1 -- 172.21.15.22:0/1131644476 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f0003ba0 con 0x7f61e0004d50 2024-03-29T04:17:50.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61fcbef640 1 -- 172.21.15.22:0/1131644476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61e0004d50 msgr2=0x7f61e00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:50.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61fcbef640 1 --2- 172.21.15.22:0/1131644476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61e0004d50 0x7f61e00051d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f61f000e320 tx=0x7f61f001cdc0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61f6d9d640 1 -- 172.21.15.22:0/1131644476 reap_dead start 2024-03-29T04:17:50.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61fcbef640 1 -- 172.21.15.22:0/1131644476 shutdown_connections 2024-03-29T04:17:50.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61fcbef640 1 -- 172.21.15.22:0/1131644476 >> 172.21.15.22:0/1131644476 conn(0x7f61f80d00a0 msgr2=0x7f61f80d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:50.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61fcbef640 1 -- 172.21.15.22:0/1131644476 shutdown_connections 2024-03-29T04:17:50.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61fcbef640 1 -- 172.21.15.22:0/1131644476 wait complete. 2024-03-29T04:17:50.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61fcbef640 1 Processor -- start 2024-03-29T04:17:50.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61fcbef640 1 -- start start 2024-03-29T04:17:50.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61fcbef640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61f813de80 0x7f61f81f2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:50.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61fcbef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f81549e0 0x7f61f8156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:50.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61fcbef640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61f81566d0 0x7f61f815c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:50.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61fcbef640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f61f813f520 con 0x7f61f81549e0 2024-03-29T04:17:50.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61fcbef640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f61f8160de0 con 0x7f61f81566d0 2024-03-29T04:17:50.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61fcbef640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f61f813f6a0 con 0x7f61f813de80 2024-03-29T04:17:50.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61f5d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f81549e0 0x7f61f8156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:50.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61f5d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f81549e0 0x7f61f8156190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49412/0 (socket says 172.21.15.22:49412) 2024-03-29T04:17:50.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61f5d9b640 1 -- 172.21.15.22:0/2309275844 learned_addr learned my addr 172.21.15.22:0/2309275844 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:50.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61f659c640 1 --2- 172.21.15.22:0/2309275844 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61f813de80 0x7f61f81f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:50.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61f5d9b640 1 -- 172.21.15.22:0/2309275844 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61f813de80 msgr2=0x7f61f81f2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:50.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61f5d9b640 1 --2- 172.21.15.22:0/2309275844 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61f813de80 0x7f61f81f2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61f6d9d640 1 --2- 172.21.15.22:0/2309275844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61f81566d0 0x7f61f815c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:50.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61f5d9b640 1 -- 172.21.15.22:0/2309275844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61f81566d0 msgr2=0x7f61f815c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:50.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61f5d9b640 1 --2- 172.21.15.22:0/2309275844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61f81566d0 0x7f61f815c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61f5d9b640 1 -- 172.21.15.22:0/2309275844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f815ab10 con 0x7f61f81549e0 2024-03-29T04:17:50.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61f5d9b640 1 --2- 172.21.15.22:0/2309275844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f81549e0 0x7f61f8156190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f61ec011980 tx=0x7f61ec011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:50.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61e6ffd640 1 -- 172.21.15.22:0/2309275844 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ec010040 con 0x7f61f81549e0 2024-03-29T04:17:50.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61e6ffd640 1 -- 172.21.15.22:0/2309275844 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61ec01a070 con 0x7f61f81549e0 2024-03-29T04:17:50.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61fcbef640 1 -- 172.21.15.22:0/2309275844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f8154e70 con 0x7f61f81549e0 2024-03-29T04:17:50.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61e6ffd640 1 -- 172.21.15.22:0/2309275844 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ec00fe60 con 0x7f61f81549e0 2024-03-29T04:17:50.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.584+0000 7f61fcbef640 1 -- 172.21.15.22:0/2309275844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f8155430 con 0x7f61f81549e0 2024-03-29T04:17:50.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.588+0000 7f61fcbef640 1 -- 172.21.15.22:0/2309275844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61f8068900 con 0x7f61f81549e0 2024-03-29T04:17:50.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.588+0000 7f61e6ffd640 1 -- 172.21.15.22:0/2309275844 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f61ec01f460 con 0x7f61f81549e0 2024-03-29T04:17:50.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.588+0000 7f61e6ffd640 1 --2- 172.21.15.22:0/2309275844 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f61d8067fc0 0x7f61d806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:50.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.588+0000 7f61e6ffd640 1 -- 172.21.15.22:0/2309275844 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f61ec089bd0 con 0x7f61f81549e0 2024-03-29T04:17:50.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.588+0000 7f61f659c640 1 --2- 172.21.15.22:0/2309275844 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f61d8067fc0 0x7f61d806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:50.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.588+0000 7f61f659c640 1 --2- 172.21.15.22:0/2309275844 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f61d8067fc0 0x7f61d806a480 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f61e8007b20 tx=0x7f61e8007bd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:50.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.592+0000 7f61e6ffd640 1 -- 172.21.15.22:0/2309275844 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f61ec05a040 con 0x7f61f81549e0 2024-03-29T04:17:50.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.748+0000 7f61fcbef640 1 -- 172.21.15.22:0/2309275844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f61f806ba50 con 0x7f61f81549e0 2024-03-29T04:17:50.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.748+0000 7f61e6ffd640 1 -- 172.21.15.22:0/2309275844 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f61ec059940 con 0x7f61f81549e0 2024-03-29T04:17:50.730 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:17:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.752+0000 7f61fcbef640 1 -- 172.21.15.22:0/2309275844 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f61d8067fc0 msgr2=0x7f61d806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.752+0000 7f61fcbef640 1 --2- 172.21.15.22:0/2309275844 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f61d8067fc0 0x7f61d806a480 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f61e8007b20 tx=0x7f61e8007bd0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.752+0000 7f61fcbef640 1 -- 172.21.15.22:0/2309275844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f81549e0 msgr2=0x7f61f8156190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.752+0000 7f61fcbef640 1 --2- 172.21.15.22:0/2309275844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f81549e0 0x7f61f8156190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f61ec011980 tx=0x7f61ec011e40 comp rx=0 tx=0).stop 2024-03-29T04:17:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.752+0000 7f61fcbef640 1 -- 172.21.15.22:0/2309275844 shutdown_connections 2024-03-29T04:17:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.752+0000 7f61fcbef640 1 --2- 172.21.15.22:0/2309275844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61f81566d0 0x7f61f815c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.752+0000 7f61fcbef640 1 --2- 172.21.15.22:0/2309275844 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f61d8067fc0 0x7f61d806a480 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.752+0000 7f61fcbef640 1 --2- 172.21.15.22:0/2309275844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f81549e0 0x7f61f8156190 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.752+0000 7f61fcbef640 1 --2- 172.21.15.22:0/2309275844 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61f813de80 0x7f61f81f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.752+0000 7f61fcbef640 1 -- 172.21.15.22:0/2309275844 >> 172.21.15.22:0/2309275844 conn(0x7f61f80d00a0 msgr2=0x7f61f806c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.752+0000 7f61fcbef640 1 -- 172.21.15.22:0/2309275844 shutdown_connections 2024-03-29T04:17:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.752+0000 7f61fcbef640 1 -- 172.21.15.22:0/2309275844 wait complete. 2024-03-29T04:17:50.749 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:17:50.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.888+0000 7f57de3a9640 1 Processor -- start 2024-03-29T04:17:50.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.888+0000 7f57de3a9640 1 -- start start 2024-03-29T04:17:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.888+0000 7f57de3a9640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f57d80b41f0 0x7f57d80cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.888+0000 7f57de3a9640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f57d806ca50 0x7f57d80ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.888+0000 7f57de3a9640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57d80ccef0 con 0x7f57d806c670 2024-03-29T04:17:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.888+0000 7f57de3a9640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f57d80b34c0 con 0x7f57d80b41f0 2024-03-29T04:17:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.888+0000 7f57de3a9640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f57d80b3710 con 0x7f57d806ca50 2024-03-29T04:17:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.892+0000 7f57cffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f57d80b41f0 0x7f57d80cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.892+0000 7f57d7fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f57d806c670 0x7f57d806e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51098/0 (socket says 172.21.15.22:51098) 2024-03-29T04:17:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.892+0000 7f57d7fff640 1 -- 172.21.15.22:0/235423507 learned_addr learned my addr 172.21.15.22:0/235423507 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.892+0000 7f57dc946640 1 --2- 172.21.15.22:0/235423507 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f57d806ca50 0x7f57d80ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.892+0000 7f57cffff640 1 -- 172.21.15.22:0/235423507 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f57d806ca50 msgr2=0x7f57d80ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.892+0000 7f57cffff640 1 --2- 172.21.15.22:0/235423507 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f57d806ca50 0x7f57d80ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.892+0000 7f57d77fe640 1 -- 172.21.15.22:0/235423507 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 894121577 0 0) 0x7f57d80ccef0 con 0x7f57d806c670 2024-03-29T04:17:50.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.892+0000 7f57cffff640 1 -- 172.21.15.22:0/235423507 >> v1:172.21.15.22:6789/0 conn(0x7f57d806c670 legacy=0x7f57d806e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:50.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.892+0000 7f57cffff640 1 -- 172.21.15.22:0/235423507 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d80b39f0 con 0x7f57d80b41f0 2024-03-29T04:17:50.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.892+0000 7f57cffff640 1 --2- 172.21.15.22:0/235423507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57d80b41f0 0x7f57d80cc910 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f57c80097c0 tx=0x7f57c801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1698f12ccd42db server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:50.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.892+0000 7f57d77fe640 1 -- 172.21.15.22:0/235423507 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c80b1070 con 0x7f57d80b41f0 2024-03-29T04:17:50.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.892+0000 7f57d77fe640 1 -- 172.21.15.22:0/235423507 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c80ac410 con 0x7f57d80b41f0 2024-03-29T04:17:50.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.892+0000 7f57d77fe640 1 -- 172.21.15.22:0/235423507 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c80ac7e0 con 0x7f57d80b41f0 2024-03-29T04:17:50.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.892+0000 7f57de3a9640 1 -- 172.21.15.22:0/235423507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57d80b41f0 msgr2=0x7f57d80cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.892+0000 7f57de3a9640 1 --2- 172.21.15.22:0/235423507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57d80b41f0 0x7f57d80cc910 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f57c80097c0 tx=0x7f57c801c4a0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.892+0000 7f57de3a9640 1 -- 172.21.15.22:0/235423507 shutdown_connections 2024-03-29T04:17:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.892+0000 7f57de3a9640 1 --2- 172.21.15.22:0/235423507 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f57d806ca50 0x7f57d80ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.892+0000 7f57de3a9640 1 --2- 172.21.15.22:0/235423507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57d80b41f0 0x7f57d80cc910 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.892+0000 7f57de3a9640 1 -- 172.21.15.22:0/235423507 >> 172.21.15.22:0/235423507 conn(0x7f57d80cfee0 msgr2=0x7f57d80d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.892+0000 7f57de3a9640 1 -- 172.21.15.22:0/235423507 shutdown_connections 2024-03-29T04:17:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.892+0000 7f57de3a9640 1 -- 172.21.15.22:0/235423507 wait complete. 2024-03-29T04:17:50.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57de3a9640 1 Processor -- start 2024-03-29T04:17:50.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57de3a9640 1 -- start start 2024-03-29T04:17:50.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57de3a9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57d806c670 0x7f57d8165280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57de3a9640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f57d806ca50 0x7f57d815f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57de3a9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57d80b41f0 0x7f57d813d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57de3a9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f57d8169070 con 0x7f57d80b41f0 2024-03-29T04:17:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57de3a9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f57d8168ef0 con 0x7f57d806c670 2024-03-29T04:17:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57de3a9640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f57d81691f0 con 0x7f57d806ca50 2024-03-29T04:17:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57dc946640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57d80b41f0 0x7f57d813d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57dc946640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57d80b41f0 0x7f57d813d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49436/0 (socket says 172.21.15.22:49436) 2024-03-29T04:17:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57dc946640 1 -- 172.21.15.22:0/1223598412 learned_addr learned my addr 172.21.15.22:0/1223598412 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57cffff640 1 --2- 172.21.15.22:0/1223598412 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f57d806ca50 0x7f57d815f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57d7fff640 1 --2- 172.21.15.22:0/1223598412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57d806c670 0x7f57d8165280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57dc946640 1 -- 172.21.15.22:0/1223598412 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f57d806ca50 msgr2=0x7f57d815f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57dc946640 1 --2- 172.21.15.22:0/1223598412 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f57d806ca50 0x7f57d815f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57dc946640 1 -- 172.21.15.22:0/1223598412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57d806c670 msgr2=0x7f57d8165280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57dc946640 1 --2- 172.21.15.22:0/1223598412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57d806c670 0x7f57d8165280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57dc946640 1 -- 172.21.15.22:0/1223598412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d815f690 con 0x7f57d80b41f0 2024-03-29T04:17:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57dc946640 1 --2- 172.21.15.22:0/1223598412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57d80b41f0 0x7f57d813d440 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f57d000cc30 tx=0x7f57d000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57d57fa640 1 -- 172.21.15.22:0/1223598412 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d000ce40 con 0x7f57d80b41f0 2024-03-29T04:17:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57d57fa640 1 -- 172.21.15.22:0/1223598412 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57d00039a0 con 0x7f57d80b41f0 2024-03-29T04:17:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57de3a9640 1 -- 172.21.15.22:0/1223598412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d8165980 con 0x7f57d80b41f0 2024-03-29T04:17:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57d57fa640 1 -- 172.21.15.22:0/1223598412 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0003d50 con 0x7f57d80b41f0 2024-03-29T04:17:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57de3a9640 1 -- 172.21.15.22:0/1223598412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d8165f40 con 0x7f57d80b41f0 2024-03-29T04:17:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.896+0000 7f57de3a9640 1 -- 172.21.15.22:0/1223598412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d8068890 con 0x7f57d80b41f0 2024-03-29T04:17:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.900+0000 7f57d57fa640 1 -- 172.21.15.22:0/1223598412 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f57d0003fb0 con 0x7f57d80b41f0 2024-03-29T04:17:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.900+0000 7f57d57fa640 1 --2- 172.21.15.22:0/1223598412 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f57b8067fc0 0x7f57b806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.900+0000 7f57d57fa640 1 -- 172.21.15.22:0/1223598412 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f57d0086d40 con 0x7f57d80b41f0 2024-03-29T04:17:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.900+0000 7f57d7fff640 1 --2- 172.21.15.22:0/1223598412 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f57b8067fc0 0x7f57b806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.900+0000 7f57d7fff640 1 --2- 172.21.15.22:0/1223598412 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f57b8067fc0 0x7f57b806a480 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f57c0000f70 tx=0x7f57c0007c10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:50.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:50.904+0000 7f57d57fa640 1 -- 172.21.15.22:0/1223598412 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f57d0014030 con 0x7f57d80b41f0 2024-03-29T04:17:51.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.068+0000 7f57de3a9640 1 -- 172.21.15.22:0/1223598412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f57d80b32a0 con 0x7f57d80b41f0 2024-03-29T04:17:51.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.068+0000 7f57d57fa640 1 -- 172.21.15.22:0/1223598412 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f57d0057120 con 0x7f57d80b41f0 2024-03-29T04:17:51.051 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-03-29T04:17:51.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.072+0000 7f57de3a9640 1 -- 172.21.15.22:0/1223598412 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f57b8067fc0 msgr2=0x7f57b806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:51.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.072+0000 7f57de3a9640 1 --2- 172.21.15.22:0/1223598412 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f57b8067fc0 0x7f57b806a480 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f57c0000f70 tx=0x7f57c0007c10 comp rx=0 tx=0).stop 2024-03-29T04:17:51.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.072+0000 7f57de3a9640 1 -- 172.21.15.22:0/1223598412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57d80b41f0 msgr2=0x7f57d813d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:51.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.072+0000 7f57de3a9640 1 --2- 172.21.15.22:0/1223598412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57d80b41f0 0x7f57d813d440 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f57d000cc30 tx=0x7f57d000a5c0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.072+0000 7f57de3a9640 1 -- 172.21.15.22:0/1223598412 shutdown_connections 2024-03-29T04:17:51.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.072+0000 7f57de3a9640 1 --2- 172.21.15.22:0/1223598412 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f57b8067fc0 0x7f57b806a480 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.072+0000 7f57de3a9640 1 --2- 172.21.15.22:0/1223598412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57d80b41f0 0x7f57d813d440 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.072+0000 7f57de3a9640 1 --2- 172.21.15.22:0/1223598412 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f57d806ca50 0x7f57d815f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.072+0000 7f57de3a9640 1 --2- 172.21.15.22:0/1223598412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57d806c670 0x7f57d8165280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.072+0000 7f57de3a9640 1 -- 172.21.15.22:0/1223598412 >> 172.21.15.22:0/1223598412 conn(0x7f57d80cfee0 msgr2=0x7f57d813f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:51.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.072+0000 7f57de3a9640 1 -- 172.21.15.22:0/1223598412 shutdown_connections 2024-03-29T04:17:51.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.072+0000 7f57de3a9640 1 -- 172.21.15.22:0/1223598412 wait complete. 2024-03-29T04:17:51.069 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi022.front.sepia.ceph.com... 2024-03-29T04:17:51.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:17:51.069 DEBUG:teuthology.orchestra.run.smithi022:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:17:51.089 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi022.stderr:ceph-fuse[33235]: fuse finished with error 0 and tester_r 0 2024-03-29T04:17:51.090 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:17:51.152 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-03-29T04:17:51.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:17:51.153 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-29T04:17:51.162 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:17:51.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.340+0000 7f6c1fb39640 1 Processor -- start 2024-03-29T04:17:51.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.340+0000 7f6c1fb39640 1 -- start start 2024-03-29T04:17:51.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.340+0000 7f6c1fb39640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6c18144270 0x7f6c18144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:51.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.340+0000 7f6c1fb39640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6c180b2ef0 0x7f6c1815f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:51.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.340+0000 7f6c1fb39640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c180b33f0 con 0x7f6c180b4490 2024-03-29T04:17:51.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.340+0000 7f6c1fb39640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f6c18144fa0 con 0x7f6c180b2ef0 2024-03-29T04:17:51.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.340+0000 7f6c1fb39640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f6c1815fce0 con 0x7f6c18144270 2024-03-29T04:17:51.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.340+0000 7f6c1d0d4640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6c180b2ef0 0x7f6c1815f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:51.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.340+0000 7f6c1e0d6640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f6c180b4490 0x7f6c1806c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51104/0 (socket says 172.21.15.22:51104) 2024-03-29T04:17:51.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.340+0000 7f6c1e0d6640 1 -- 172.21.15.22:0/510383071 learned_addr learned my addr 172.21.15.22:0/510383071 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:51.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.340+0000 7f6c1d8d5640 1 --2- 172.21.15.22:0/510383071 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6c18144270 0x7f6c18144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:51.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.340+0000 7f6c1d0d4640 1 -- 172.21.15.22:0/510383071 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6c18144270 msgr2=0x7f6c18144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:51.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.340+0000 7f6c1d0d4640 1 --2- 172.21.15.22:0/510383071 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6c18144270 0x7f6c18144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.340+0000 7f6c1c8d3640 1 -- 172.21.15.22:0/510383071 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4210529809 0 0) 0x7f6c180b33f0 con 0x7f6c180b4490 2024-03-29T04:17:51.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.340+0000 7f6c1d0d4640 1 -- 172.21.15.22:0/510383071 >> v1:172.21.15.22:6789/0 conn(0x7f6c180b4490 legacy=0x7f6c1806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:51.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.340+0000 7f6c1d0d4640 1 -- 172.21.15.22:0/510383071 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c180b36d0 con 0x7f6c180b2ef0 2024-03-29T04:17:51.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1d0d4640 1 --2- 172.21.15.22:0/510383071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c180b2ef0 0x7f6c1815f7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6c080097c0 tx=0x7f6c0801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9081768805163b15 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:51.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1c8d3640 1 -- 172.21.15.22:0/510383071 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c080a7070 con 0x7f6c180b2ef0 2024-03-29T04:17:51.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1c8d3640 1 -- 172.21.15.22:0/510383071 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c080b0070 con 0x7f6c180b2ef0 2024-03-29T04:17:51.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1c8d3640 1 -- 172.21.15.22:0/510383071 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c080ab890 con 0x7f6c180b2ef0 2024-03-29T04:17:51.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1fb39640 1 -- 172.21.15.22:0/510383071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c180b2ef0 msgr2=0x7f6c1815f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:51.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1fb39640 1 --2- 172.21.15.22:0/510383071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c180b2ef0 0x7f6c1815f7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6c080097c0 tx=0x7f6c0801c470 comp rx=0 tx=0).stop 2024-03-29T04:17:51.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1fb39640 1 -- 172.21.15.22:0/510383071 shutdown_connections 2024-03-29T04:17:51.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1fb39640 1 --2- 172.21.15.22:0/510383071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c180b2ef0 0x7f6c1815f7a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1fb39640 1 --2- 172.21.15.22:0/510383071 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6c18144270 0x7f6c18144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1fb39640 1 -- 172.21.15.22:0/510383071 >> 172.21.15.22:0/510383071 conn(0x7f6c180cfee0 msgr2=0x7f6c180d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:51.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1fb39640 1 -- 172.21.15.22:0/510383071 shutdown_connections 2024-03-29T04:17:51.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1fb39640 1 -- 172.21.15.22:0/510383071 wait complete. 2024-03-29T04:17:51.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1fb39640 1 Processor -- start 2024-03-29T04:17:51.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1fb39640 1 -- start start 2024-03-29T04:17:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1fb39640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6c180b2ef0 0x7f6c18156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1fb39640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c180b4490 0x7f6c18140480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1fb39640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c18144270 0x7f6c1813e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1fb39640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6c180cdd10 con 0x7f6c180b4490 2024-03-29T04:17:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1fb39640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1806cf90 con 0x7f6c18144270 2024-03-29T04:17:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1fb39640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f6c180cde90 con 0x7f6c180b2ef0 2024-03-29T04:17:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1d0d4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c180b4490 0x7f6c18140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1d0d4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c180b4490 0x7f6c18140480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49460/0 (socket says 172.21.15.22:49460) 2024-03-29T04:17:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1d0d4640 1 -- 172.21.15.22:0/536424880 learned_addr learned my addr 172.21.15.22:0/536424880 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1e0d6640 1 --2- 172.21.15.22:0/536424880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c18144270 0x7f6c1813e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1d8d5640 1 --2- 172.21.15.22:0/536424880 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6c180b2ef0 0x7f6c18156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1d0d4640 1 -- 172.21.15.22:0/536424880 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6c180b2ef0 msgr2=0x7f6c18156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1d0d4640 1 --2- 172.21.15.22:0/536424880 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6c180b2ef0 0x7f6c18156d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1d0d4640 1 -- 172.21.15.22:0/536424880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c18144270 msgr2=0x7f6c1813e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1d0d4640 1 --2- 172.21.15.22:0/536424880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c18144270 0x7f6c1813e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1d0d4640 1 -- 172.21.15.22:0/536424880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c18071650 con 0x7f6c180b4490 2024-03-29T04:17:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.344+0000 7f6c1e0d6640 1 --2- 172.21.15.22:0/536424880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c18144270 0x7f6c1813e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.348+0000 7f6c1d0d4640 1 --2- 172.21.15.22:0/536424880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c180b4490 0x7f6c18140480 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6c0801a0a0 tx=0x7f6c0801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.348+0000 7f6c0e7fc640 1 -- 172.21.15.22:0/536424880 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c080bf610 con 0x7f6c180b4490 2024-03-29T04:17:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.348+0000 7f6c0e7fc640 1 -- 172.21.15.22:0/536424880 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c080bf7d0 con 0x7f6c180b4490 2024-03-29T04:17:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.348+0000 7f6c1fb39640 1 -- 172.21.15.22:0/536424880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c1813f3e0 con 0x7f6c180b4490 2024-03-29T04:17:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.348+0000 7f6c0e7fc640 1 -- 172.21.15.22:0/536424880 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c080bfbc0 con 0x7f6c180b4490 2024-03-29T04:17:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.348+0000 7f6c1fb39640 1 -- 172.21.15.22:0/536424880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c1813f9a0 con 0x7f6c180b4490 2024-03-29T04:17:51.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.348+0000 7f6c1fb39640 1 -- 172.21.15.22:0/536424880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c18068890 con 0x7f6c180b4490 2024-03-29T04:17:51.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.348+0000 7f6c0e7fc640 1 -- 172.21.15.22:0/536424880 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f6c080bd020 con 0x7f6c180b4490 2024-03-29T04:17:51.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.348+0000 7f6c0e7fc640 1 --2- 172.21.15.22:0/536424880 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6bfc067fc0 0x7f6bfc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:51.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.348+0000 7f6c0e7fc640 1 -- 172.21.15.22:0/536424880 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f6c08120e90 con 0x7f6c180b4490 2024-03-29T04:17:51.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.348+0000 7f6c1d8d5640 1 --2- 172.21.15.22:0/536424880 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6bfc067fc0 0x7f6bfc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:51.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.348+0000 7f6c1d8d5640 1 --2- 172.21.15.22:0/536424880 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6bfc067fc0 0x7f6bfc06a480 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6c1806e2f0 tx=0x7f6c14008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:51.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.352+0000 7f6c0e7fc640 1 -- 172.21.15.22:0/536424880 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f6c08016090 con 0x7f6c180b4490 2024-03-29T04:17:51.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.516+0000 7f6c1fb39640 1 -- 172.21.15.22:0/536424880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c180722d0 con 0x7f6c180b4490 2024-03-29T04:17:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.516+0000 7f6c0e7fc640 1 -- 172.21.15.22:0/536424880 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18514 (secure 0 0 0) 0x7f6c080f1300 con 0x7f6c180b4490 2024-03-29T04:17:51.497 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:51.498 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:17:42.645632+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:17:51.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.516+0000 7f6c1fb39640 1 -- 172.21.15.22:0/536424880 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6bfc067fc0 msgr2=0x7f6bfc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:51.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.516+0000 7f6c1fb39640 1 --2- 172.21.15.22:0/536424880 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6bfc067fc0 0x7f6bfc06a480 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6c1806e2f0 tx=0x7f6c14008040 comp rx=0 tx=0).stop 2024-03-29T04:17:51.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.516+0000 7f6c1fb39640 1 -- 172.21.15.22:0/536424880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c180b4490 msgr2=0x7f6c18140480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:51.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.516+0000 7f6c1fb39640 1 --2- 172.21.15.22:0/536424880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c180b4490 0x7f6c18140480 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6c0801a0a0 tx=0x7f6c0801acb0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.520+0000 7f6c1fb39640 1 -- 172.21.15.22:0/536424880 shutdown_connections 2024-03-29T04:17:51.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.520+0000 7f6c1fb39640 1 --2- 172.21.15.22:0/536424880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c18144270 0x7f6c1813e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.520+0000 7f6c1fb39640 1 --2- 172.21.15.22:0/536424880 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6bfc067fc0 0x7f6bfc06a480 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.520+0000 7f6c1fb39640 1 --2- 172.21.15.22:0/536424880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c180b4490 0x7f6c18140480 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.520+0000 7f6c1fb39640 1 --2- 172.21.15.22:0/536424880 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6c180b2ef0 0x7f6c18156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.520+0000 7f6c1fb39640 1 -- 172.21.15.22:0/536424880 >> 172.21.15.22:0/536424880 conn(0x7f6c180cfee0 msgr2=0x7f6c180730f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:51.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.520+0000 7f6c1fb39640 1 -- 172.21.15.22:0/536424880 shutdown_connections 2024-03-29T04:17:51.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.520+0000 7f6c1fb39640 1 -- 172.21.15.22:0/536424880 wait complete. 2024-03-29T04:17:51.516 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:17:51.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.656+0000 7f1389986640 1 Processor -- start 2024-03-29T04:17:51.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.660+0000 7f1389986640 1 -- start start 2024-03-29T04:17:51.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.660+0000 7f1389986640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f138415c230 0x7f138406f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:51.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.660+0000 7f1389986640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f1384160910 0x7f13840cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:51.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.660+0000 7f1389986640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f138415c610 con 0x7f138415fda0 2024-03-29T04:17:51.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.660+0000 7f1389986640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f13840b4fa0 con 0x7f1384160910 2024-03-29T04:17:51.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.660+0000 7f1389986640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f13840cf4f0 con 0x7f138415c230 2024-03-29T04:17:51.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.660+0000 7f1382ffd640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f138415c230 0x7f138406f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:51.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.660+0000 7f13837fe640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f1384160910 0x7f13840cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:51.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.660+0000 7f13837fe640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f1384160910 0x7f13840cfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:44374/0 (socket says 172.21.15.22:44374) 2024-03-29T04:17:51.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.660+0000 7f13837fe640 1 -- 172.21.15.22:0/3109654126 learned_addr learned my addr 172.21.15.22:0/3109654126 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:51.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.660+0000 7f13837fe640 1 -- 172.21.15.22:0/3109654126 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f138415c230 msgr2=0x7f138406f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:51.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.660+0000 7f13837fe640 1 --2- 172.21.15.22:0/3109654126 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f138415c230 0x7f138406f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.660+0000 7f13837fe640 1 -- 172.21.15.22:0/3109654126 >> v1:172.21.15.151:6789/0 conn(0x7f138415fda0 legacy=0x7f1384160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:51.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.660+0000 7f13837fe640 1 -- 172.21.15.22:0/3109654126 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13840cc9f0 con 0x7f1384160910 2024-03-29T04:17:51.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.660+0000 7f1381ffb640 1 -- 172.21.15.22:0/3109654126 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2078903512 0 0) 0x7f138415c610 con 0x7f138415fda0 2024-03-29T04:17:51.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f13837fe640 1 --2- 172.21.15.22:0/3109654126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1384160910 0x7f13840cfd20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f137c0098a0 tx=0x7f137c01c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=497d0b9ebb18c94b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:51.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1381ffb640 1 -- 172.21.15.22:0/3109654126 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c0b1070 con 0x7f1384160910 2024-03-29T04:17:51.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1381ffb640 1 -- 172.21.15.22:0/3109654126 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f137c0ac620 con 0x7f1384160910 2024-03-29T04:17:51.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1381ffb640 1 -- 172.21.15.22:0/3109654126 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c0ac9f0 con 0x7f1384160910 2024-03-29T04:17:51.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1389986640 1 -- 172.21.15.22:0/3109654126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1384160910 msgr2=0x7f13840cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:51.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1389986640 1 --2- 172.21.15.22:0/3109654126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1384160910 0x7f13840cfd20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f137c0098a0 tx=0x7f137c01c4b0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1389986640 1 -- 172.21.15.22:0/3109654126 shutdown_connections 2024-03-29T04:17:51.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1389986640 1 --2- 172.21.15.22:0/3109654126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1384160910 0x7f13840cfd20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1389986640 1 --2- 172.21.15.22:0/3109654126 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f138415c230 0x7f138406f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1389986640 1 -- 172.21.15.22:0/3109654126 >> 172.21.15.22:0/3109654126 conn(0x7f13840b3c80 msgr2=0x7f13840b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:51.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1389986640 1 -- 172.21.15.22:0/3109654126 shutdown_connections 2024-03-29T04:17:51.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1389986640 1 -- 172.21.15.22:0/3109654126 wait complete. 2024-03-29T04:17:51.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1389986640 1 Processor -- start 2024-03-29T04:17:51.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1389986640 1 -- start start 2024-03-29T04:17:51.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1389986640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f138415c230 0x7f1384154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:51.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1389986640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f138415fda0 0x7f1384141310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:51.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1389986640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1384160910 0x7f13841574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:51.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1389986640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f13840d0a70 con 0x7f138415c230 2024-03-29T04:17:51.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1389986640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13840cc7a0 con 0x7f1384160910 2024-03-29T04:17:51.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1389986640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f13840d0bf0 con 0x7f138415fda0 2024-03-29T04:17:51.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1382ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f138415c230 0x7f1384154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:51.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1382ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f138415c230 0x7f1384154690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49462/0 (socket says 172.21.15.22:49462) 2024-03-29T04:17:51.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1382ffd640 1 -- 172.21.15.22:0/3225750646 learned_addr learned my addr 172.21.15.22:0/3225750646 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:51.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f13827fc640 1 --2- 172.21.15.22:0/3225750646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f138415fda0 0x7f1384141310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:51.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f13837fe640 1 --2- 172.21.15.22:0/3225750646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1384160910 0x7f13841574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:51.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1382ffd640 1 -- 172.21.15.22:0/3225750646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f138415fda0 msgr2=0x7f1384141310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:51.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1382ffd640 1 --2- 172.21.15.22:0/3225750646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f138415fda0 0x7f1384141310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1382ffd640 1 -- 172.21.15.22:0/3225750646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1384160910 msgr2=0x7f13841574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:51.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1382ffd640 1 --2- 172.21.15.22:0/3225750646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1384160910 0x7f13841574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f1382ffd640 1 -- 172.21.15.22:0/3225750646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13840cb560 con 0x7f138415c230 2024-03-29T04:17:51.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f13837fe640 1 --2- 172.21.15.22:0/3225750646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1384160910 0x7f13841574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:51.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.664+0000 7f13827fc640 1 --2- 172.21.15.22:0/3225750646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f138415fda0 0x7f1384141310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:51.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.668+0000 7f1382ffd640 1 --2- 172.21.15.22:0/3225750646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f138415c230 0x7f1384154690 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f137400e8a0 tx=0x7f137400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:51.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.668+0000 7f136bfff640 1 -- 172.21.15.22:0/3225750646 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374017070 con 0x7f138415c230 2024-03-29T04:17:51.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.668+0000 7f136bfff640 1 -- 172.21.15.22:0/3225750646 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f137400c940 con 0x7f138415c230 2024-03-29T04:17:51.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.668+0000 7f1389986640 1 -- 172.21.15.22:0/3225750646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13840cb840 con 0x7f138415c230 2024-03-29T04:17:51.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.668+0000 7f136bfff640 1 -- 172.21.15.22:0/3225750646 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137400ccf0 con 0x7f138415c230 2024-03-29T04:17:51.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.668+0000 7f1389986640 1 -- 172.21.15.22:0/3225750646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13840cbe00 con 0x7f138415c230 2024-03-29T04:17:51.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.668+0000 7f1389986640 1 -- 172.21.15.22:0/3225750646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1384068890 con 0x7f138415c230 2024-03-29T04:17:51.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.668+0000 7f136bfff640 1 -- 172.21.15.22:0/3225750646 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f137401a070 con 0x7f138415c230 2024-03-29T04:17:51.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.668+0000 7f136bfff640 1 --2- 172.21.15.22:0/3225750646 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1370067fc0 0x7f137006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:51.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.668+0000 7f136bfff640 1 -- 172.21.15.22:0/3225750646 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1374086af0 con 0x7f138415c230 2024-03-29T04:17:51.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.668+0000 7f13827fc640 1 --2- 172.21.15.22:0/3225750646 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1370067fc0 0x7f137006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:51.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.668+0000 7f13827fc640 1 --2- 172.21.15.22:0/3225750646 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1370067fc0 0x7f137006a480 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f13840b2c40 tx=0x7f1378009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:51.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.672+0000 7f136bfff640 1 -- 172.21.15.22:0/3225750646 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1374056f70 con 0x7f138415c230 2024-03-29T04:17:51.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.832+0000 7f1389986640 1 -- 172.21.15.22:0/3225750646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1384070230 con 0x7f138415c230 2024-03-29T04:17:51.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.836+0000 7f136bfff640 1 -- 172.21.15.22:0/3225750646 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18514 (secure 0 0 0) 0x7f1374056870 con 0x7f138415c230 2024-03-29T04:17:51.817 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:51.818 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:17:42.645632+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:17:51.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.836+0000 7f1389986640 1 -- 172.21.15.22:0/3225750646 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1370067fc0 msgr2=0x7f137006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:51.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.836+0000 7f1389986640 1 --2- 172.21.15.22:0/3225750646 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1370067fc0 0x7f137006a480 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f13840b2c40 tx=0x7f1378009040 comp rx=0 tx=0).stop 2024-03-29T04:17:51.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.836+0000 7f1389986640 1 -- 172.21.15.22:0/3225750646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f138415c230 msgr2=0x7f1384154690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:51.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.836+0000 7f1389986640 1 --2- 172.21.15.22:0/3225750646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f138415c230 0x7f1384154690 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f137400e8a0 tx=0x7f137400eca0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.840+0000 7f1389986640 1 -- 172.21.15.22:0/3225750646 shutdown_connections 2024-03-29T04:17:51.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.840+0000 7f1389986640 1 --2- 172.21.15.22:0/3225750646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1384160910 0x7f13841574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.840+0000 7f1389986640 1 --2- 172.21.15.22:0/3225750646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f138415fda0 0x7f1384141310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.840+0000 7f1389986640 1 --2- 172.21.15.22:0/3225750646 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1370067fc0 0x7f137006a480 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.840+0000 7f1389986640 1 --2- 172.21.15.22:0/3225750646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f138415c230 0x7f1384154690 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.840+0000 7f1389986640 1 -- 172.21.15.22:0/3225750646 >> 172.21.15.22:0/3225750646 conn(0x7f13840b3c80 msgr2=0x7f13840b3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:51.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.840+0000 7f1389986640 1 -- 172.21.15.22:0/3225750646 shutdown_connections 2024-03-29T04:17:51.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.840+0000 7f1389986640 1 -- 172.21.15.22:0/3225750646 wait complete. 2024-03-29T04:17:51.836 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-29T04:17:51.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.972+0000 7efdd185c640 1 Processor -- start 2024-03-29T04:17:51.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.972+0000 7efdd185c640 1 -- start start 2024-03-29T04:17:51.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.972+0000 7efdd185c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efdcc0b2e80 0x7efdcc15f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:51.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.972+0000 7efdd185c640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdcc162830 con 0x7efdcc155160 2024-03-29T04:17:51.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.972+0000 7efdd185c640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdcc162b10 con 0x7efdcc06cd80 2024-03-29T04:17:51.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.972+0000 7efdd185c640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7efdcc162df0 con 0x7efdcc0b2e80 2024-03-29T04:17:51.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.972+0000 7efdcaffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7efdcc155160 0x7efdcc06c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51112/0 (socket says 172.21.15.22:51112) 2024-03-29T04:17:51.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.972+0000 7efdcaffd640 1 -- 172.21.15.22:0/1213681454 learned_addr learned my addr 172.21.15.22:0/1213681454 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:51.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.972+0000 7efdcb7fe640 1 --2- 172.21.15.22:0/1213681454 >> v2:172.21.15.22:3300/0 conn(0x7efdcc0b2e80 0x7efdcc15f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:51.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.972+0000 7efdc9ffb640 1 -- 172.21.15.22:0/1213681454 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 879717247 0 0) 0x7efdcc162830 con 0x7efdcc155160 2024-03-29T04:17:51.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.972+0000 7efdc9ffb640 1 -- 172.21.15.22:0/1213681454 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdb8003530 con 0x7efdcc155160 2024-03-29T04:17:51.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.972+0000 7efdcb7fe640 1 -- 172.21.15.22:0/1213681454 >> v1:172.21.15.151:6789/0 conn(0x7efdcc06cd80 legacy=0x7efdcc071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:51.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.972+0000 7efdcb7fe640 1 -- 172.21.15.22:0/1213681454 >> v1:172.21.15.22:6789/0 conn(0x7efdcc155160 legacy=0x7efdcc06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:51.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.972+0000 7efdcb7fe640 1 -- 172.21.15.22:0/1213681454 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdcc0719e0 con 0x7efdcc0b2e80 2024-03-29T04:17:51.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdcb7fe640 1 --2- 172.21.15.22:0/1213681454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc0b2e80 0x7efdcc15f5e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009d90 tx=0x7efdc401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410b1839fa9bb66f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:51.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdc9ffb640 1 -- 172.21.15.22:0/1213681454 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc40b2070 con 0x7efdcc0b2e80 2024-03-29T04:17:51.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdc9ffb640 1 -- 172.21.15.22:0/1213681454 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdc40a5920 con 0x7efdcc0b2e80 2024-03-29T04:17:51.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdc9ffb640 1 -- 172.21.15.22:0/1213681454 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc40a5cd0 con 0x7efdcc0b2e80 2024-03-29T04:17:51.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdd185c640 1 -- 172.21.15.22:0/1213681454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc0b2e80 msgr2=0x7efdcc15f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:51.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdd185c640 1 --2- 172.21.15.22:0/1213681454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc0b2e80 0x7efdcc15f5e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009d90 tx=0x7efdc401c970 comp rx=0 tx=0).stop 2024-03-29T04:17:51.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdd185c640 1 -- 172.21.15.22:0/1213681454 shutdown_connections 2024-03-29T04:17:51.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdd185c640 1 --2- 172.21.15.22:0/1213681454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc0b2e80 0x7efdcc15f5e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdd185c640 1 -- 172.21.15.22:0/1213681454 >> 172.21.15.22:0/1213681454 conn(0x7efdcc0d00a0 msgr2=0x7efdcc0d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:51.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdd185c640 1 -- 172.21.15.22:0/1213681454 shutdown_connections 2024-03-29T04:17:51.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdd185c640 1 -- 172.21.15.22:0/1213681454 wait complete. 2024-03-29T04:17:51.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdd185c640 1 Processor -- start 2024-03-29T04:17:51.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdd185c640 1 -- start start 2024-03-29T04:17:51.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdd185c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efdcc06cd80 0x7efdcc14c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:51.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdd185c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc0b2e80 0x7efdcc14b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:51.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdd185c640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efdcc155160 0x7efdcc1497a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:51.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdd185c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc072770 con 0x7efdcc0b2e80 2024-03-29T04:17:51.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdd185c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc0725f0 con 0x7efdcc06cd80 2024-03-29T04:17:51.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdd185c640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7efdcc0728f0 con 0x7efdcc155160 2024-03-29T04:17:51.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdca7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc0b2e80 0x7efdcc14b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:51.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdca7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc0b2e80 0x7efdcc14b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49482/0 (socket says 172.21.15.22:49482) 2024-03-29T04:17:51.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.976+0000 7efdca7fc640 1 -- 172.21.15.22:0/3346154191 learned_addr learned my addr 172.21.15.22:0/3346154191 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:51.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.980+0000 7efdcb7fe640 1 --2- 172.21.15.22:0/3346154191 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efdcc155160 0x7efdcc1497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:51.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.980+0000 7efdcaffd640 1 --2- 172.21.15.22:0/3346154191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efdcc06cd80 0x7efdcc14c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:51.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.980+0000 7efdca7fc640 1 -- 172.21.15.22:0/3346154191 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efdcc155160 msgr2=0x7efdcc1497a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:51.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.980+0000 7efdca7fc640 1 --2- 172.21.15.22:0/3346154191 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efdcc155160 0x7efdcc1497a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.980+0000 7efdca7fc640 1 -- 172.21.15.22:0/3346154191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efdcc06cd80 msgr2=0x7efdcc14c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:51.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.980+0000 7efdca7fc640 1 --2- 172.21.15.22:0/3346154191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efdcc06cd80 0x7efdcc14c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:51.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.980+0000 7efdca7fc640 1 -- 172.21.15.22:0/3346154191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdcc14c8d0 con 0x7efdcc0b2e80 2024-03-29T04:17:51.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.980+0000 7efdca7fc640 1 --2- 172.21.15.22:0/3346154191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc0b2e80 0x7efdcc14b4d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efdb400c500 tx=0x7efdb400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:51.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.980+0000 7efdc3fff640 1 -- 172.21.15.22:0/3346154191 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb4016020 con 0x7efdcc0b2e80 2024-03-29T04:17:51.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.980+0000 7efdc3fff640 1 -- 172.21.15.22:0/3346154191 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb4005220 con 0x7efdcc0b2e80 2024-03-29T04:17:51.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.980+0000 7efdd185c640 1 -- 172.21.15.22:0/3346154191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdcc14cbb0 con 0x7efdcc0b2e80 2024-03-29T04:17:51.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.980+0000 7efdc3fff640 1 -- 172.21.15.22:0/3346154191 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb4007030 con 0x7efdcc0b2e80 2024-03-29T04:17:51.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.980+0000 7efdd185c640 1 -- 172.21.15.22:0/3346154191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdcc14d170 con 0x7efdcc0b2e80 2024-03-29T04:17:51.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.980+0000 7efdd185c640 1 -- 172.21.15.22:0/3346154191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdcc068900 con 0x7efdcc0b2e80 2024-03-29T04:17:51.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.980+0000 7efdc3fff640 1 -- 172.21.15.22:0/3346154191 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7efdb40071d0 con 0x7efdcc0b2e80 2024-03-29T04:17:51.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.980+0000 7efdc3fff640 1 --2- 172.21.15.22:0/3346154191 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efdac067ff0 0x7efdac06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:51.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.980+0000 7efdc3fff640 1 -- 172.21.15.22:0/3346154191 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7efdb4086320 con 0x7efdcc0b2e80 2024-03-29T04:17:51.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.980+0000 7efdcaffd640 1 --2- 172.21.15.22:0/3346154191 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efdac067ff0 0x7efdac06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:51.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.980+0000 7efdcaffd640 1 --2- 172.21.15.22:0/3346154191 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efdac067ff0 0x7efdac06a4b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0011f0 tx=0x7efdbc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:51.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:51.984+0000 7efdc3fff640 1 -- 172.21.15.22:0/3346154191 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7efdb4020020 con 0x7efdcc0b2e80 2024-03-29T04:17:52.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.184+0000 7efdd185c640 1 -- 172.21.15.22:0/3346154191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7efdcc1608f0 con 0x7efdcc0b2e80 2024-03-29T04:17:52.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.784+0000 7efdc3fff640 1 -- 172.21.15.22:0/3346154191 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7efdb40074c0 con 0x7efdcc0b2e80 2024-03-29T04:17:52.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.788+0000 7efdd185c640 1 -- 172.21.15.22:0/3346154191 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efdac067ff0 msgr2=0x7efdac06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:52.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.788+0000 7efdd185c640 1 --2- 172.21.15.22:0/3346154191 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efdac067ff0 0x7efdac06a4b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0011f0 tx=0x7efdbc009040 comp rx=0 tx=0).stop 2024-03-29T04:17:52.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.788+0000 7efdd185c640 1 -- 172.21.15.22:0/3346154191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc0b2e80 msgr2=0x7efdcc14b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:52.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.788+0000 7efdd185c640 1 --2- 172.21.15.22:0/3346154191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc0b2e80 0x7efdcc14b4d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efdb400c500 tx=0x7efdb400c9c0 comp rx=0 tx=0).stop 2024-03-29T04:17:52.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.788+0000 7efdd185c640 1 -- 172.21.15.22:0/3346154191 shutdown_connections 2024-03-29T04:17:52.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.788+0000 7efdd185c640 1 --2- 172.21.15.22:0/3346154191 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efdcc155160 0x7efdcc1497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:52.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.788+0000 7efdd185c640 1 --2- 172.21.15.22:0/3346154191 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efdac067ff0 0x7efdac06a4b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:52.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.788+0000 7efdd185c640 1 --2- 172.21.15.22:0/3346154191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc0b2e80 0x7efdcc14b4d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:52.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.788+0000 7efdd185c640 1 --2- 172.21.15.22:0/3346154191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efdcc06cd80 0x7efdcc14c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:52.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.788+0000 7efdd185c640 1 -- 172.21.15.22:0/3346154191 >> 172.21.15.22:0/3346154191 conn(0x7efdcc0d00a0 msgr2=0x7efdcc160e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:52.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.788+0000 7efdd185c640 1 -- 172.21.15.22:0/3346154191 shutdown_connections 2024-03-29T04:17:52.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.788+0000 7efdd185c640 1 -- 172.21.15.22:0/3346154191 wait complete. 2024-03-29T04:17:52.789 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-03-29T04:17:52.790 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-29T04:17:52.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f5241a18640 1 Processor -- start 2024-03-29T04:17:52.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f5241a18640 1 -- start start 2024-03-29T04:17:52.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f5241a18640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f523c06fd20 0x7f523c06eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:52.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f5241a18640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f523c06dc00 con 0x7f523c15bd70 2024-03-29T04:17:52.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f5241a18640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f523c06dee0 con 0x7f523c0afb90 2024-03-29T04:17:52.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f5241a18640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f523c06e1c0 con 0x7f523c06fd20 2024-03-29T04:17:52.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f523affd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f523c15bd70 0x7f523c15c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51120/0 (socket says 172.21.15.22:51120) 2024-03-29T04:17:52.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f523affd640 1 -- 172.21.15.22:0/1240004466 learned_addr learned my addr 172.21.15.22:0/1240004466 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:52.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f523b7fe640 1 --2- 172.21.15.22:0/1240004466 >> v2:172.21.15.151:3300/0 conn(0x7f523c06fd20 0x7f523c06eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:52.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f5239ffb640 1 -- 172.21.15.22:0/1240004466 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1311012474 0 0) 0x7f523c06dc00 con 0x7f523c15bd70 2024-03-29T04:17:52.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f5239ffb640 1 -- 172.21.15.22:0/1240004466 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5224005740 con 0x7f523c15bd70 2024-03-29T04:17:52.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f5239ffb640 1 -- 172.21.15.22:0/1240004466 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2627390007 0 0) 0x7f5224005740 con 0x7f523c15bd70 2024-03-29T04:17:52.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f5239ffb640 1 -- 172.21.15.22:0/1240004466 >> v2:172.21.15.151:3300/0 conn(0x7f523c06fd20 msgr2=0x7f523c06eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:52.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f5239ffb640 1 --2- 172.21.15.22:0/1240004466 >> v2:172.21.15.151:3300/0 conn(0x7f523c06fd20 0x7f523c06eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:52.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f5239ffb640 1 -- 172.21.15.22:0/1240004466 >> v1:172.21.15.151:6789/0 conn(0x7f523c0afb90 legacy=0x7f523c06f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:52.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f5239ffb640 1 -- 172.21.15.22:0/1240004466 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523c06e4a0 con 0x7f523c15bd70 2024-03-29T04:17:52.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f5239ffb640 1 -- 172.21.15.22:0/1240004466 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f5234001a00 con 0x7f523c15bd70 2024-03-29T04:17:52.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f5239ffb640 1 -- 172.21.15.22:0/1240004466 >> v1:172.21.15.22:6789/0 conn(0x7f523c15bd70 legacy=0x7f523c15c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:52.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f5239ffb640 1 --2- 172.21.15.22:0/1240004466 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5224005bd0 0x7f5224005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:52.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f5239ffb640 1 --2- 172.21.15.22:0/1240004466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5224006a20 0x7f5224006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:52.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f5239ffb640 1 --2- 172.21.15.22:0/1240004466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52240073e0 0x7f522400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:52.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f5239ffb640 1 -- 172.21.15.22:0/1240004466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f523c06e1c0 con 0x7f5224006a20 2024-03-29T04:17:52.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f5239ffb640 1 -- 172.21.15.22:0/1240004466 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f522400e280 con 0x7f52240073e0 2024-03-29T04:17:52.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f5239ffb640 1 -- 172.21.15.22:0/1240004466 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f522400e480 con 0x7f5224005bd0 2024-03-29T04:17:52.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f523a7fc640 1 --2- 172.21.15.22:0/1240004466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5224006a20 0x7f5224006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:52.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f523affd640 1 --2- 172.21.15.22:0/1240004466 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5224005bd0 0x7f5224005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:52.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f523b7fe640 1 --2- 172.21.15.22:0/1240004466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52240073e0 0x7f522400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:52.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f523a7fc640 1 -- 172.21.15.22:0/1240004466 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5224005bd0 msgr2=0x7f5224005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:52.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f523a7fc640 1 --2- 172.21.15.22:0/1240004466 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5224005bd0 0x7f5224005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:52.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f523a7fc640 1 -- 172.21.15.22:0/1240004466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52240073e0 msgr2=0x7f522400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:52.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f523a7fc640 1 --2- 172.21.15.22:0/1240004466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52240073e0 0x7f522400dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:52.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f523a7fc640 1 -- 172.21.15.22:0/1240004466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5224005740 con 0x7f5224006a20 2024-03-29T04:17:52.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.940+0000 7f523b7fe640 1 -- 172.21.15.22:0/1240004466 reap_dead start 2024-03-29T04:17:52.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.944+0000 7f523a7fc640 1 --2- 172.21.15.22:0/1240004466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5224006a20 0x7f5224006ea0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5230001750 tx=0x7f523001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3284298e3bb56c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:52.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.944+0000 7f5239ffb640 1 -- 172.21.15.22:0/1240004466 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52300013a0 con 0x7f5224006a20 2024-03-29T04:17:52.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.944+0000 7f5239ffb640 1 -- 172.21.15.22:0/1240004466 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52300052f0 con 0x7f5224006a20 2024-03-29T04:17:52.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.944+0000 7f5239ffb640 1 -- 172.21.15.22:0/1240004466 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230003840 con 0x7f5224006a20 2024-03-29T04:17:52.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f5241a18640 1 -- 172.21.15.22:0/1240004466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5224006a20 msgr2=0x7f5224006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:52.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f5241a18640 1 --2- 172.21.15.22:0/1240004466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5224006a20 0x7f5224006ea0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5230001750 tx=0x7f523001c3c0 comp rx=0 tx=0).stop 2024-03-29T04:17:52.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f5241a18640 1 -- 172.21.15.22:0/1240004466 shutdown_connections 2024-03-29T04:17:52.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f5241a18640 1 --2- 172.21.15.22:0/1240004466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5224006a20 0x7f5224006ea0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:52.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f5241a18640 1 -- 172.21.15.22:0/1240004466 >> 172.21.15.22:0/1240004466 conn(0x7f523c0d0c90 msgr2=0x7f523c0d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:52.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f5241a18640 1 -- 172.21.15.22:0/1240004466 shutdown_connections 2024-03-29T04:17:52.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f5241a18640 1 -- 172.21.15.22:0/1240004466 wait complete. 2024-03-29T04:17:52.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f5241a18640 1 Processor -- start 2024-03-29T04:17:52.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f5241a18640 1 -- start start 2024-03-29T04:17:52.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f5241a18640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5224006a20 0x7f523c1f0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:52.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f5241a18640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f523c14f2b0 0x7f523c142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:52.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f5241a18640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f523c14caa0 0x7f523c14cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:52.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f5241a18640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f523c06ee20 con 0x7f523c14f2b0 2024-03-29T04:17:52.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f5241a18640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f523c06eca0 con 0x7f5224006a20 2024-03-29T04:17:52.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f5241a18640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f523c06efa0 con 0x7f523c14caa0 2024-03-29T04:17:52.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f523a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f523c14f2b0 0x7f523c142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:52.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f523a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f523c14f2b0 0x7f523c142250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49506/0 (socket says 172.21.15.22:49506) 2024-03-29T04:17:52.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f523a7fc640 1 -- 172.21.15.22:0/1120080646 learned_addr learned my addr 172.21.15.22:0/1120080646 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:52.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f523b7fe640 1 --2- 172.21.15.22:0/1120080646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f523c14caa0 0x7f523c14cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:52.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f523affd640 1 --2- 172.21.15.22:0/1120080646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5224006a20 0x7f523c1f0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:52.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f523a7fc640 1 -- 172.21.15.22:0/1120080646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f523c14caa0 msgr2=0x7f523c14cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:52.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f523a7fc640 1 --2- 172.21.15.22:0/1120080646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f523c14caa0 0x7f523c14cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:52.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f523a7fc640 1 -- 172.21.15.22:0/1120080646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5224006a20 msgr2=0x7f523c1f0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:52.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f523a7fc640 1 --2- 172.21.15.22:0/1120080646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5224006a20 0x7f523c1f0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:52.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f523a7fc640 1 -- 172.21.15.22:0/1120080646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523c150be0 con 0x7f523c14f2b0 2024-03-29T04:17:52.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f523a7fc640 1 --2- 172.21.15.22:0/1120080646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f523c14f2b0 0x7f523c142250 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f523001c1f0 tx=0x7f5224007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:52.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f5223fff640 1 -- 172.21.15.22:0/1120080646 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52300013a0 con 0x7f523c14f2b0 2024-03-29T04:17:52.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f5223fff640 1 -- 172.21.15.22:0/1120080646 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52300ba070 con 0x7f523c14f2b0 2024-03-29T04:17:52.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f5241a18640 1 -- 172.21.15.22:0/1120080646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523c14ee50 con 0x7f523c14f2b0 2024-03-29T04:17:52.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f5223fff640 1 -- 172.21.15.22:0/1120080646 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230007a40 con 0x7f523c14f2b0 2024-03-29T04:17:52.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.948+0000 7f5241a18640 1 -- 172.21.15.22:0/1120080646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523c1f0f40 con 0x7f523c14f2b0 2024-03-29T04:17:52.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.952+0000 7f5241a18640 1 -- 172.21.15.22:0/1120080646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f523c068900 con 0x7f523c14f2b0 2024-03-29T04:17:52.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.952+0000 7f5223fff640 1 -- 172.21.15.22:0/1120080646 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f5230007470 con 0x7f523c14f2b0 2024-03-29T04:17:52.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.952+0000 7f5223fff640 1 --2- 172.21.15.22:0/1120080646 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5228067fc0 0x7f522806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:52.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.952+0000 7f5223fff640 1 -- 172.21.15.22:0/1120080646 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5230120a50 con 0x7f523c14f2b0 2024-03-29T04:17:52.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.952+0000 7f523affd640 1 --2- 172.21.15.22:0/1120080646 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5228067fc0 0x7f522806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:52.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.952+0000 7f523affd640 1 --2- 172.21.15.22:0/1120080646 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5228067fc0 0x7f522806a480 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5234000f90 tx=0x7f52340017c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:52.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:52.956+0000 7f5223fff640 1 -- 172.21.15.22:0/1120080646 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f52300160d0 con 0x7f523c14f2b0 2024-03-29T04:17:53.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.152+0000 7f5241a18640 1 -- 172.21.15.22:0/1120080646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f523c150db0 con 0x7f523c14f2b0 2024-03-29T04:17:53.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.792+0000 7f5223fff640 1 -- 172.21.15.22:0/1120080646 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f52300f0870 con 0x7f523c14f2b0 2024-03-29T04:17:53.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.792+0000 7f5241a18640 1 -- 172.21.15.22:0/1120080646 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5228067fc0 msgr2=0x7f522806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:53.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.792+0000 7f5241a18640 1 --2- 172.21.15.22:0/1120080646 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5228067fc0 0x7f522806a480 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5234000f90 tx=0x7f52340017c0 comp rx=0 tx=0).stop 2024-03-29T04:17:53.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.792+0000 7f5241a18640 1 -- 172.21.15.22:0/1120080646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f523c14f2b0 msgr2=0x7f523c142250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:53.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.792+0000 7f5241a18640 1 --2- 172.21.15.22:0/1120080646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f523c14f2b0 0x7f523c142250 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f523001c1f0 tx=0x7f5224007a50 comp rx=0 tx=0).stop 2024-03-29T04:17:53.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.796+0000 7f5241a18640 1 -- 172.21.15.22:0/1120080646 shutdown_connections 2024-03-29T04:17:53.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.796+0000 7f5241a18640 1 --2- 172.21.15.22:0/1120080646 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f523c14caa0 0x7f523c14cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:53.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.796+0000 7f5241a18640 1 --2- 172.21.15.22:0/1120080646 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5228067fc0 0x7f522806a480 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:53.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.796+0000 7f5241a18640 1 --2- 172.21.15.22:0/1120080646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f523c14f2b0 0x7f523c142250 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:53.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.796+0000 7f5241a18640 1 --2- 172.21.15.22:0/1120080646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5224006a20 0x7f523c1f0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:53.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.796+0000 7f5241a18640 1 -- 172.21.15.22:0/1120080646 >> 172.21.15.22:0/1120080646 conn(0x7f523c0d0c90 msgr2=0x7f523c06f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:53.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.796+0000 7f5241a18640 1 -- 172.21.15.22:0/1120080646 shutdown_connections 2024-03-29T04:17:53.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.796+0000 7f5241a18640 1 -- 172.21.15.22:0/1120080646 wait complete. 2024-03-29T04:17:53.798 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-29T04:17:53.798 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:17:53.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.940+0000 7fa30f824640 1 Processor -- start 2024-03-29T04:17:53.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.940+0000 7fa30f824640 1 -- start start 2024-03-29T04:17:53.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.944+0000 7fa30f824640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa30806c670 0x7fa3080cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:53.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.944+0000 7fa30f824640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fa30806ca50 0x7fa3080ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:53.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.944+0000 7fa30f824640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3080ccef0 con 0x7fa3080b41f0 2024-03-29T04:17:53.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.944+0000 7fa30f824640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fa3080b34c0 con 0x7fa30806ca50 2024-03-29T04:17:53.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.944+0000 7fa30f824640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa3080b3710 con 0x7fa30806c670 2024-03-29T04:17:53.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.944+0000 7fa30d5c0640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa30806c670 0x7fa3080cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:53.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.944+0000 7fa30d5c0640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa30806c670 0x7fa3080cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.22:36326/0 (socket says 172.21.15.22:36326) 2024-03-29T04:17:53.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.944+0000 7fa30d5c0640 1 -- 172.21.15.22:0/4238079624 learned_addr learned my addr 172.21.15.22:0/4238079624 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:53.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.944+0000 7fa30ddc1640 1 --2- 172.21.15.22:0/4238079624 >> v2:172.21.15.151:3300/0 conn(0x7fa30806ca50 0x7fa3080ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:53.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.944+0000 7fa30ddc1640 1 -- 172.21.15.22:0/4238079624 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa30806c670 msgr2=0x7fa3080cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:53.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.944+0000 7fa30ddc1640 1 --2- 172.21.15.22:0/4238079624 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa30806c670 0x7fa3080cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:53.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.944+0000 7fa30ddc1640 1 -- 172.21.15.22:0/4238079624 >> v1:172.21.15.151:6789/0 conn(0x7fa3080b41f0 legacy=0x7fa30806e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:53.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.944+0000 7fa30ddc1640 1 -- 172.21.15.22:0/4238079624 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3080b39f0 con 0x7fa30806ca50 2024-03-29T04:17:53.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.944+0000 7fa30d5c0640 1 --2- 172.21.15.22:0/4238079624 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa30806c670 0x7fa3080cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:17:53.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30ddc1640 1 --2- 172.21.15.22:0/4238079624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa30806ca50 0x7fa3080ce5b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa300009d90 tx=0x7fa30001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb522c515271a375 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:53.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa2fffff640 1 -- 172.21.15.22:0/4238079624 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3000b2070 con 0x7fa30806ca50 2024-03-29T04:17:53.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa2fffff640 1 -- 172.21.15.22:0/4238079624 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3000a5940 con 0x7fa30806ca50 2024-03-29T04:17:53.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa2fffff640 1 -- 172.21.15.22:0/4238079624 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3000a5d10 con 0x7fa30806ca50 2024-03-29T04:17:53.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30f824640 1 -- 172.21.15.22:0/4238079624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa30806ca50 msgr2=0x7fa3080ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:53.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30f824640 1 --2- 172.21.15.22:0/4238079624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa30806ca50 0x7fa3080ce5b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa300009d90 tx=0x7fa30001c970 comp rx=0 tx=0).stop 2024-03-29T04:17:53.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30f824640 1 -- 172.21.15.22:0/4238079624 shutdown_connections 2024-03-29T04:17:53.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30f824640 1 --2- 172.21.15.22:0/4238079624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa30806ca50 0x7fa3080ce5b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:53.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30f824640 1 --2- 172.21.15.22:0/4238079624 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa30806c670 0x7fa3080cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:53.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30f824640 1 -- 172.21.15.22:0/4238079624 >> 172.21.15.22:0/4238079624 conn(0x7fa3080cfee0 msgr2=0x7fa3080d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:53.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30f824640 1 -- 172.21.15.22:0/4238079624 shutdown_connections 2024-03-29T04:17:53.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30f824640 1 -- 172.21.15.22:0/4238079624 wait complete. 2024-03-29T04:17:53.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30f824640 1 Processor -- start 2024-03-29T04:17:53.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30f824640 1 -- start start 2024-03-29T04:17:53.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30f824640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa30806c670 0x7fa3081f8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:53.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30f824640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa30806ca50 0x7fa3081f8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:53.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30f824640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa3080b41f0 0x7fa30813c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:53.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30f824640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa308169070 con 0x7fa30806ca50 2024-03-29T04:17:53.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30f824640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa308168ef0 con 0x7fa30806c670 2024-03-29T04:17:53.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30f824640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa3081691f0 con 0x7fa3080b41f0 2024-03-29T04:17:53.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30cdbf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa30806ca50 0x7fa3081f8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:53.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30cdbf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa30806ca50 0x7fa3081f8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49516/0 (socket says 172.21.15.22:49516) 2024-03-29T04:17:53.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30cdbf640 1 -- 172.21.15.22:0/3727779711 learned_addr learned my addr 172.21.15.22:0/3727779711 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:53.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30ddc1640 1 --2- 172.21.15.22:0/3727779711 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa3080b41f0 0x7fa30813c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:53.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30d5c0640 1 --2- 172.21.15.22:0/3727779711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa30806c670 0x7fa3081f8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:53.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30cdbf640 1 -- 172.21.15.22:0/3727779711 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa3080b41f0 msgr2=0x7fa30813c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:53.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30cdbf640 1 --2- 172.21.15.22:0/3727779711 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa3080b41f0 0x7fa30813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:53.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30cdbf640 1 -- 172.21.15.22:0/3727779711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa30806c670 msgr2=0x7fa3081f8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:53.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30cdbf640 1 --2- 172.21.15.22:0/3727779711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa30806c670 0x7fa3081f8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:53.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30cdbf640 1 -- 172.21.15.22:0/3727779711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30813d610 con 0x7fa30806ca50 2024-03-29T04:17:53.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30cdbf640 1 --2- 172.21.15.22:0/3727779711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa30806ca50 0x7fa3081f8f80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800c500 tx=0x7fa2f800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:53.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa2fdffb640 1 -- 172.21.15.22:0/3727779711 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8016020 con 0x7fa30806ca50 2024-03-29T04:17:53.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa2fdffb640 1 -- 172.21.15.22:0/3727779711 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2f8005220 con 0x7fa30806ca50 2024-03-29T04:17:53.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa2fdffb640 1 -- 172.21.15.22:0/3727779711 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8007030 con 0x7fa30806ca50 2024-03-29T04:17:53.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.948+0000 7fa30f824640 1 -- 172.21.15.22:0/3727779711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30813d8f0 con 0x7fa30806ca50 2024-03-29T04:17:53.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.952+0000 7fa30f824640 1 -- 172.21.15.22:0/3727779711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa30813deb0 con 0x7fa30806ca50 2024-03-29T04:17:53.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.952+0000 7fa30f824640 1 -- 172.21.15.22:0/3727779711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa308068890 con 0x7fa30806ca50 2024-03-29T04:17:53.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.952+0000 7fa2fdffb640 1 -- 172.21.15.22:0/3727779711 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fa2f80071d0 con 0x7fa30806ca50 2024-03-29T04:17:53.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.952+0000 7fa2fdffb640 1 --2- 172.21.15.22:0/3727779711 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa2ec067fc0 0x7fa2ec06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:53.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.952+0000 7fa2fdffb640 1 -- 172.21.15.22:0/3727779711 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa2f8086230 con 0x7fa30806ca50 2024-03-29T04:17:53.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.952+0000 7fa30d5c0640 1 --2- 172.21.15.22:0/3727779711 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa2ec067fc0 0x7fa2ec06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:53.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.952+0000 7fa30d5c0640 1 --2- 172.21.15.22:0/3727779711 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa2ec067fc0 0x7fa2ec06a480 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa304009a40 tx=0x7fa304008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:53.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:53.956+0000 7fa2fdffb640 1 -- 172.21.15.22:0/3727779711 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fa2f8020020 con 0x7fa30806ca50 2024-03-29T04:17:54.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.156+0000 7fa30f824640 1 -- 172.21.15.22:0/3727779711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa30813b0b0 con 0x7fa30806ca50 2024-03-29T04:17:54.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.156+0000 7fa2fdffb640 1 -- 172.21.15.22:0/3727779711 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7fa2f80074c0 con 0x7fa30806ca50 2024-03-29T04:17:54.139 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:54.139 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3094991863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3094991863},{"type":"v1","addr":"172.21.15.22:6834","nonce":3094991863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2464050026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2464050026},{"type":"v1","addr":"172.21.15.151:6835","nonce":2464050026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2999052384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2999052384},{"type":"v1","addr":"172.21.15.22:6835","nonce":2999052384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:17:35.918069+0000","modified":"2024-03-29T04:17:53.156365+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6837/1466916671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":1466916671},{"type":"v1","addr":"172.21.15.151:6837","nonce":1466916671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-03-29T04:17:54.139 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2024-03-29T04:17:54.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.160+0000 7fa30f824640 1 -- 172.21.15.22:0/3727779711 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa2ec067fc0 msgr2=0x7fa2ec06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:54.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.160+0000 7fa30f824640 1 --2- 172.21.15.22:0/3727779711 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa2ec067fc0 0x7fa2ec06a480 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa304009a40 tx=0x7fa304008040 comp rx=0 tx=0).stop 2024-03-29T04:17:54.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.160+0000 7fa30f824640 1 -- 172.21.15.22:0/3727779711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa30806ca50 msgr2=0x7fa3081f8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:54.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.160+0000 7fa30f824640 1 --2- 172.21.15.22:0/3727779711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa30806ca50 0x7fa3081f8f80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800c500 tx=0x7fa2f800c9c0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.160+0000 7fa30f824640 1 -- 172.21.15.22:0/3727779711 shutdown_connections 2024-03-29T04:17:54.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.160+0000 7fa30f824640 1 --2- 172.21.15.22:0/3727779711 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa3080b41f0 0x7fa30813c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.160+0000 7fa30f824640 1 --2- 172.21.15.22:0/3727779711 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa2ec067fc0 0x7fa2ec06a480 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.160+0000 7fa30f824640 1 --2- 172.21.15.22:0/3727779711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa30806ca50 0x7fa3081f8f80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.160+0000 7fa30f824640 1 --2- 172.21.15.22:0/3727779711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa30806c670 0x7fa3081f8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.160+0000 7fa30f824640 1 -- 172.21.15.22:0/3727779711 >> 172.21.15.22:0/3727779711 conn(0x7fa3080cfee0 msgr2=0x7fa30815b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:54.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.160+0000 7fa30f824640 1 -- 172.21.15.22:0/3727779711 shutdown_connections 2024-03-29T04:17:54.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.160+0000 7fa30f824640 1 -- 172.21.15.22:0/3727779711 wait complete. 2024-03-29T04:17:54.158 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:17:54.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.304+0000 7f024172f640 1 Processor -- start 2024-03-29T04:17:54.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.304+0000 7f024172f640 1 -- start start 2024-03-29T04:17:54.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.304+0000 7f024172f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f023c144270 0x7f023c144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:54.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.304+0000 7f024172f640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f023c0b2ef0 0x7f023c15f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:54.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.304+0000 7f024172f640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f023c0b33f0 con 0x7f023c0b4490 2024-03-29T04:17:54.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.304+0000 7f024172f640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f023c144fa0 con 0x7f023c144270 2024-03-29T04:17:54.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.304+0000 7f024172f640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f023c15fce0 con 0x7f023c0b2ef0 2024-03-29T04:17:54.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.304+0000 7f023affd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f023c144270 0x7f023c144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:54.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.304+0000 7f023b7fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f023c0b4490 0x7f023c06c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51134/0 (socket says 172.21.15.22:51134) 2024-03-29T04:17:54.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.304+0000 7f023b7fe640 1 -- 172.21.15.22:0/228477845 learned_addr learned my addr 172.21.15.22:0/228477845 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:54.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.304+0000 7f023a7fc640 1 --2- 172.21.15.22:0/228477845 >> v2:172.21.15.151:3300/0 conn(0x7f023c0b2ef0 0x7f023c15f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:54.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.304+0000 7f0239ffb640 1 -- 172.21.15.22:0/228477845 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1365315695 0 0) 0x7f023c0b33f0 con 0x7f023c0b4490 2024-03-29T04:17:54.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.304+0000 7f0239ffb640 1 -- 172.21.15.22:0/228477845 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0224003530 con 0x7f023c0b4490 2024-03-29T04:17:54.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.304+0000 7f023affd640 1 -- 172.21.15.22:0/228477845 >> v2:172.21.15.151:3300/0 conn(0x7f023c0b2ef0 msgr2=0x7f023c15f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:54.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.304+0000 7f023affd640 1 --2- 172.21.15.22:0/228477845 >> v2:172.21.15.151:3300/0 conn(0x7f023c0b2ef0 0x7f023c15f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.304+0000 7f023affd640 1 -- 172.21.15.22:0/228477845 >> v1:172.21.15.22:6789/0 conn(0x7f023c0b4490 legacy=0x7f023c06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:54.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.304+0000 7f023affd640 1 -- 172.21.15.22:0/228477845 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023c0b36d0 con 0x7f023c144270 2024-03-29T04:17:54.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.308+0000 7f023affd640 1 --2- 172.21.15.22:0/228477845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f023c144270 0x7f023c144650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f023c0b4e70 tx=0x7f022c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694a111fd62dc5d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:54.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.308+0000 7f0239ffb640 1 -- 172.21.15.22:0/228477845 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c0ab650 con 0x7f023c144270 2024-03-29T04:17:54.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.308+0000 7f0239ffb640 1 -- 172.21.15.22:0/228477845 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f022c0b0070 con 0x7f023c144270 2024-03-29T04:17:54.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.308+0000 7f0239ffb640 1 -- 172.21.15.22:0/228477845 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c0ab9f0 con 0x7f023c144270 2024-03-29T04:17:54.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.308+0000 7f024172f640 1 -- 172.21.15.22:0/228477845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f023c144270 msgr2=0x7f023c144650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:54.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.308+0000 7f024172f640 1 --2- 172.21.15.22:0/228477845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f023c144270 0x7f023c144650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f023c0b4e70 tx=0x7f022c01c450 comp rx=0 tx=0).stop 2024-03-29T04:17:54.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.308+0000 7f024172f640 1 -- 172.21.15.22:0/228477845 shutdown_connections 2024-03-29T04:17:54.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.308+0000 7f024172f640 1 --2- 172.21.15.22:0/228477845 >> v2:172.21.15.151:3300/0 conn(0x7f023c0b2ef0 0x7f023c15f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.308+0000 7f024172f640 1 --2- 172.21.15.22:0/228477845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f023c144270 0x7f023c144650 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.308+0000 7f024172f640 1 -- 172.21.15.22:0/228477845 >> 172.21.15.22:0/228477845 conn(0x7f023c0cfee0 msgr2=0x7f023c0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:54.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.308+0000 7f024172f640 1 -- 172.21.15.22:0/228477845 shutdown_connections 2024-03-29T04:17:54.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.308+0000 7f024172f640 1 -- 172.21.15.22:0/228477845 wait complete. 2024-03-29T04:17:54.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.308+0000 7f024172f640 1 Processor -- start 2024-03-29T04:17:54.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.308+0000 7f024172f640 1 -- start start 2024-03-29T04:17:54.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.308+0000 7f024172f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f023c0b2ef0 0x7f023c1f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:54.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.308+0000 7f024172f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f023c0b4490 0x7f023c164cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:54.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.308+0000 7f024172f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f023c144270 0x7f023c165210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:54.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.308+0000 7f024172f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f023c0cdd10 con 0x7f023c0b4490 2024-03-29T04:17:54.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f024172f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f023c06cf90 con 0x7f023c144270 2024-03-29T04:17:54.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f024172f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f023c0cde90 con 0x7f023c0b2ef0 2024-03-29T04:17:54.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f023a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f023c0b4490 0x7f023c164cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:54.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f023a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f023c0b4490 0x7f023c164cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49546/0 (socket says 172.21.15.22:49546) 2024-03-29T04:17:54.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f023a7fc640 1 -- 172.21.15.22:0/2522093311 learned_addr learned my addr 172.21.15.22:0/2522093311 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:54.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f023affd640 1 --2- 172.21.15.22:0/2522093311 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f023c0b2ef0 0x7f023c1f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:54.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f023b7fe640 1 --2- 172.21.15.22:0/2522093311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f023c144270 0x7f023c165210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:54.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f023a7fc640 1 -- 172.21.15.22:0/2522093311 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f023c0b2ef0 msgr2=0x7f023c1f63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:54.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f023a7fc640 1 --2- 172.21.15.22:0/2522093311 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f023c0b2ef0 0x7f023c1f63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f023a7fc640 1 -- 172.21.15.22:0/2522093311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f023c144270 msgr2=0x7f023c165210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:54.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f023a7fc640 1 --2- 172.21.15.22:0/2522093311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f023c144270 0x7f023c165210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f023a7fc640 1 -- 172.21.15.22:0/2522093311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023c165840 con 0x7f023c0b4490 2024-03-29T04:17:54.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f023b7fe640 1 --2- 172.21.15.22:0/2522093311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f023c144270 0x7f023c165210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:54.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f023a7fc640 1 --2- 172.21.15.22:0/2522093311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f023c0b4490 0x7f023c164cd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f023000e840 tx=0x7f023000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:54.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f02237fe640 1 -- 172.21.15.22:0/2522093311 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0230017070 con 0x7f023c0b4490 2024-03-29T04:17:54.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f02237fe640 1 -- 172.21.15.22:0/2522093311 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f023000ee90 con 0x7f023c0b4490 2024-03-29T04:17:54.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f024172f640 1 -- 172.21.15.22:0/2522093311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f023c1f6db0 con 0x7f023c0b4490 2024-03-29T04:17:54.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f02237fe640 1 -- 172.21.15.22:0/2522093311 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023000cb20 con 0x7f023c0b4490 2024-03-29T04:17:54.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f024172f640 1 -- 172.21.15.22:0/2522093311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f023c13b680 con 0x7f023c0b4490 2024-03-29T04:17:54.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f024172f640 1 -- 172.21.15.22:0/2522093311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f023c068890 con 0x7f023c0b4490 2024-03-29T04:17:54.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f02237fe640 1 -- 172.21.15.22:0/2522093311 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f023001a070 con 0x7f023c0b4490 2024-03-29T04:17:54.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f02237fe640 1 --2- 172.21.15.22:0/2522093311 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0228067ff0 0x7f022806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:54.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f02237fe640 1 -- 172.21.15.22:0/2522093311 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0230086a30 con 0x7f023c0b4490 2024-03-29T04:17:54.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f023affd640 1 --2- 172.21.15.22:0/2522093311 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0228067ff0 0x7f022806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:54.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.312+0000 7f023affd640 1 --2- 172.21.15.22:0/2522093311 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0228067ff0 0x7f022806a4b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f022c0af000 tx=0x7f022c0a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:54.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.316+0000 7f02237fe640 1 -- 172.21.15.22:0/2522093311 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f0230056e90 con 0x7f023c0b4490 2024-03-29T04:17:54.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.476+0000 7f024172f640 1 -- 172.21.15.22:0/2522093311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f023c073500 con 0x7f023c0b4490 2024-03-29T04:17:54.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.480+0000 7f02237fe640 1 -- 172.21.15.22:0/2522093311 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18514 (secure 0 0 0) 0x7f0230056790 con 0x7f023c0b4490 2024-03-29T04:17:54.462 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:54.462 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:17:42.645632+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:17:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.480+0000 7f024172f640 1 -- 172.21.15.22:0/2522093311 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0228067ff0 msgr2=0x7f022806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.480+0000 7f024172f640 1 --2- 172.21.15.22:0/2522093311 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0228067ff0 0x7f022806a4b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f022c0af000 tx=0x7f022c0a7000 comp rx=0 tx=0).stop 2024-03-29T04:17:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.484+0000 7f024172f640 1 -- 172.21.15.22:0/2522093311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f023c0b4490 msgr2=0x7f023c164cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.484+0000 7f024172f640 1 --2- 172.21.15.22:0/2522093311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f023c0b4490 0x7f023c164cd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f023000e840 tx=0x7f023000ec40 comp rx=0 tx=0).stop 2024-03-29T04:17:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.484+0000 7f024172f640 1 -- 172.21.15.22:0/2522093311 shutdown_connections 2024-03-29T04:17:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.484+0000 7f024172f640 1 --2- 172.21.15.22:0/2522093311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f023c144270 0x7f023c165210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.484+0000 7f024172f640 1 --2- 172.21.15.22:0/2522093311 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0228067ff0 0x7f022806a4b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.484+0000 7f024172f640 1 --2- 172.21.15.22:0/2522093311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f023c0b4490 0x7f023c164cd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.484+0000 7f024172f640 1 --2- 172.21.15.22:0/2522093311 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f023c0b2ef0 0x7f023c1f63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.484+0000 7f024172f640 1 -- 172.21.15.22:0/2522093311 >> 172.21.15.22:0/2522093311 conn(0x7f023c0cfee0 msgr2=0x7f023c06b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.484+0000 7f024172f640 1 -- 172.21.15.22:0/2522093311 shutdown_connections 2024-03-29T04:17:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.484+0000 7f024172f640 1 -- 172.21.15.22:0/2522093311 wait complete. 2024-03-29T04:17:54.480 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-29T04:17:54.480 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:17:54.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.608+0000 7f1d9b8ec640 1 Processor -- start 2024-03-29T04:17:54.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.608+0000 7f1d9b8ec640 1 -- start start 2024-03-29T04:17:54.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.608+0000 7f1d9b8ec640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1d9415e600 0x7f1d940b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:54.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.608+0000 7f1d9b8ec640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f1d940d1100 0x7f1d940d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:54.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.608+0000 7f1d9b8ec640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d940d3e60 con 0x7f1d940b8160 2024-03-29T04:17:54.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.608+0000 7f1d9b8ec640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f1d940b7450 con 0x7f1d9415e600 2024-03-29T04:17:54.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.608+0000 7f1d9b8ec640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f1d940d1aa0 con 0x7f1d940d1100 2024-03-29T04:17:54.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.612+0000 7f1d99688640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1d9415e600 0x7f1d940b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:54.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.612+0000 7f1d98e87640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f1d940d1100 0x7f1d940d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:54.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.612+0000 7f1d99688640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1d9415e600 0x7f1d940b9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49562/0 (socket says 172.21.15.22:49562) 2024-03-29T04:17:54.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.612+0000 7f1d98e87640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f1d940d1100 0x7f1d940d1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:44482/0 (socket says 172.21.15.22:44482) 2024-03-29T04:17:54.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.612+0000 7f1d99688640 1 -- 172.21.15.22:0/4272636430 learned_addr learned my addr 172.21.15.22:0/4272636430 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:54.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.612+0000 7f1d99688640 1 -- 172.21.15.22:0/4272636430 >> v2:172.21.15.151:3300/0 conn(0x7f1d940d1100 msgr2=0x7f1d940d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:54.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.612+0000 7f1d99688640 1 --2- 172.21.15.22:0/4272636430 >> v2:172.21.15.151:3300/0 conn(0x7f1d940d1100 0x7f1d940d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.612+0000 7f1d99688640 1 -- 172.21.15.22:0/4272636430 >> v1:172.21.15.151:6789/0 conn(0x7f1d940b8160 legacy=0x7f1d940b85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:54.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.612+0000 7f1d99688640 1 -- 172.21.15.22:0/4272636430 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d940b7680 con 0x7f1d9415e600 2024-03-29T04:17:54.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.612+0000 7f1d99688640 1 --2- 172.21.15.22:0/4272636430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d9415e600 0x7f1d940b9f00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1d940d1df0 tx=0x7f1d8401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51012cae84358fa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:54.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.612+0000 7f1d8bfff640 1 -- 172.21.15.22:0/4272636430 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d840ab610 con 0x7f1d9415e600 2024-03-29T04:17:54.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.612+0000 7f1d8bfff640 1 -- 172.21.15.22:0/4272636430 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d840b0070 con 0x7f1d9415e600 2024-03-29T04:17:54.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.612+0000 7f1d8bfff640 1 -- 172.21.15.22:0/4272636430 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d840ab9d0 con 0x7f1d9415e600 2024-03-29T04:17:54.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.612+0000 7f1d9b8ec640 1 -- 172.21.15.22:0/4272636430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d9415e600 msgr2=0x7f1d940b9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:54.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.612+0000 7f1d9b8ec640 1 --2- 172.21.15.22:0/4272636430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d9415e600 0x7f1d940b9f00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1d940d1df0 tx=0x7f1d8401c520 comp rx=0 tx=0).stop 2024-03-29T04:17:54.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.612+0000 7f1d9b8ec640 1 -- 172.21.15.22:0/4272636430 shutdown_connections 2024-03-29T04:17:54.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.612+0000 7f1d9b8ec640 1 --2- 172.21.15.22:0/4272636430 >> v2:172.21.15.151:3300/0 conn(0x7f1d940d1100 0x7f1d940d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.612+0000 7f1d9b8ec640 1 --2- 172.21.15.22:0/4272636430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d9415e600 0x7f1d940b9f00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.612+0000 7f1d9b8ec640 1 -- 172.21.15.22:0/4272636430 >> 172.21.15.22:0/4272636430 conn(0x7f1d940b6230 msgr2=0x7f1d940b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:54.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.612+0000 7f1d9b8ec640 1 -- 172.21.15.22:0/4272636430 shutdown_connections 2024-03-29T04:17:54.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.612+0000 7f1d9b8ec640 1 -- 172.21.15.22:0/4272636430 wait complete. 2024-03-29T04:17:54.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d9b8ec640 1 Processor -- start 2024-03-29T04:17:54.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d9b8ec640 1 -- start start 2024-03-29T04:17:54.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d9b8ec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d940b8160 0x7f1d941f8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:54.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d9b8ec640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d940d1100 0x7f1d9414c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:54.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d9b8ec640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d9415e600 0x7f1d940c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:54.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d9b8ec640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9416b150 con 0x7f1d940b8160 2024-03-29T04:17:54.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d9b8ec640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9416afd0 con 0x7f1d940d1100 2024-03-29T04:17:54.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d9b8ec640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1d9416b2d0 con 0x7f1d9415e600 2024-03-29T04:17:54.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d99688640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d940b8160 0x7f1d941f8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:54.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d99688640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d940b8160 0x7f1d941f8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49570/0 (socket says 172.21.15.22:49570) 2024-03-29T04:17:54.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d99688640 1 -- 172.21.15.22:0/49712702 learned_addr learned my addr 172.21.15.22:0/49712702 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:54.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d98e87640 1 --2- 172.21.15.22:0/49712702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d940d1100 0x7f1d9414c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:54.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d99688640 1 -- 172.21.15.22:0/49712702 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d9415e600 msgr2=0x7f1d940c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:54.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d99688640 1 --2- 172.21.15.22:0/49712702 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d9415e600 0x7f1d940c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d99688640 1 -- 172.21.15.22:0/49712702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d940d1100 msgr2=0x7f1d9414c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:54.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d99688640 1 --2- 172.21.15.22:0/49712702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d940d1100 0x7f1d9414c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d99688640 1 -- 172.21.15.22:0/49712702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d940c85a0 con 0x7f1d940b8160 2024-03-29T04:17:54.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d98e87640 1 --2- 172.21.15.22:0/49712702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d940d1100 0x7f1d9414c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:54.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d99688640 1 --2- 172.21.15.22:0/49712702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d940b8160 0x7f1d941f8330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1d840ad040 tx=0x7f1d8401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:54.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d89ffb640 1 -- 172.21.15.22:0/49712702 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d840be5f0 con 0x7f1d940b8160 2024-03-29T04:17:54.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d89ffb640 1 -- 172.21.15.22:0/49712702 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d840be7b0 con 0x7f1d940b8160 2024-03-29T04:17:54.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d9b8ec640 1 -- 172.21.15.22:0/49712702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d940c8880 con 0x7f1d940b8160 2024-03-29T04:17:54.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d89ffb640 1 -- 172.21.15.22:0/49712702 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d840beba0 con 0x7f1d940b8160 2024-03-29T04:17:54.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d9b8ec640 1 -- 172.21.15.22:0/49712702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d940c8e40 con 0x7f1d940b8160 2024-03-29T04:17:54.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d9b8ec640 1 -- 172.21.15.22:0/49712702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d94068890 con 0x7f1d940b8160 2024-03-29T04:17:54.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d89ffb640 1 -- 172.21.15.22:0/49712702 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1d840b43f0 con 0x7f1d940b8160 2024-03-29T04:17:54.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.616+0000 7f1d89ffb640 1 --2- 172.21.15.22:0/49712702 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1d78067fc0 0x7f1d7806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:54.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.620+0000 7f1d89ffb640 1 -- 172.21.15.22:0/49712702 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1d84120d50 con 0x7f1d940b8160 2024-03-29T04:17:54.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.620+0000 7f1d98e87640 1 --2- 172.21.15.22:0/49712702 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1d78067fc0 0x7f1d7806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:54.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.620+0000 7f1d98e87640 1 --2- 172.21.15.22:0/49712702 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1d78067fc0 0x7f1d7806a480 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1d9415dc80 tx=0x7f1d90008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:54.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.624+0000 7f1d89ffb640 1 -- 172.21.15.22:0/49712702 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1d84124030 con 0x7f1d940b8160 2024-03-29T04:17:54.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.780+0000 7f1d9b8ec640 1 -- 172.21.15.22:0/49712702 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1d9406d440 con 0x7f1d78067fc0 2024-03-29T04:17:54.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.792+0000 7f1d89ffb640 1 -- 172.21.15.22:0/49712702 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332558 (secure 0 0 0) 0x7f1d9406d440 con 0x7f1d78067fc0 2024-03-29T04:17:54.778 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:54.780 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-29T04:17:54.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.796+0000 7f1d9b8ec640 1 -- 172.21.15.22:0/49712702 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1d78067fc0 msgr2=0x7f1d7806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:54.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.796+0000 7f1d9b8ec640 1 --2- 172.21.15.22:0/49712702 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1d78067fc0 0x7f1d7806a480 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1d9415dc80 tx=0x7f1d90008040 comp rx=0 tx=0).stop 2024-03-29T04:17:54.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.796+0000 7f1d9b8ec640 1 -- 172.21.15.22:0/49712702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d940b8160 msgr2=0x7f1d941f8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:54.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.796+0000 7f1d9b8ec640 1 --2- 172.21.15.22:0/49712702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d940b8160 0x7f1d941f8330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1d840ad040 tx=0x7f1d8401ad60 comp rx=0 tx=0).stop 2024-03-29T04:17:54.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.796+0000 7f1d9b8ec640 1 -- 172.21.15.22:0/49712702 shutdown_connections 2024-03-29T04:17:54.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.796+0000 7f1d9b8ec640 1 --2- 172.21.15.22:0/49712702 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d9415e600 0x7f1d940c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.796+0000 7f1d9b8ec640 1 --2- 172.21.15.22:0/49712702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d940d1100 0x7f1d9414c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.796+0000 7f1d9b8ec640 1 --2- 172.21.15.22:0/49712702 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1d78067fc0 0x7f1d7806a480 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.796+0000 7f1d9b8ec640 1 --2- 172.21.15.22:0/49712702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d940b8160 0x7f1d941f8330 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.796+0000 7f1d9b8ec640 1 -- 172.21.15.22:0/49712702 >> 172.21.15.22:0/49712702 conn(0x7f1d940b6230 msgr2=0x7f1d9414f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:54.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.796+0000 7f1d9b8ec640 1 -- 172.21.15.22:0/49712702 shutdown_connections 2024-03-29T04:17:54.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.796+0000 7f1d9b8ec640 1 -- 172.21.15.22:0/49712702 wait complete. 2024-03-29T04:17:54.798 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-03-29T04:17:53.697360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236208,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738512,"statfs":{"total":773094113280,"available":772852236288,"internally_reserved":0,"allocated":3481600,"data_stored":2086528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.043496"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653655+0000","last_change":"2024-03-29T04:17:35.587951+0000","last_active":"2024-03-29T04:17:42.653655+0000","last_peered":"2024-03-29T04:17:42.653655+0000","last_clean":"2024-03-29T04:17:42.653655+0000","last_became_active":"2024-03-29T04:17:35.587644+0000","last_became_peered":"2024-03-29T04:17:35.587644+0000","last_unstale":"2024-03-29T04:17:42.653655+0000","last_undegraded":"2024-03-29T04:17:42.653655+0000","last_fullsized":"2024-03-29T04:17:42.653655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:07.631047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650468+0000","last_change":"2024-03-29T04:17:36.604695+0000","last_active":"2024-03-29T04:17:42.650468+0000","last_peered":"2024-03-29T04:17:42.650468+0000","last_clean":"2024-03-29T04:17:42.650468+0000","last_became_active":"2024-03-29T04:17:36.602172+0000","last_became_peered":"2024-03-29T04:17:36.602172+0000","last_unstale":"2024-03-29T04:17:42.650468+0000","last_undegraded":"2024-03-29T04:17:42.650468+0000","last_fullsized":"2024-03-29T04:17:42.650468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:30.729030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649719+0000","last_change":"2024-03-29T04:17:41.168150+0000","last_active":"2024-03-29T04:17:42.649719+0000","last_peered":"2024-03-29T04:17:42.649719+0000","last_clean":"2024-03-29T04:17:42.649719+0000","last_became_active":"2024-03-29T04:17:41.166854+0000","last_became_peered":"2024-03-29T04:17:41.166854+0000","last_unstale":"2024-03-29T04:17:42.649719+0000","last_undegraded":"2024-03-29T04:17:42.649719+0000","last_fullsized":"2024-03-29T04:17:42.649719+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:01:07.401841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650673+0000","last_change":"2024-03-29T04:17:35.575675+0000","last_active":"2024-03-29T04:17:42.650673+0000","last_peered":"2024-03-29T04:17:42.650673+0000","last_clean":"2024-03-29T04:17:42.650673+0000","last_became_active":"2024-03-29T04:17:35.575189+0000","last_became_peered":"2024-03-29T04:17:35.575189+0000","last_unstale":"2024-03-29T04:17:42.650673+0000","last_undegraded":"2024-03-29T04:17:42.650673+0000","last_fullsized":"2024-03-29T04:17:42.650673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:55:50.253366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653534+0000","last_change":"2024-03-29T04:17:36.598621+0000","last_active":"2024-03-29T04:17:42.653534+0000","last_peered":"2024-03-29T04:17:42.653534+0000","last_clean":"2024-03-29T04:17:42.653534+0000","last_became_active":"2024-03-29T04:17:36.596823+0000","last_became_peered":"2024-03-29T04:17:36.596823+0000","last_unstale":"2024-03-29T04:17:42.653534+0000","last_undegraded":"2024-03-29T04:17:42.653534+0000","last_fullsized":"2024-03-29T04:17:42.653534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:11.037419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656537+0000","last_change":"2024-03-29T04:17:40.663898+0000","last_active":"2024-03-29T04:17:42.656537+0000","last_peered":"2024-03-29T04:17:42.656537+0000","last_clean":"2024-03-29T04:17:42.656537+0000","last_became_active":"2024-03-29T04:17:40.663387+0000","last_became_peered":"2024-03-29T04:17:40.663387+0000","last_unstale":"2024-03-29T04:17:42.656537+0000","last_undegraded":"2024-03-29T04:17:42.656537+0000","last_fullsized":"2024-03-29T04:17:42.656537+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:46.993432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.635633+0000","last_change":"2024-03-29T04:17:35.604714+0000","last_active":"2024-03-29T04:17:44.635633+0000","last_peered":"2024-03-29T04:17:44.635633+0000","last_clean":"2024-03-29T04:17:44.635633+0000","last_became_active":"2024-03-29T04:17:35.604080+0000","last_became_peered":"2024-03-29T04:17:35.604080+0000","last_unstale":"2024-03-29T04:17:44.635633+0000","last_undegraded":"2024-03-29T04:17:44.635633+0000","last_fullsized":"2024-03-29T04:17:44.635633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:12:36.590426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553394+0000","last_change":"2024-03-29T04:17:36.615673+0000","last_active":"2024-03-29T04:17:43.553394+0000","last_peered":"2024-03-29T04:17:43.553394+0000","last_clean":"2024-03-29T04:17:43.553394+0000","last_became_active":"2024-03-29T04:17:36.615348+0000","last_became_peered":"2024-03-29T04:17:36.615348+0000","last_unstale":"2024-03-29T04:17:43.553394+0000","last_undegraded":"2024-03-29T04:17:43.553394+0000","last_fullsized":"2024-03-29T04:17:43.553394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:30.729030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656960+0000","last_change":"2024-03-29T04:17:40.653544+0000","last_active":"2024-03-29T04:17:42.656960+0000","last_peered":"2024-03-29T04:17:42.656960+0000","last_clean":"2024-03-29T04:17:42.656960+0000","last_became_active":"2024-03-29T04:17:40.652980+0000","last_became_peered":"2024-03-29T04:17:40.652980+0000","last_unstale":"2024-03-29T04:17:42.656960+0000","last_undegraded":"2024-03-29T04:17:42.656960+0000","last_fullsized":"2024-03-29T04:17:42.656960+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:25:02.717791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634331+0000","last_change":"2024-03-29T04:17:35.604849+0000","last_active":"2024-03-29T04:17:44.634331+0000","last_peered":"2024-03-29T04:17:44.634331+0000","last_clean":"2024-03-29T04:17:44.634331+0000","last_became_active":"2024-03-29T04:17:35.604004+0000","last_became_peered":"2024-03-29T04:17:35.604004+0000","last_unstale":"2024-03-29T04:17:44.634331+0000","last_undegraded":"2024-03-29T04:17:44.634331+0000","last_fullsized":"2024-03-29T04:17:44.634331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:50:11.912084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552984+0000","last_change":"2024-03-29T04:17:36.592055+0000","last_active":"2024-03-29T04:17:43.552984+0000","last_peered":"2024-03-29T04:17:43.552984+0000","last_clean":"2024-03-29T04:17:43.552984+0000","last_became_active":"2024-03-29T04:17:36.588805+0000","last_became_peered":"2024-03-29T04:17:36.588805+0000","last_unstale":"2024-03-29T04:17:43.552984+0000","last_undegraded":"2024-03-29T04:17:43.552984+0000","last_fullsized":"2024-03-29T04:17:43.552984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:13.677092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656938+0000","last_change":"2024-03-29T04:17:40.651131+0000","last_active":"2024-03-29T04:17:42.656938+0000","last_peered":"2024-03-29T04:17:42.656938+0000","last_clean":"2024-03-29T04:17:42.656938+0000","last_became_active":"2024-03-29T04:17:40.650319+0000","last_became_peered":"2024-03-29T04:17:40.650319+0000","last_unstale":"2024-03-29T04:17:42.656938+0000","last_undegraded":"2024-03-29T04:17:42.656938+0000","last_fullsized":"2024-03-29T04:17:42.656938+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:42:25.113757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.269694+0000","last_change":"2024-03-29T04:17:36.620761+0000","last_active":"2024-03-29T04:17:44.269694+0000","last_peered":"2024-03-29T04:17:44.269694+0000","last_clean":"2024-03-29T04:17:44.269694+0000","last_became_active":"2024-03-29T04:17:36.620483+0000","last_became_peered":"2024-03-29T04:17:36.620483+0000","last_unstale":"2024-03-29T04:17:44.269694+0000","last_undegraded":"2024-03-29T04:17:44.269694+0000","last_fullsized":"2024-03-29T04:17:44.269694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:10:00.802551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634192+0000","last_change":"2024-03-29T04:17:35.602583+0000","last_active":"2024-03-29T04:17:44.634192+0000","last_peered":"2024-03-29T04:17:44.634192+0000","last_clean":"2024-03-29T04:17:44.634192+0000","last_became_active":"2024-03-29T04:17:35.601902+0000","last_became_peered":"2024-03-29T04:17:35.601902+0000","last_unstale":"2024-03-29T04:17:44.634192+0000","last_undegraded":"2024-03-29T04:17:44.634192+0000","last_fullsized":"2024-03-29T04:17:44.634192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:43:57.713086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.658621+0000","last_change":"2024-03-29T04:17:40.661925+0000","last_active":"2024-03-29T04:17:42.658621+0000","last_peered":"2024-03-29T04:17:42.658621+0000","last_clean":"2024-03-29T04:17:42.658621+0000","last_became_active":"2024-03-29T04:17:40.661028+0000","last_became_peered":"2024-03-29T04:17:40.661028+0000","last_unstale":"2024-03-29T04:17:42.658621+0000","last_undegraded":"2024-03-29T04:17:42.658621+0000","last_fullsized":"2024-03-29T04:17:42.658621+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:33:27.957344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650137+0000","last_change":"2024-03-29T04:17:36.620569+0000","last_active":"2024-03-29T04:17:42.650137+0000","last_peered":"2024-03-29T04:17:42.650137+0000","last_clean":"2024-03-29T04:17:42.650137+0000","last_became_active":"2024-03-29T04:17:36.620079+0000","last_became_peered":"2024-03-29T04:17:36.620079+0000","last_unstale":"2024-03-29T04:17:42.650137+0000","last_undegraded":"2024-03-29T04:17:42.650137+0000","last_fullsized":"2024-03-29T04:17:42.650137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:41.872399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.469887+0000","last_change":"2024-03-29T04:17:35.577167+0000","last_active":"2024-03-29T04:17:44.469887+0000","last_peered":"2024-03-29T04:17:44.469887+0000","last_clean":"2024-03-29T04:17:44.469887+0000","last_became_active":"2024-03-29T04:17:35.576145+0000","last_became_peered":"2024-03-29T04:17:35.576145+0000","last_unstale":"2024-03-29T04:17:44.469887+0000","last_undegraded":"2024-03-29T04:17:44.469887+0000","last_fullsized":"2024-03-29T04:17:44.469887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:11:43.663978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.657612+0000","last_change":"2024-03-29T04:17:40.664962+0000","last_active":"2024-03-29T04:17:42.657612+0000","last_peered":"2024-03-29T04:17:42.657612+0000","last_clean":"2024-03-29T04:17:42.657612+0000","last_became_active":"2024-03-29T04:17:40.664470+0000","last_became_peered":"2024-03-29T04:17:40.664470+0000","last_unstale":"2024-03-29T04:17:42.657612+0000","last_undegraded":"2024-03-29T04:17:42.657612+0000","last_fullsized":"2024-03-29T04:17:42.657612+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:43:44.093178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650038+0000","last_change":"2024-03-29T04:17:36.617517+0000","last_active":"2024-03-29T04:17:42.650038+0000","last_peered":"2024-03-29T04:17:42.650038+0000","last_clean":"2024-03-29T04:17:42.650038+0000","last_became_active":"2024-03-29T04:17:36.616399+0000","last_became_peered":"2024-03-29T04:17:36.616399+0000","last_unstale":"2024-03-29T04:17:42.650038+0000","last_undegraded":"2024-03-29T04:17:42.650038+0000","last_fullsized":"2024-03-29T04:17:42.650038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:24:47.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470278+0000","last_change":"2024-03-29T04:17:35.576729+0000","last_active":"2024-03-29T04:17:44.470278+0000","last_peered":"2024-03-29T04:17:44.470278+0000","last_clean":"2024-03-29T04:17:44.470278+0000","last_became_active":"2024-03-29T04:17:35.576446+0000","last_became_peered":"2024-03-29T04:17:35.576446+0000","last_unstale":"2024-03-29T04:17:44.470278+0000","last_undegraded":"2024-03-29T04:17:44.470278+0000","last_fullsized":"2024-03-29T04:17:44.470278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:53.508449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470815+0000","last_change":"2024-03-29T04:17:41.170866+0000","last_active":"2024-03-29T04:17:44.470815+0000","last_peered":"2024-03-29T04:17:44.470815+0000","last_clean":"2024-03-29T04:17:44.470815+0000","last_became_active":"2024-03-29T04:17:41.169521+0000","last_became_peered":"2024-03-29T04:17:41.169521+0000","last_unstale":"2024-03-29T04:17:44.470815+0000","last_undegraded":"2024-03-29T04:17:44.470815+0000","last_fullsized":"2024-03-29T04:17:44.470815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:33:27.957344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.657095+0000","last_change":"2024-03-29T04:17:36.624392+0000","last_active":"2024-03-29T04:17:42.657095+0000","last_peered":"2024-03-29T04:17:42.657095+0000","last_clean":"2024-03-29T04:17:42.657095+0000","last_became_active":"2024-03-29T04:17:36.612708+0000","last_became_peered":"2024-03-29T04:17:36.612708+0000","last_unstale":"2024-03-29T04:17:42.657095+0000","last_undegraded":"2024-03-29T04:17:42.657095+0000","last_fullsized":"2024-03-29T04:17:42.657095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:59.355649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634144+0000","last_change":"2024-03-29T04:17:35.596985+0000","last_active":"2024-03-29T04:17:44.634144+0000","last_peered":"2024-03-29T04:17:44.634144+0000","last_clean":"2024-03-29T04:17:44.634144+0000","last_became_active":"2024-03-29T04:17:35.595917+0000","last_became_peered":"2024-03-29T04:17:35.595917+0000","last_unstale":"2024-03-29T04:17:44.634144+0000","last_undegraded":"2024-03-29T04:17:44.634144+0000","last_fullsized":"2024-03-29T04:17:44.634144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:24:46.122034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.658384+0000","last_change":"2024-03-29T04:17:40.664618+0000","last_active":"2024-03-29T04:17:42.658384+0000","last_peered":"2024-03-29T04:17:42.658384+0000","last_clean":"2024-03-29T04:17:42.658384+0000","last_became_active":"2024-03-29T04:17:40.664144+0000","last_became_peered":"2024-03-29T04:17:40.664144+0000","last_unstale":"2024-03-29T04:17:42.658384+0000","last_undegraded":"2024-03-29T04:17:42.658384+0000","last_fullsized":"2024-03-29T04:17:42.658384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:59:47.099511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653827+0000","last_change":"2024-03-29T04:17:36.596925+0000","last_active":"2024-03-29T04:17:42.653827+0000","last_peered":"2024-03-29T04:17:42.653827+0000","last_clean":"2024-03-29T04:17:42.653827+0000","last_became_active":"2024-03-29T04:17:36.596307+0000","last_became_peered":"2024-03-29T04:17:36.596307+0000","last_unstale":"2024-03-29T04:17:42.653827+0000","last_undegraded":"2024-03-29T04:17:42.653827+0000","last_fullsized":"2024-03-29T04:17:42.653827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:39.635039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.271188+0000","last_change":"2024-03-29T04:17:35.598885+0000","last_active":"2024-03-29T04:17:44.271188+0000","last_peered":"2024-03-29T04:17:44.271188+0000","last_clean":"2024-03-29T04:17:44.271188+0000","last_became_active":"2024-03-29T04:17:35.598155+0000","last_became_peered":"2024-03-29T04:17:35.598155+0000","last_unstale":"2024-03-29T04:17:44.271188+0000","last_undegraded":"2024-03-29T04:17:44.271188+0000","last_fullsized":"2024-03-29T04:17:44.271188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:58.346822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656759+0000","last_change":"2024-03-29T04:17:41.164185+0000","last_active":"2024-03-29T04:17:42.656759+0000","last_peered":"2024-03-29T04:17:42.656759+0000","last_clean":"2024-03-29T04:17:42.656759+0000","last_became_active":"2024-03-29T04:17:41.163368+0000","last_became_peered":"2024-03-29T04:17:41.163368+0000","last_unstale":"2024-03-29T04:17:42.656759+0000","last_undegraded":"2024-03-29T04:17:42.656759+0000","last_fullsized":"2024-03-29T04:17:42.656759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:04:15.384842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650724+0000","last_change":"2024-03-29T04:17:36.611583+0000","last_active":"2024-03-29T04:17:42.650724+0000","last_peered":"2024-03-29T04:17:42.650724+0000","last_clean":"2024-03-29T04:17:42.650724+0000","last_became_active":"2024-03-29T04:17:36.611204+0000","last_became_peered":"2024-03-29T04:17:36.611204+0000","last_unstale":"2024-03-29T04:17:42.650724+0000","last_undegraded":"2024-03-29T04:17:42.650724+0000","last_fullsized":"2024-03-29T04:17:42.650724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:13.677092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470861+0000","last_change":"2024-03-29T04:17:35.574541+0000","last_active":"2024-03-29T04:17:44.470861+0000","last_peered":"2024-03-29T04:17:44.470861+0000","last_clean":"2024-03-29T04:17:44.470861+0000","last_became_active":"2024-03-29T04:17:35.573297+0000","last_became_peered":"2024-03-29T04:17:35.573297+0000","last_unstale":"2024-03-29T04:17:44.470861+0000","last_undegraded":"2024-03-29T04:17:44.470861+0000","last_fullsized":"2024-03-29T04:17:44.470861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:58.346822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649700+0000","last_change":"2024-03-29T04:17:41.165155+0000","last_active":"2024-03-29T04:17:42.649700+0000","last_peered":"2024-03-29T04:17:42.649700+0000","last_clean":"2024-03-29T04:17:42.649700+0000","last_became_active":"2024-03-29T04:17:41.163801+0000","last_became_peered":"2024-03-29T04:17:41.163801+0000","last_unstale":"2024-03-29T04:17:42.649700+0000","last_undegraded":"2024-03-29T04:17:42.649700+0000","last_fullsized":"2024-03-29T04:17:42.649700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:05:37.595239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470679+0000","last_change":"2024-03-29T04:17:36.590406+0000","last_active":"2024-03-29T04:17:44.470679+0000","last_peered":"2024-03-29T04:17:44.470679+0000","last_clean":"2024-03-29T04:17:44.470679+0000","last_became_active":"2024-03-29T04:17:36.589975+0000","last_became_peered":"2024-03-29T04:17:36.589975+0000","last_unstale":"2024-03-29T04:17:44.470679+0000","last_undegraded":"2024-03-29T04:17:44.470679+0000","last_fullsized":"2024-03-29T04:17:44.470679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:39.635039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634555+0000","last_change":"2024-03-29T04:17:35.602939+0000","last_active":"2024-03-29T04:17:44.634555+0000","last_peered":"2024-03-29T04:17:44.634555+0000","last_clean":"2024-03-29T04:17:44.634555+0000","last_became_active":"2024-03-29T04:17:35.602250+0000","last_became_peered":"2024-03-29T04:17:35.602250+0000","last_unstale":"2024-03-29T04:17:44.634555+0000","last_undegraded":"2024-03-29T04:17:44.634555+0000","last_fullsized":"2024-03-29T04:17:44.634555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:16:13.431728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653145+0000","last_change":"2024-03-29T04:17:41.165561+0000","last_active":"2024-03-29T04:17:42.653145+0000","last_peered":"2024-03-29T04:17:42.653145+0000","last_clean":"2024-03-29T04:17:42.653145+0000","last_became_active":"2024-03-29T04:17:41.163703+0000","last_became_peered":"2024-03-29T04:17:41.163703+0000","last_unstale":"2024-03-29T04:17:42.653145+0000","last_undegraded":"2024-03-29T04:17:42.653145+0000","last_fullsized":"2024-03-29T04:17:42.653145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:10:04.139549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553157+0000","last_change":"2024-03-29T04:17:36.602865+0000","last_active":"2024-03-29T04:17:43.553157+0000","last_peered":"2024-03-29T04:17:43.553157+0000","last_clean":"2024-03-29T04:17:43.553157+0000","last_became_active":"2024-03-29T04:17:36.602578+0000","last_became_peered":"2024-03-29T04:17:36.602578+0000","last_unstale":"2024-03-29T04:17:43.553157+0000","last_undegraded":"2024-03-29T04:17:43.553157+0000","last_fullsized":"2024-03-29T04:17:43.553157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:55:11.413735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.469877+0000","last_change":"2024-03-29T04:17:35.560382+0000","last_active":"2024-03-29T04:17:44.469877+0000","last_peered":"2024-03-29T04:17:44.469877+0000","last_clean":"2024-03-29T04:17:44.469877+0000","last_became_active":"2024-03-29T04:17:35.560122+0000","last_became_peered":"2024-03-29T04:17:35.560122+0000","last_unstale":"2024-03-29T04:17:44.469877+0000","last_undegraded":"2024-03-29T04:17:44.469877+0000","last_fullsized":"2024-03-29T04:17:44.469877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:52:18.910727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470397+0000","last_change":"2024-03-29T04:17:40.663870+0000","last_active":"2024-03-29T04:17:44.470397+0000","last_peered":"2024-03-29T04:17:44.470397+0000","last_clean":"2024-03-29T04:17:44.470397+0000","last_became_active":"2024-03-29T04:17:40.662957+0000","last_became_peered":"2024-03-29T04:17:40.662957+0000","last_unstale":"2024-03-29T04:17:44.470397+0000","last_undegraded":"2024-03-29T04:17:44.470397+0000","last_fullsized":"2024-03-29T04:17:44.470397+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:10:04.139549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.657408+0000","last_change":"2024-03-29T04:17:36.624632+0000","last_active":"2024-03-29T04:17:42.657408+0000","last_peered":"2024-03-29T04:17:42.657408+0000","last_clean":"2024-03-29T04:17:42.657408+0000","last_became_active":"2024-03-29T04:17:36.621308+0000","last_became_peered":"2024-03-29T04:17:36.621308+0000","last_unstale":"2024-03-29T04:17:42.657408+0000","last_undegraded":"2024-03-29T04:17:42.657408+0000","last_fullsized":"2024-03-29T04:17:42.657408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:11:44.672805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553434+0000","last_change":"2024-03-29T04:17:35.590306+0000","last_active":"2024-03-29T04:17:43.553434+0000","last_peered":"2024-03-29T04:17:43.553434+0000","last_clean":"2024-03-29T04:17:43.553434+0000","last_became_active":"2024-03-29T04:17:35.582422+0000","last_became_peered":"2024-03-29T04:17:35.582422+0000","last_unstale":"2024-03-29T04:17:43.553434+0000","last_undegraded":"2024-03-29T04:17:43.553434+0000","last_fullsized":"2024-03-29T04:17:43.553434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:58.346822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470758+0000","last_change":"2024-03-29T04:17:40.661919+0000","last_active":"2024-03-29T04:17:44.470758+0000","last_peered":"2024-03-29T04:17:44.470758+0000","last_clean":"2024-03-29T04:17:44.470758+0000","last_became_active":"2024-03-29T04:17:40.661251+0000","last_became_peered":"2024-03-29T04:17:40.661251+0000","last_unstale":"2024-03-29T04:17:44.470758+0000","last_undegraded":"2024-03-29T04:17:44.470758+0000","last_fullsized":"2024-03-29T04:17:44.470758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:59:47.099511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470257+0000","last_change":"2024-03-29T04:17:36.590933+0000","last_active":"2024-03-29T04:17:44.470257+0000","last_peered":"2024-03-29T04:17:44.470257+0000","last_clean":"2024-03-29T04:17:44.470257+0000","last_became_active":"2024-03-29T04:17:36.590535+0000","last_became_peered":"2024-03-29T04:17:36.590535+0000","last_unstale":"2024-03-29T04:17:44.470257+0000","last_undegraded":"2024-03-29T04:17:44.470257+0000","last_fullsized":"2024-03-29T04:17:44.470257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:41.872399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578021+0000","last_change":"2024-03-29T04:17:35.565257+0000","last_active":"2024-03-29T04:17:43.578021+0000","last_peered":"2024-03-29T04:17:43.578021+0000","last_clean":"2024-03-29T04:17:43.578021+0000","last_became_active":"2024-03-29T04:17:35.564547+0000","last_became_peered":"2024-03-29T04:17:35.564547+0000","last_unstale":"2024-03-29T04:17:43.578021+0000","last_undegraded":"2024-03-29T04:17:43.578021+0000","last_fullsized":"2024-03-29T04:17:43.578021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:04:44.150824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470921+0000","last_change":"2024-03-29T04:17:40.657201+0000","last_active":"2024-03-29T04:17:44.470921+0000","last_peered":"2024-03-29T04:17:44.470921+0000","last_clean":"2024-03-29T04:17:44.470921+0000","last_became_active":"2024-03-29T04:17:40.655333+0000","last_became_peered":"2024-03-29T04:17:40.655333+0000","last_unstale":"2024-03-29T04:17:44.470921+0000","last_undegraded":"2024-03-29T04:17:44.470921+0000","last_fullsized":"2024-03-29T04:17:44.470921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:37:53.228170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.652829+0000","last_change":"2024-03-29T04:17:36.625393+0000","last_active":"2024-03-29T04:17:42.652829+0000","last_peered":"2024-03-29T04:17:42.652829+0000","last_clean":"2024-03-29T04:17:42.652829+0000","last_became_active":"2024-03-29T04:17:36.625053+0000","last_became_peered":"2024-03-29T04:17:36.625053+0000","last_unstale":"2024-03-29T04:17:42.652829+0000","last_undegraded":"2024-03-29T04:17:42.652829+0000","last_fullsized":"2024-03-29T04:17:42.652829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:16:58.574477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.635249+0000","last_change":"2024-03-29T04:17:35.597695+0000","last_active":"2024-03-29T04:17:44.635249+0000","last_peered":"2024-03-29T04:17:44.635249+0000","last_clean":"2024-03-29T04:17:44.635249+0000","last_became_active":"2024-03-29T04:17:35.596065+0000","last_became_peered":"2024-03-29T04:17:35.596065+0000","last_unstale":"2024-03-29T04:17:44.635249+0000","last_undegraded":"2024-03-29T04:17:44.635249+0000","last_fullsized":"2024-03-29T04:17:44.635249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:11:43.663978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653676+0000","last_change":"2024-03-29T04:17:41.169719+0000","last_active":"2024-03-29T04:17:42.653676+0000","last_peered":"2024-03-29T04:17:42.653676+0000","last_clean":"2024-03-29T04:17:42.653676+0000","last_became_active":"2024-03-29T04:17:41.168241+0000","last_became_peered":"2024-03-29T04:17:41.168241+0000","last_unstale":"2024-03-29T04:17:42.653676+0000","last_undegraded":"2024-03-29T04:17:42.653676+0000","last_fullsized":"2024-03-29T04:17:42.653676+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:37:53.228170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649994+0000","last_change":"2024-03-29T04:17:36.599562+0000","last_active":"2024-03-29T04:17:42.649994+0000","last_peered":"2024-03-29T04:17:42.649994+0000","last_clean":"2024-03-29T04:17:42.649994+0000","last_became_active":"2024-03-29T04:17:36.597078+0000","last_became_peered":"2024-03-29T04:17:36.597078+0000","last_unstale":"2024-03-29T04:17:42.649994+0000","last_undegraded":"2024-03-29T04:17:42.649994+0000","last_fullsized":"2024-03-29T04:17:42.649994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:43:27.568676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553439+0000","last_change":"2024-03-29T04:17:35.589469+0000","last_active":"2024-03-29T04:17:43.553439+0000","last_peered":"2024-03-29T04:17:43.553439+0000","last_clean":"2024-03-29T04:17:43.553439+0000","last_became_active":"2024-03-29T04:17:35.572474+0000","last_became_peered":"2024-03-29T04:17:35.572474+0000","last_unstale":"2024-03-29T04:17:43.553439+0000","last_undegraded":"2024-03-29T04:17:43.553439+0000","last_fullsized":"2024-03-29T04:17:43.553439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:55:50.253366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.635415+0000","last_change":"2024-03-29T04:17:25.361826+0000","last_active":"2024-03-29T04:17:44.635415+0000","last_peered":"2024-03-29T04:17:44.635415+0000","last_clean":"2024-03-29T04:17:44.635415+0000","last_became_active":"2024-03-29T04:17:25.361259+0000","last_became_peered":"2024-03-29T04:17:25.361259+0000","last_unstale":"2024-03-29T04:17:44.635415+0000","last_undegraded":"2024-03-29T04:17:44.635415+0000","last_fullsized":"2024-03-29T04:17:44.635415+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_clean_scrub_stamp":"2024-03-29T04:17:24.334639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:40.027873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270077+0000","last_change":"2024-03-29T04:17:41.167132+0000","last_active":"2024-03-29T04:17:44.270077+0000","last_peered":"2024-03-29T04:17:44.270077+0000","last_clean":"2024-03-29T04:17:44.270077+0000","last_became_active":"2024-03-29T04:17:41.166660+0000","last_became_peered":"2024-03-29T04:17:41.166660+0000","last_unstale":"2024-03-29T04:17:44.270077+0000","last_undegraded":"2024-03-29T04:17:44.270077+0000","last_fullsized":"2024-03-29T04:17:44.270077+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:42:25.113757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634556+0000","last_change":"2024-03-29T04:17:36.612535+0000","last_active":"2024-03-29T04:17:44.634556+0000","last_peered":"2024-03-29T04:17:44.634556+0000","last_clean":"2024-03-29T04:17:44.634556+0000","last_became_active":"2024-03-29T04:17:36.610106+0000","last_became_peered":"2024-03-29T04:17:36.610106+0000","last_unstale":"2024-03-29T04:17:44.634556+0000","last_undegraded":"2024-03-29T04:17:44.634556+0000","last_fullsized":"2024-03-29T04:17:44.634556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:24:58.639606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578939+0000","last_change":"2024-03-29T04:17:35.600209+0000","last_active":"2024-03-29T04:17:43.578939+0000","last_peered":"2024-03-29T04:17:43.578939+0000","last_clean":"2024-03-29T04:17:43.578939+0000","last_became_active":"2024-03-29T04:17:35.599686+0000","last_became_peered":"2024-03-29T04:17:35.599686+0000","last_unstale":"2024-03-29T04:17:43.578939+0000","last_undegraded":"2024-03-29T04:17:43.578939+0000","last_fullsized":"2024-03-29T04:17:43.578939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:55:50.253366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.269503+0000","last_change":"2024-03-29T04:17:41.165901+0000","last_active":"2024-03-29T04:17:44.269503+0000","last_peered":"2024-03-29T04:17:44.269503+0000","last_clean":"2024-03-29T04:17:44.269503+0000","last_became_active":"2024-03-29T04:17:41.165453+0000","last_became_peered":"2024-03-29T04:17:41.165453+0000","last_unstale":"2024-03-29T04:17:44.269503+0000","last_undegraded":"2024-03-29T04:17:44.269503+0000","last_fullsized":"2024-03-29T04:17:44.269503+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:04:15.384842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553553+0000","last_change":"2024-03-29T04:17:36.596650+0000","last_active":"2024-03-29T04:17:43.553553+0000","last_peered":"2024-03-29T04:17:43.553553+0000","last_clean":"2024-03-29T04:17:43.553553+0000","last_became_active":"2024-03-29T04:17:36.596458+0000","last_became_peered":"2024-03-29T04:17:36.596458+0000","last_unstale":"2024-03-29T04:17:43.553553+0000","last_undegraded":"2024-03-29T04:17:43.553553+0000","last_fullsized":"2024-03-29T04:17:43.553553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:42:46.422245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.469894+0000","last_change":"2024-03-29T04:17:35.554693+0000","last_active":"2024-03-29T04:17:44.469894+0000","last_peered":"2024-03-29T04:17:44.469894+0000","last_clean":"2024-03-29T04:17:44.469894+0000","last_became_active":"2024-03-29T04:17:35.554347+0000","last_became_peered":"2024-03-29T04:17:35.554347+0000","last_unstale":"2024-03-29T04:17:44.469894+0000","last_undegraded":"2024-03-29T04:17:44.469894+0000","last_fullsized":"2024-03-29T04:17:44.469894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:24:46.122034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653319+0000","last_change":"2024-03-29T04:17:40.656032+0000","last_active":"2024-03-29T04:17:42.653319+0000","last_peered":"2024-03-29T04:17:42.653319+0000","last_clean":"2024-03-29T04:17:42.653319+0000","last_became_active":"2024-03-29T04:17:40.654947+0000","last_became_peered":"2024-03-29T04:17:40.654947+0000","last_unstale":"2024-03-29T04:17:42.653319+0000","last_undegraded":"2024-03-29T04:17:42.653319+0000","last_fullsized":"2024-03-29T04:17:42.653319+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:59:47.099511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634965+0000","last_change":"2024-03-29T04:17:36.627875+0000","last_active":"2024-03-29T04:17:44.634965+0000","last_peered":"2024-03-29T04:17:44.634965+0000","last_clean":"2024-03-29T04:17:44.634965+0000","last_became_active":"2024-03-29T04:17:36.627498+0000","last_became_peered":"2024-03-29T04:17:36.627498+0000","last_unstale":"2024-03-29T04:17:44.634965+0000","last_undegraded":"2024-03-29T04:17:44.634965+0000","last_fullsized":"2024-03-29T04:17:44.634965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:10:00.061365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578092+0000","last_change":"2024-03-29T04:17:35.568135+0000","last_active":"2024-03-29T04:17:43.578092+0000","last_peered":"2024-03-29T04:17:43.578092+0000","last_clean":"2024-03-29T04:17:43.578092+0000","last_became_active":"2024-03-29T04:17:35.567800+0000","last_became_peered":"2024-03-29T04:17:35.567800+0000","last_unstale":"2024-03-29T04:17:43.578092+0000","last_undegraded":"2024-03-29T04:17:43.578092+0000","last_fullsized":"2024-03-29T04:17:43.578092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:23.283694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552529+0000","last_change":"2024-03-29T04:17:41.166538+0000","last_active":"2024-03-29T04:17:43.552529+0000","last_peered":"2024-03-29T04:17:43.552529+0000","last_clean":"2024-03-29T04:17:43.552529+0000","last_became_active":"2024-03-29T04:17:41.165914+0000","last_became_peered":"2024-03-29T04:17:41.165914+0000","last_unstale":"2024-03-29T04:17:43.552529+0000","last_undegraded":"2024-03-29T04:17:43.552529+0000","last_fullsized":"2024-03-29T04:17:43.552529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:37:53.228170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634358+0000","last_change":"2024-03-29T04:17:36.626068+0000","last_active":"2024-03-29T04:17:44.634358+0000","last_peered":"2024-03-29T04:17:44.634358+0000","last_clean":"2024-03-29T04:17:44.634358+0000","last_became_active":"2024-03-29T04:17:36.625776+0000","last_became_peered":"2024-03-29T04:17:36.625776+0000","last_unstale":"2024-03-29T04:17:44.634358+0000","last_undegraded":"2024-03-29T04:17:44.634358+0000","last_fullsized":"2024-03-29T04:17:44.634358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:55:11.413735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.658456+0000","last_change":"2024-03-29T04:17:35.573245+0000","last_active":"2024-03-29T04:17:42.658456+0000","last_peered":"2024-03-29T04:17:42.658456+0000","last_clean":"2024-03-29T04:17:42.658456+0000","last_became_active":"2024-03-29T04:17:35.572324+0000","last_became_peered":"2024-03-29T04:17:35.572324+0000","last_unstale":"2024-03-29T04:17:42.658456+0000","last_undegraded":"2024-03-29T04:17:42.658456+0000","last_fullsized":"2024-03-29T04:17:42.658456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:07:32.265670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656484+0000","last_change":"2024-03-29T04:17:40.653790+0000","last_active":"2024-03-29T04:17:42.656484+0000","last_peered":"2024-03-29T04:17:42.656484+0000","last_clean":"2024-03-29T04:17:42.656484+0000","last_became_active":"2024-03-29T04:17:40.653115+0000","last_became_peered":"2024-03-29T04:17:40.653115+0000","last_unstale":"2024-03-29T04:17:42.656484+0000","last_undegraded":"2024-03-29T04:17:42.656484+0000","last_fullsized":"2024-03-29T04:17:42.656484+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:55:15.491920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634122+0000","last_change":"2024-03-29T04:17:36.625368+0000","last_active":"2024-03-29T04:17:44.634122+0000","last_peered":"2024-03-29T04:17:44.634122+0000","last_clean":"2024-03-29T04:17:44.634122+0000","last_became_active":"2024-03-29T04:17:36.625073+0000","last_became_peered":"2024-03-29T04:17:36.625073+0000","last_unstale":"2024-03-29T04:17:44.634122+0000","last_undegraded":"2024-03-29T04:17:44.634122+0000","last_fullsized":"2024-03-29T04:17:44.634122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:13.677092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578401+0000","last_change":"2024-03-29T04:17:35.564073+0000","last_active":"2024-03-29T04:17:43.578401+0000","last_peered":"2024-03-29T04:17:43.578401+0000","last_clean":"2024-03-29T04:17:43.578401+0000","last_became_active":"2024-03-29T04:17:35.563521+0000","last_became_peered":"2024-03-29T04:17:35.563521+0000","last_unstale":"2024-03-29T04:17:43.578401+0000","last_undegraded":"2024-03-29T04:17:43.578401+0000","last_fullsized":"2024-03-29T04:17:43.578401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:53.508449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653476+0000","last_change":"2024-03-29T04:17:40.648424+0000","last_active":"2024-03-29T04:17:42.653476+0000","last_peered":"2024-03-29T04:17:42.653476+0000","last_clean":"2024-03-29T04:17:42.653476+0000","last_became_active":"2024-03-29T04:17:40.647665+0000","last_became_peered":"2024-03-29T04:17:40.647665+0000","last_unstale":"2024-03-29T04:17:42.653476+0000","last_undegraded":"2024-03-29T04:17:42.653476+0000","last_fullsized":"2024-03-29T04:17:42.653476+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:33:27.957344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.269744+0000","last_change":"2024-03-29T04:17:36.614419+0000","last_active":"2024-03-29T04:17:44.269744+0000","last_peered":"2024-03-29T04:17:44.269744+0000","last_clean":"2024-03-29T04:17:44.269744+0000","last_became_active":"2024-03-29T04:17:36.614052+0000","last_became_peered":"2024-03-29T04:17:36.614052+0000","last_unstale":"2024-03-29T04:17:44.269744+0000","last_undegraded":"2024-03-29T04:17:44.269744+0000","last_fullsized":"2024-03-29T04:17:44.269744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:41:31.910133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.268906+0000","last_change":"2024-03-29T04:17:35.576576+0000","last_active":"2024-03-29T04:17:44.268906+0000","last_peered":"2024-03-29T04:17:44.268906+0000","last_clean":"2024-03-29T04:17:44.268906+0000","last_became_active":"2024-03-29T04:17:35.575156+0000","last_became_peered":"2024-03-29T04:17:35.575156+0000","last_unstale":"2024-03-29T04:17:44.268906+0000","last_undegraded":"2024-03-29T04:17:44.268906+0000","last_fullsized":"2024-03-29T04:17:44.268906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:52:18.910727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650517+0000","last_change":"2024-03-29T04:17:41.165455+0000","last_active":"2024-03-29T04:17:42.650517+0000","last_peered":"2024-03-29T04:17:42.650517+0000","last_clean":"2024-03-29T04:17:42.650517+0000","last_became_active":"2024-03-29T04:17:41.164305+0000","last_became_peered":"2024-03-29T04:17:41.164305+0000","last_unstale":"2024-03-29T04:17:42.650517+0000","last_undegraded":"2024-03-29T04:17:42.650517+0000","last_fullsized":"2024-03-29T04:17:42.650517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:58:14.948478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634370+0000","last_change":"2024-03-29T04:17:36.626514+0000","last_active":"2024-03-29T04:17:44.634370+0000","last_peered":"2024-03-29T04:17:44.634370+0000","last_clean":"2024-03-29T04:17:44.634370+0000","last_became_active":"2024-03-29T04:17:36.626184+0000","last_became_peered":"2024-03-29T04:17:36.626184+0000","last_unstale":"2024-03-29T04:17:44.634370+0000","last_undegraded":"2024-03-29T04:17:44.634370+0000","last_fullsized":"2024-03-29T04:17:44.634370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:42:46.422245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553262+0000","last_change":"2024-03-29T04:17:35.599574+0000","last_active":"2024-03-29T04:17:43.553262+0000","last_peered":"2024-03-29T04:17:43.553262+0000","last_clean":"2024-03-29T04:17:43.553262+0000","last_became_active":"2024-03-29T04:17:35.589264+0000","last_became_peered":"2024-03-29T04:17:35.589264+0000","last_unstale":"2024-03-29T04:17:43.553262+0000","last_undegraded":"2024-03-29T04:17:43.553262+0000","last_fullsized":"2024-03-29T04:17:43.553262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:52:18.910727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650586+0000","last_change":"2024-03-29T04:17:41.164863+0000","last_active":"2024-03-29T04:17:42.650586+0000","last_peered":"2024-03-29T04:17:42.650586+0000","last_clean":"2024-03-29T04:17:42.650586+0000","last_became_active":"2024-03-29T04:17:41.163756+0000","last_became_peered":"2024-03-29T04:17:41.163756+0000","last_unstale":"2024-03-29T04:17:42.650586+0000","last_undegraded":"2024-03-29T04:17:42.650586+0000","last_fullsized":"2024-03-29T04:17:42.650586+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:54.890778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653149+0000","last_change":"2024-03-29T04:17:36.611832+0000","last_active":"2024-03-29T04:17:42.653149+0000","last_peered":"2024-03-29T04:17:42.653149+0000","last_clean":"2024-03-29T04:17:42.653149+0000","last_became_active":"2024-03-29T04:17:36.611106+0000","last_became_peered":"2024-03-29T04:17:36.611106+0000","last_unstale":"2024-03-29T04:17:42.653149+0000","last_undegraded":"2024-03-29T04:17:42.653149+0000","last_fullsized":"2024-03-29T04:17:42.653149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:10:00.802551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653387+0000","last_change":"2024-03-29T04:17:35.569650+0000","last_active":"2024-03-29T04:17:42.653387+0000","last_peered":"2024-03-29T04:17:42.653387+0000","last_clean":"2024-03-29T04:17:42.653387+0000","last_became_active":"2024-03-29T04:17:35.568995+0000","last_became_peered":"2024-03-29T04:17:35.568995+0000","last_unstale":"2024-03-29T04:17:42.653387+0000","last_undegraded":"2024-03-29T04:17:42.653387+0000","last_fullsized":"2024-03-29T04:17:42.653387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:55:50.253366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552920+0000","last_change":"2024-03-29T04:17:41.166364+0000","last_active":"2024-03-29T04:17:43.552920+0000","last_peered":"2024-03-29T04:17:43.552920+0000","last_clean":"2024-03-29T04:17:43.552920+0000","last_became_active":"2024-03-29T04:17:41.165953+0000","last_became_peered":"2024-03-29T04:17:41.165953+0000","last_unstale":"2024-03-29T04:17:43.552920+0000","last_undegraded":"2024-03-29T04:17:43.552920+0000","last_fullsized":"2024-03-29T04:17:43.552920+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:58.692538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634707+0000","last_change":"2024-03-29T04:17:36.610409+0000","last_active":"2024-03-29T04:17:44.634707+0000","last_peered":"2024-03-29T04:17:44.634707+0000","last_clean":"2024-03-29T04:17:44.634707+0000","last_became_active":"2024-03-29T04:17:36.609783+0000","last_became_peered":"2024-03-29T04:17:36.609783+0000","last_unstale":"2024-03-29T04:17:44.634707+0000","last_undegraded":"2024-03-29T04:17:44.634707+0000","last_fullsized":"2024-03-29T04:17:44.634707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:46:38.204638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:47.106160+0000","last_change":"2024-03-29T04:17:35.576067+0000","last_active":"2024-03-29T04:17:47.106160+0000","last_peered":"2024-03-29T04:17:47.106160+0000","last_clean":"2024-03-29T04:17:47.106160+0000","last_became_active":"2024-03-29T04:17:35.573099+0000","last_became_peered":"2024-03-29T04:17:35.573099+0000","last_unstale":"2024-03-29T04:17:47.106160+0000","last_undegraded":"2024-03-29T04:17:47.106160+0000","last_fullsized":"2024-03-29T04:17:47.106160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:53.508449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470529+0000","last_change":"2024-03-29T04:17:40.658773+0000","last_active":"2024-03-29T04:17:44.470529+0000","last_peered":"2024-03-29T04:17:44.470529+0000","last_clean":"2024-03-29T04:17:44.470529+0000","last_became_active":"2024-03-29T04:17:40.658277+0000","last_became_peered":"2024-03-29T04:17:40.658277+0000","last_unstale":"2024-03-29T04:17:44.470529+0000","last_undegraded":"2024-03-29T04:17:44.470529+0000","last_fullsized":"2024-03-29T04:17:44.470529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:43:44.093178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.658046+0000","last_change":"2024-03-29T04:17:36.620386+0000","last_active":"2024-03-29T04:17:42.658046+0000","last_peered":"2024-03-29T04:17:42.658046+0000","last_clean":"2024-03-29T04:17:42.658046+0000","last_became_active":"2024-03-29T04:17:36.620093+0000","last_became_peered":"2024-03-29T04:17:36.620093+0000","last_unstale":"2024-03-29T04:17:42.658046+0000","last_undegraded":"2024-03-29T04:17:42.658046+0000","last_fullsized":"2024-03-29T04:17:42.658046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:12:37.599253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.469929+0000","last_change":"2024-03-29T04:17:35.592723+0000","last_active":"2024-03-29T04:17:44.469929+0000","last_peered":"2024-03-29T04:17:44.469929+0000","last_clean":"2024-03-29T04:17:44.469929+0000","last_became_active":"2024-03-29T04:17:35.592409+0000","last_became_peered":"2024-03-29T04:17:35.592409+0000","last_unstale":"2024-03-29T04:17:44.469929+0000","last_undegraded":"2024-03-29T04:17:44.469929+0000","last_fullsized":"2024-03-29T04:17:44.469929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:12:36.590426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.657265+0000","last_change":"2024-03-29T04:17:40.654909+0000","last_active":"2024-03-29T04:17:42.657265+0000","last_peered":"2024-03-29T04:17:42.657265+0000","last_clean":"2024-03-29T04:17:42.657265+0000","last_became_active":"2024-03-29T04:17:40.654420+0000","last_became_peered":"2024-03-29T04:17:40.654420+0000","last_unstale":"2024-03-29T04:17:42.657265+0000","last_undegraded":"2024-03-29T04:17:42.657265+0000","last_fullsized":"2024-03-29T04:17:42.657265+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:10:04.139549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653614+0000","last_change":"2024-03-29T04:17:36.599274+0000","last_active":"2024-03-29T04:17:42.653614+0000","last_peered":"2024-03-29T04:17:42.653614+0000","last_clean":"2024-03-29T04:17:42.653614+0000","last_became_active":"2024-03-29T04:17:36.597822+0000","last_became_peered":"2024-03-29T04:17:36.597822+0000","last_unstale":"2024-03-29T04:17:42.653614+0000","last_undegraded":"2024-03-29T04:17:42.653614+0000","last_fullsized":"2024-03-29T04:17:42.653614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:24:47.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553100+0000","last_change":"2024-03-29T04:17:35.592120+0000","last_active":"2024-03-29T04:17:43.553100+0000","last_peered":"2024-03-29T04:17:43.553100+0000","last_clean":"2024-03-29T04:17:43.553100+0000","last_became_active":"2024-03-29T04:17:35.590699+0000","last_became_peered":"2024-03-29T04:17:35.590699+0000","last_unstale":"2024-03-29T04:17:43.553100+0000","last_undegraded":"2024-03-29T04:17:43.553100+0000","last_fullsized":"2024-03-29T04:17:43.553100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:53.508449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649946+0000","last_change":"2024-03-29T04:17:40.661266+0000","last_active":"2024-03-29T04:17:42.649946+0000","last_peered":"2024-03-29T04:17:42.649946+0000","last_clean":"2024-03-29T04:17:42.649946+0000","last_became_active":"2024-03-29T04:17:40.660780+0000","last_became_peered":"2024-03-29T04:17:40.660780+0000","last_unstale":"2024-03-29T04:17:42.649946+0000","last_undegraded":"2024-03-29T04:17:42.649946+0000","last_fullsized":"2024-03-29T04:17:42.649946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:02.859957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552978+0000","last_change":"2024-03-29T04:17:36.610408+0000","last_active":"2024-03-29T04:17:43.552978+0000","last_peered":"2024-03-29T04:17:43.552978+0000","last_clean":"2024-03-29T04:17:43.552978+0000","last_became_active":"2024-03-29T04:17:36.610111+0000","last_became_peered":"2024-03-29T04:17:36.610111+0000","last_unstale":"2024-03-29T04:17:43.552978+0000","last_undegraded":"2024-03-29T04:17:43.552978+0000","last_fullsized":"2024-03-29T04:17:43.552978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:46:38.204638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.269316+0000","last_change":"2024-03-29T04:17:35.601002+0000","last_active":"2024-03-29T04:17:44.269316+0000","last_peered":"2024-03-29T04:17:44.269316+0000","last_clean":"2024-03-29T04:17:44.269316+0000","last_became_active":"2024-03-29T04:17:35.600276+0000","last_became_peered":"2024-03-29T04:17:35.600276+0000","last_unstale":"2024-03-29T04:17:44.269316+0000","last_undegraded":"2024-03-29T04:17:44.269316+0000","last_fullsized":"2024-03-29T04:17:44.269316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:55:50.253366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649873+0000","last_change":"2024-03-29T04:17:41.164850+0000","last_active":"2024-03-29T04:17:42.649873+0000","last_peered":"2024-03-29T04:17:42.649873+0000","last_clean":"2024-03-29T04:17:42.649873+0000","last_became_active":"2024-03-29T04:17:41.163894+0000","last_became_peered":"2024-03-29T04:17:41.163894+0000","last_unstale":"2024-03-29T04:17:42.649873+0000","last_undegraded":"2024-03-29T04:17:42.649873+0000","last_fullsized":"2024-03-29T04:17:42.649873+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:03:39.448893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653276+0000","last_change":"2024-03-29T04:17:41.171200+0000","last_active":"2024-03-29T04:17:42.653276+0000","last_peered":"2024-03-29T04:17:42.653276+0000","last_clean":"2024-03-29T04:17:42.653276+0000","last_became_active":"2024-03-29T04:17:41.169774+0000","last_became_peered":"2024-03-29T04:17:41.169774+0000","last_unstale":"2024-03-29T04:17:42.653276+0000","last_undegraded":"2024-03-29T04:17:42.653276+0000","last_fullsized":"2024-03-29T04:17:42.653276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:43:44.093178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470127+0000","last_change":"2024-03-29T04:17:36.598385+0000","last_active":"2024-03-29T04:17:44.470127+0000","last_peered":"2024-03-29T04:17:44.470127+0000","last_clean":"2024-03-29T04:17:44.470127+0000","last_became_active":"2024-03-29T04:17:36.597989+0000","last_became_peered":"2024-03-29T04:17:36.597989+0000","last_unstale":"2024-03-29T04:17:44.470127+0000","last_undegraded":"2024-03-29T04:17:44.470127+0000","last_fullsized":"2024-03-29T04:17:44.470127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:11.037419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553421+0000","last_change":"2024-03-29T04:17:35.591199+0000","last_active":"2024-03-29T04:17:43.553421+0000","last_peered":"2024-03-29T04:17:43.553421+0000","last_clean":"2024-03-29T04:17:43.553421+0000","last_became_active":"2024-03-29T04:17:35.590444+0000","last_became_peered":"2024-03-29T04:17:35.590444+0000","last_unstale":"2024-03-29T04:17:43.553421+0000","last_undegraded":"2024-03-29T04:17:43.553421+0000","last_fullsized":"2024-03-29T04:17:43.553421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:23.283694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270972+0000","last_change":"2024-03-29T04:17:40.661020+0000","last_active":"2024-03-29T04:17:44.270972+0000","last_peered":"2024-03-29T04:17:44.270972+0000","last_clean":"2024-03-29T04:17:44.270972+0000","last_became_active":"2024-03-29T04:17:40.660552+0000","last_became_peered":"2024-03-29T04:17:40.660552+0000","last_unstale":"2024-03-29T04:17:44.270972+0000","last_undegraded":"2024-03-29T04:17:44.270972+0000","last_fullsized":"2024-03-29T04:17:44.270972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:25:02.717791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649896+0000","last_change":"2024-03-29T04:17:36.619765+0000","last_active":"2024-03-29T04:17:42.649896+0000","last_peered":"2024-03-29T04:17:42.649896+0000","last_clean":"2024-03-29T04:17:42.649896+0000","last_became_active":"2024-03-29T04:17:36.619507+0000","last_became_peered":"2024-03-29T04:17:36.619507+0000","last_unstale":"2024-03-29T04:17:42.649896+0000","last_undegraded":"2024-03-29T04:17:42.649896+0000","last_fullsized":"2024-03-29T04:17:42.649896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:39.635039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649952+0000","last_change":"2024-03-29T04:17:35.575983+0000","last_active":"2024-03-29T04:17:42.649952+0000","last_peered":"2024-03-29T04:17:42.649952+0000","last_clean":"2024-03-29T04:17:42.649952+0000","last_became_active":"2024-03-29T04:17:35.575381+0000","last_became_peered":"2024-03-29T04:17:35.575381+0000","last_unstale":"2024-03-29T04:17:42.649952+0000","last_undegraded":"2024-03-29T04:17:42.649952+0000","last_fullsized":"2024-03-29T04:17:42.649952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:07.631047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.268935+0000","last_change":"2024-03-29T04:17:35.577005+0000","last_active":"2024-03-29T04:17:44.268935+0000","last_peered":"2024-03-29T04:17:44.268935+0000","last_clean":"2024-03-29T04:17:44.268935+0000","last_became_active":"2024-03-29T04:17:35.574501+0000","last_became_peered":"2024-03-29T04:17:35.574501+0000","last_unstale":"2024-03-29T04:17:44.268935+0000","last_undegraded":"2024-03-29T04:17:44.268935+0000","last_fullsized":"2024-03-29T04:17:44.268935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:23.283694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653382+0000","last_change":"2024-03-29T04:17:36.615300+0000","last_active":"2024-03-29T04:17:42.653382+0000","last_peered":"2024-03-29T04:17:42.653382+0000","last_clean":"2024-03-29T04:17:42.653382+0000","last_became_active":"2024-03-29T04:17:36.614767+0000","last_became_peered":"2024-03-29T04:17:36.614767+0000","last_unstale":"2024-03-29T04:17:42.653382+0000","last_undegraded":"2024-03-29T04:17:42.653382+0000","last_fullsized":"2024-03-29T04:17:42.653382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:30.598140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578719+0000","last_change":"2024-03-29T04:17:40.666173+0000","last_active":"2024-03-29T04:17:43.578719+0000","last_peered":"2024-03-29T04:17:43.578719+0000","last_clean":"2024-03-29T04:17:43.578719+0000","last_became_active":"2024-03-29T04:17:40.665205+0000","last_became_peered":"2024-03-29T04:17:40.665205+0000","last_unstale":"2024-03-29T04:17:43.578719+0000","last_undegraded":"2024-03-29T04:17:43.578719+0000","last_fullsized":"2024-03-29T04:17:43.578719+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:10:04.880736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650203+0000","last_change":"2024-03-29T04:17:35.597727+0000","last_active":"2024-03-29T04:17:42.650203+0000","last_peered":"2024-03-29T04:17:42.650203+0000","last_clean":"2024-03-29T04:17:42.650203+0000","last_became_active":"2024-03-29T04:17:35.597476+0000","last_became_peered":"2024-03-29T04:17:35.597476+0000","last_unstale":"2024-03-29T04:17:42.650203+0000","last_undegraded":"2024-03-29T04:17:42.650203+0000","last_fullsized":"2024-03-29T04:17:42.650203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:04:44.150824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656845+0000","last_change":"2024-03-29T04:17:36.593245+0000","last_active":"2024-03-29T04:17:42.656845+0000","last_peered":"2024-03-29T04:17:42.656845+0000","last_clean":"2024-03-29T04:17:42.656845+0000","last_became_active":"2024-03-29T04:17:36.592508+0000","last_became_peered":"2024-03-29T04:17:36.592508+0000","last_unstale":"2024-03-29T04:17:42.656845+0000","last_undegraded":"2024-03-29T04:17:42.656845+0000","last_fullsized":"2024-03-29T04:17:42.656845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:52:19.919554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650486+0000","last_change":"2024-03-29T04:17:40.664734+0000","last_active":"2024-03-29T04:17:42.650486+0000","last_peered":"2024-03-29T04:17:42.650486+0000","last_clean":"2024-03-29T04:17:42.650486+0000","last_became_active":"2024-03-29T04:17:40.664326+0000","last_became_peered":"2024-03-29T04:17:40.664326+0000","last_unstale":"2024-03-29T04:17:42.650486+0000","last_undegraded":"2024-03-29T04:17:42.650486+0000","last_fullsized":"2024-03-29T04:17:42.650486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:03.774120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.269324+0000","last_change":"2024-03-29T04:17:36.618652+0000","last_active":"2024-03-29T04:17:44.269324+0000","last_peered":"2024-03-29T04:17:44.269324+0000","last_clean":"2024-03-29T04:17:44.269324+0000","last_became_active":"2024-03-29T04:17:36.618384+0000","last_became_peered":"2024-03-29T04:17:36.618384+0000","last_unstale":"2024-03-29T04:17:44.269324+0000","last_undegraded":"2024-03-29T04:17:44.269324+0000","last_fullsized":"2024-03-29T04:17:44.269324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:30.598140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578474+0000","last_change":"2024-03-29T04:17:36.606149+0000","last_active":"2024-03-29T04:17:43.578474+0000","last_peered":"2024-03-29T04:17:43.578474+0000","last_clean":"2024-03-29T04:17:43.578474+0000","last_became_active":"2024-03-29T04:17:36.605253+0000","last_became_peered":"2024-03-29T04:17:36.605253+0000","last_unstale":"2024-03-29T04:17:43.578474+0000","last_undegraded":"2024-03-29T04:17:43.578474+0000","last_fullsized":"2024-03-29T04:17:43.578474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:24:47.130861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553107+0000","last_change":"2024-03-29T04:17:36.602472+0000","last_active":"2024-03-29T04:17:43.553107+0000","last_peered":"2024-03-29T04:17:43.553107+0000","last_clean":"2024-03-29T04:17:43.553107+0000","last_became_active":"2024-03-29T04:17:36.601990+0000","last_became_peered":"2024-03-29T04:17:36.601990+0000","last_unstale":"2024-03-29T04:17:43.553107+0000","last_undegraded":"2024-03-29T04:17:43.553107+0000","last_fullsized":"2024-03-29T04:17:43.553107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:38:20.641332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653238+0000","last_change":"2024-03-29T04:17:36.607285+0000","last_active":"2024-03-29T04:17:42.653238+0000","last_peered":"2024-03-29T04:17:42.653238+0000","last_clean":"2024-03-29T04:17:42.653238+0000","last_became_active":"2024-03-29T04:17:36.606419+0000","last_became_peered":"2024-03-29T04:17:36.606419+0000","last_unstale":"2024-03-29T04:17:42.653238+0000","last_undegraded":"2024-03-29T04:17:42.653238+0000","last_fullsized":"2024-03-29T04:17:42.653238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:41.872399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650775+0000","last_change":"2024-03-29T04:17:36.627762+0000","last_active":"2024-03-29T04:17:42.650775+0000","last_peered":"2024-03-29T04:17:42.650775+0000","last_clean":"2024-03-29T04:17:42.650775+0000","last_became_active":"2024-03-29T04:17:36.627476+0000","last_became_peered":"2024-03-29T04:17:36.627476+0000","last_unstale":"2024-03-29T04:17:42.650775+0000","last_undegraded":"2024-03-29T04:17:42.650775+0000","last_fullsized":"2024-03-29T04:17:42.650775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:38:20.641332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470447+0000","last_change":"2024-03-29T04:17:36.597795+0000","last_active":"2024-03-29T04:17:44.470447+0000","last_peered":"2024-03-29T04:17:44.470447+0000","last_clean":"2024-03-29T04:17:44.470447+0000","last_became_active":"2024-03-29T04:17:36.597490+0000","last_became_peered":"2024-03-29T04:17:36.597490+0000","last_unstale":"2024-03-29T04:17:44.470447+0000","last_undegraded":"2024-03-29T04:17:44.470447+0000","last_fullsized":"2024-03-29T04:17:44.470447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:43:27.568676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270661+0000","last_change":"2024-03-29T04:17:36.609886+0000","last_active":"2024-03-29T04:17:44.270661+0000","last_peered":"2024-03-29T04:17:44.270661+0000","last_clean":"2024-03-29T04:17:44.270661+0000","last_became_active":"2024-03-29T04:17:36.609354+0000","last_became_peered":"2024-03-29T04:17:36.609354+0000","last_unstale":"2024-03-29T04:17:44.270661+0000","last_undegraded":"2024-03-29T04:17:44.270661+0000","last_fullsized":"2024-03-29T04:17:44.270661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:24:47.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650587+0000","last_change":"2024-03-29T04:17:36.627964+0000","last_active":"2024-03-29T04:17:42.650587+0000","last_peered":"2024-03-29T04:17:42.650587+0000","last_clean":"2024-03-29T04:17:42.650587+0000","last_became_active":"2024-03-29T04:17:36.627606+0000","last_became_peered":"2024-03-29T04:17:36.627606+0000","last_unstale":"2024-03-29T04:17:42.650587+0000","last_undegraded":"2024-03-29T04:17:42.650587+0000","last_fullsized":"2024-03-29T04:17:42.650587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:44.877184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552870+0000","last_change":"2024-03-29T04:17:36.617738+0000","last_active":"2024-03-29T04:17:43.552870+0000","last_peered":"2024-03-29T04:17:43.552870+0000","last_clean":"2024-03-29T04:17:43.552870+0000","last_became_active":"2024-03-29T04:17:36.617532+0000","last_became_peered":"2024-03-29T04:17:36.617532+0000","last_unstale":"2024-03-29T04:17:43.552870+0000","last_undegraded":"2024-03-29T04:17:43.552870+0000","last_fullsized":"2024-03-29T04:17:43.552870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:41.872399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649836+0000","last_change":"2024-03-29T04:17:36.627025+0000","last_active":"2024-03-29T04:17:42.649836+0000","last_peered":"2024-03-29T04:17:42.649836+0000","last_clean":"2024-03-29T04:17:42.649836+0000","last_became_active":"2024-03-29T04:17:36.626671+0000","last_became_peered":"2024-03-29T04:17:36.626671+0000","last_unstale":"2024-03-29T04:17:42.649836+0000","last_undegraded":"2024-03-29T04:17:42.649836+0000","last_fullsized":"2024-03-29T04:17:42.649836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:31:36.783083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552909+0000","last_change":"2024-03-29T04:17:36.609953+0000","last_active":"2024-03-29T04:17:43.552909+0000","last_peered":"2024-03-29T04:17:43.552909+0000","last_clean":"2024-03-29T04:17:43.552909+0000","last_became_active":"2024-03-29T04:17:36.609455+0000","last_became_peered":"2024-03-29T04:17:36.609455+0000","last_unstale":"2024-03-29T04:17:43.552909+0000","last_undegraded":"2024-03-29T04:17:43.552909+0000","last_fullsized":"2024-03-29T04:17:43.552909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:31:36.783083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653087+0000","last_change":"2024-03-29T04:17:36.618490+0000","last_active":"2024-03-29T04:17:42.653087+0000","last_peered":"2024-03-29T04:17:42.653087+0000","last_clean":"2024-03-29T04:17:42.653087+0000","last_became_active":"2024-03-29T04:17:36.618012+0000","last_became_peered":"2024-03-29T04:17:36.618012+0000","last_unstale":"2024-03-29T04:17:42.653087+0000","last_undegraded":"2024-03-29T04:17:42.653087+0000","last_fullsized":"2024-03-29T04:17:42.653087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:43:27.568676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270104+0000","last_change":"2024-03-29T04:17:36.621359+0000","last_active":"2024-03-29T04:17:44.270104+0000","last_peered":"2024-03-29T04:17:44.270104+0000","last_clean":"2024-03-29T04:17:44.270104+0000","last_became_active":"2024-03-29T04:17:36.620689+0000","last_became_peered":"2024-03-29T04:17:36.620689+0000","last_unstale":"2024-03-29T04:17:44.270104+0000","last_undegraded":"2024-03-29T04:17:44.270104+0000","last_fullsized":"2024-03-29T04:17:44.270104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:43:27.568676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634352+0000","last_change":"2024-03-29T04:17:36.610675+0000","last_active":"2024-03-29T04:17:44.634352+0000","last_peered":"2024-03-29T04:17:44.634352+0000","last_clean":"2024-03-29T04:17:44.634352+0000","last_became_active":"2024-03-29T04:17:36.609797+0000","last_became_peered":"2024-03-29T04:17:36.609797+0000","last_unstale":"2024-03-29T04:17:44.634352+0000","last_undegraded":"2024-03-29T04:17:44.634352+0000","last_fullsized":"2024-03-29T04:17:44.634352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:04:11.306658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.579363+0000","last_change":"2024-03-29T04:17:36.605354+0000","last_active":"2024-03-29T04:17:43.579363+0000","last_peered":"2024-03-29T04:17:43.579363+0000","last_clean":"2024-03-29T04:17:43.579363+0000","last_became_active":"2024-03-29T04:17:36.605028+0000","last_became_peered":"2024-03-29T04:17:36.605028+0000","last_unstale":"2024-03-29T04:17:43.579363+0000","last_undegraded":"2024-03-29T04:17:43.579363+0000","last_fullsized":"2024-03-29T04:17:43.579363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:12:37.599253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553747+0000","last_change":"2024-03-29T04:17:36.611310+0000","last_active":"2024-03-29T04:17:43.553747+0000","last_peered":"2024-03-29T04:17:43.553747+0000","last_clean":"2024-03-29T04:17:43.553747+0000","last_became_active":"2024-03-29T04:17:36.611020+0000","last_became_peered":"2024-03-29T04:17:36.611020+0000","last_unstale":"2024-03-29T04:17:43.553747+0000","last_undegraded":"2024-03-29T04:17:43.553747+0000","last_fullsized":"2024-03-29T04:17:43.553747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:42.915247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552533+0000","last_change":"2024-03-29T04:17:36.618624+0000","last_active":"2024-03-29T04:17:43.552533+0000","last_peered":"2024-03-29T04:17:43.552533+0000","last_clean":"2024-03-29T04:17:43.552533+0000","last_became_active":"2024-03-29T04:17:36.618282+0000","last_became_peered":"2024-03-29T04:17:36.618282+0000","last_unstale":"2024-03-29T04:17:43.552533+0000","last_undegraded":"2024-03-29T04:17:43.552533+0000","last_fullsized":"2024-03-29T04:17:43.552533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:24:47.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470271+0000","last_change":"2024-03-29T04:17:36.587187+0000","last_active":"2024-03-29T04:17:44.470271+0000","last_peered":"2024-03-29T04:17:44.470271+0000","last_clean":"2024-03-29T04:17:44.470271+0000","last_became_active":"2024-03-29T04:17:36.586740+0000","last_became_peered":"2024-03-29T04:17:36.586740+0000","last_unstale":"2024-03-29T04:17:44.470271+0000","last_undegraded":"2024-03-29T04:17:44.470271+0000","last_fullsized":"2024-03-29T04:17:44.470271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:24:47.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653115+0000","last_change":"2024-03-29T04:17:36.615925+0000","last_active":"2024-03-29T04:17:42.653115+0000","last_peered":"2024-03-29T04:17:42.653115+0000","last_clean":"2024-03-29T04:17:42.653115+0000","last_became_active":"2024-03-29T04:17:36.615143+0000","last_became_peered":"2024-03-29T04:17:36.615143+0000","last_unstale":"2024-03-29T04:17:42.653115+0000","last_undegraded":"2024-03-29T04:17:42.653115+0000","last_fullsized":"2024-03-29T04:17:42.653115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:50:12.920911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650032+0000","last_change":"2024-03-29T04:17:36.605090+0000","last_active":"2024-03-29T04:17:42.650032+0000","last_peered":"2024-03-29T04:17:42.650032+0000","last_clean":"2024-03-29T04:17:42.650032+0000","last_became_active":"2024-03-29T04:17:36.604669+0000","last_became_peered":"2024-03-29T04:17:36.604669+0000","last_unstale":"2024-03-29T04:17:42.650032+0000","last_undegraded":"2024-03-29T04:17:42.650032+0000","last_fullsized":"2024-03-29T04:17:42.650032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:30.598140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.657981+0000","last_change":"2024-03-29T04:17:36.614472+0000","last_active":"2024-03-29T04:17:42.657981+0000","last_peered":"2024-03-29T04:17:42.657981+0000","last_clean":"2024-03-29T04:17:42.657981+0000","last_became_active":"2024-03-29T04:17:36.613881+0000","last_became_peered":"2024-03-29T04:17:36.613881+0000","last_unstale":"2024-03-29T04:17:42.657981+0000","last_undegraded":"2024-03-29T04:17:42.657981+0000","last_fullsized":"2024-03-29T04:17:42.657981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:24:47.130861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270375+0000","last_change":"2024-03-29T04:17:36.600125+0000","last_active":"2024-03-29T04:17:44.270375+0000","last_peered":"2024-03-29T04:17:44.270375+0000","last_clean":"2024-03-29T04:17:44.270375+0000","last_became_active":"2024-03-29T04:17:36.599450+0000","last_became_peered":"2024-03-29T04:17:36.599450+0000","last_unstale":"2024-03-29T04:17:44.270375+0000","last_undegraded":"2024-03-29T04:17:44.270375+0000","last_fullsized":"2024-03-29T04:17:44.270375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:11.037419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270626+0000","last_change":"2024-03-29T04:17:36.600177+0000","last_active":"2024-03-29T04:17:44.270626+0000","last_peered":"2024-03-29T04:17:44.270626+0000","last_clean":"2024-03-29T04:17:44.270626+0000","last_became_active":"2024-03-29T04:17:36.599696+0000","last_became_peered":"2024-03-29T04:17:36.599696+0000","last_unstale":"2024-03-29T04:17:44.270626+0000","last_undegraded":"2024-03-29T04:17:44.270626+0000","last_fullsized":"2024-03-29T04:17:44.270626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:39.635039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.635042+0000","last_change":"2024-03-29T04:17:36.610038+0000","last_active":"2024-03-29T04:17:44.635042+0000","last_peered":"2024-03-29T04:17:44.635042+0000","last_clean":"2024-03-29T04:17:44.635042+0000","last_became_active":"2024-03-29T04:17:36.609175+0000","last_became_peered":"2024-03-29T04:17:36.609175+0000","last_unstale":"2024-03-29T04:17:44.635042+0000","last_undegraded":"2024-03-29T04:17:44.635042+0000","last_fullsized":"2024-03-29T04:17:44.635042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:42:21.035572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649728+0000","last_change":"2024-03-29T04:17:36.616578+0000","last_active":"2024-03-29T04:17:42.649728+0000","last_peered":"2024-03-29T04:17:42.649728+0000","last_clean":"2024-03-29T04:17:42.649728+0000","last_became_active":"2024-03-29T04:17:36.612656+0000","last_became_peered":"2024-03-29T04:17:36.612656+0000","last_unstale":"2024-03-29T04:17:42.649728+0000","last_undegraded":"2024-03-29T04:17:42.649728+0000","last_fullsized":"2024-03-29T04:17:42.649728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:11.037419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634140+0000","last_change":"2024-03-29T04:17:36.618859+0000","last_active":"2024-03-29T04:17:44.634140+0000","last_peered":"2024-03-29T04:17:44.634140+0000","last_clean":"2024-03-29T04:17:44.634140+0000","last_became_active":"2024-03-29T04:17:36.618475+0000","last_became_peered":"2024-03-29T04:17:36.618475+0000","last_unstale":"2024-03-29T04:17:44.634140+0000","last_undegraded":"2024-03-29T04:17:44.634140+0000","last_fullsized":"2024-03-29T04:17:44.634140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:30.729030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552858+0000","last_change":"2024-03-29T04:17:36.614982+0000","last_active":"2024-03-29T04:17:43.552858+0000","last_peered":"2024-03-29T04:17:43.552858+0000","last_clean":"2024-03-29T04:17:43.552858+0000","last_became_active":"2024-03-29T04:17:36.614701+0000","last_became_peered":"2024-03-29T04:17:36.614701+0000","last_unstale":"2024-03-29T04:17:43.552858+0000","last_undegraded":"2024-03-29T04:17:43.552858+0000","last_fullsized":"2024-03-29T04:17:43.552858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:30.598140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634536+0000","last_change":"2024-03-29T04:17:36.608699+0000","last_active":"2024-03-29T04:17:44.634536+0000","last_peered":"2024-03-29T04:17:44.634536+0000","last_clean":"2024-03-29T04:17:44.634536+0000","last_became_active":"2024-03-29T04:17:36.608428+0000","last_became_peered":"2024-03-29T04:17:36.608428+0000","last_unstale":"2024-03-29T04:17:44.634536+0000","last_undegraded":"2024-03-29T04:17:44.634536+0000","last_fullsized":"2024-03-29T04:17:44.634536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:44.877184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553129+0000","last_change":"2024-03-29T04:17:36.596870+0000","last_active":"2024-03-29T04:17:43.553129+0000","last_peered":"2024-03-29T04:17:43.553129+0000","last_clean":"2024-03-29T04:17:43.553129+0000","last_became_active":"2024-03-29T04:17:36.596557+0000","last_became_peered":"2024-03-29T04:17:36.596557+0000","last_unstale":"2024-03-29T04:17:43.553129+0000","last_undegraded":"2024-03-29T04:17:43.553129+0000","last_fullsized":"2024-03-29T04:17:43.553129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:44.877184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653260+0000","last_change":"2024-03-29T04:17:36.593793+0000","last_active":"2024-03-29T04:17:42.653260+0000","last_peered":"2024-03-29T04:17:42.653260+0000","last_clean":"2024-03-29T04:17:42.653260+0000","last_became_active":"2024-03-29T04:17:36.592632+0000","last_became_peered":"2024-03-29T04:17:36.592632+0000","last_unstale":"2024-03-29T04:17:42.653260+0000","last_undegraded":"2024-03-29T04:17:42.653260+0000","last_fullsized":"2024-03-29T04:17:42.653260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:41:31.910133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470745+0000","last_change":"2024-03-29T04:17:36.593148+0000","last_active":"2024-03-29T04:17:44.470745+0000","last_peered":"2024-03-29T04:17:44.470745+0000","last_clean":"2024-03-29T04:17:44.470745+0000","last_became_active":"2024-03-29T04:17:36.592754+0000","last_became_peered":"2024-03-29T04:17:36.592754+0000","last_unstale":"2024-03-29T04:17:44.470745+0000","last_undegraded":"2024-03-29T04:17:44.470745+0000","last_fullsized":"2024-03-29T04:17:44.470745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:30.598140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.635518+0000","last_change":"2024-03-29T04:17:36.615787+0000","last_active":"2024-03-29T04:17:44.635518+0000","last_peered":"2024-03-29T04:17:44.635518+0000","last_clean":"2024-03-29T04:17:44.635518+0000","last_became_active":"2024-03-29T04:17:36.615476+0000","last_became_peered":"2024-03-29T04:17:36.615476+0000","last_unstale":"2024-03-29T04:17:44.635518+0000","last_undegraded":"2024-03-29T04:17:44.635518+0000","last_fullsized":"2024-03-29T04:17:44.635518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:42.915247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":876544,"data_stored":703014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":112154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":112190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":278528,"data_stored":112064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":286720,"data_stored":112224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":884736,"data_stored":703070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:17:54.804 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:17:54.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.984+0000 7fe0e9a42640 1 Processor -- start 2024-03-29T04:17:54.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.984+0000 7fe0e9a42640 1 -- start start 2024-03-29T04:17:54.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.984+0000 7fe0e9a42640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe0e40d1810 0x7fe0e40b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:54.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.984+0000 7fe0e9a42640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0e40d3e60 con 0x7fe0e40b9170 2024-03-29T04:17:54.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.984+0000 7fe0e9a42640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0e40b7680 con 0x7fe0e415e600 2024-03-29T04:17:54.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.984+0000 7fe0e9a42640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fe0e40b7450 con 0x7fe0e40d1810 2024-03-29T04:17:54.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.984+0000 7fe0e27fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe0e40d1810 0x7fe0e40b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:54.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.984+0000 7fe0e37fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe0e40b9170 0x7fe0e40d2600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44132/0 (socket says 172.21.15.22:44132) 2024-03-29T04:17:54.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.984+0000 7fe0e37fe640 1 -- 172.21.15.22:0/2922007481 learned_addr learned my addr 172.21.15.22:0/2922007481 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:54.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.984+0000 7fe0e27fc640 1 -- 172.21.15.22:0/2922007481 >> v1:172.21.15.151:6789/0 conn(0x7fe0e415e600 legacy=0x7fe0e40d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:54.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.984+0000 7fe0e27fc640 1 -- 172.21.15.22:0/2922007481 >> v1:172.21.15.22:6789/0 conn(0x7fe0e40b9170 legacy=0x7fe0e40d2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:54.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.984+0000 7fe0e1ffb640 1 -- 172.21.15.22:0/2922007481 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1414311082 0 0) 0x7fe0e40d3e60 con 0x7fe0e40b9170 2024-03-29T04:17:54.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.984+0000 7fe0e27fc640 1 -- 172.21.15.22:0/2922007481 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e40b7960 con 0x7fe0e40d1810 2024-03-29T04:17:54.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.988+0000 7fe0e27fc640 1 --2- 172.21.15.22:0/2922007481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e40d1810 0x7fe0e40b9f00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80097c0 tx=0x7fe0d801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a1b8bd80490c41 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:54.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.988+0000 7fe0e1ffb640 1 -- 172.21.15.22:0/2922007481 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d80b0070 con 0x7fe0e40d1810 2024-03-29T04:17:54.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.988+0000 7fe0e1ffb640 1 -- 172.21.15.22:0/2922007481 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d80ab600 con 0x7fe0e40d1810 2024-03-29T04:17:54.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.988+0000 7fe0e1ffb640 1 -- 172.21.15.22:0/2922007481 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d80ab9d0 con 0x7fe0e40d1810 2024-03-29T04:17:54.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.988+0000 7fe0e9a42640 1 -- 172.21.15.22:0/2922007481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e40d1810 msgr2=0x7fe0e40b9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:54.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.988+0000 7fe0e9a42640 1 --2- 172.21.15.22:0/2922007481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e40d1810 0x7fe0e40b9f00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80097c0 tx=0x7fe0d801c410 comp rx=0 tx=0).stop 2024-03-29T04:17:54.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.988+0000 7fe0e9a42640 1 -- 172.21.15.22:0/2922007481 shutdown_connections 2024-03-29T04:17:54.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.988+0000 7fe0e9a42640 1 --2- 172.21.15.22:0/2922007481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e40d1810 0x7fe0e40b9f00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.988+0000 7fe0e9a42640 1 -- 172.21.15.22:0/2922007481 >> 172.21.15.22:0/2922007481 conn(0x7fe0e40b6230 msgr2=0x7fe0e40b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:54.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.988+0000 7fe0e9a42640 1 -- 172.21.15.22:0/2922007481 shutdown_connections 2024-03-29T04:17:54.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.988+0000 7fe0e9a42640 1 -- 172.21.15.22:0/2922007481 wait complete. 2024-03-29T04:17:54.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e9a42640 1 Processor -- start 2024-03-29T04:17:54.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e9a42640 1 -- start start 2024-03-29T04:17:54.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e9a42640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0e40b9170 0x7fe0e41495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:54.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e9a42640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe0e40d1810 0x7fe0e4156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:54.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e9a42640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e415e600 0x7fe0e4156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:54.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e9a42640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e416bb70 con 0x7fe0e415e600 2024-03-29T04:17:54.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e9a42640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e416b9f0 con 0x7fe0e40b9170 2024-03-29T04:17:54.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e9a42640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e416bcf0 con 0x7fe0e40d1810 2024-03-29T04:17:54.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e415e600 0x7fe0e4156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:54.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e415e600 0x7fe0e4156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51230/0 (socket says 172.21.15.22:51230) 2024-03-29T04:17:54.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e37fe640 1 -- 172.21.15.22:0/2617766667 learned_addr learned my addr 172.21.15.22:0/2617766667 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:54.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e27fc640 1 --2- 172.21.15.22:0/2617766667 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe0e40d1810 0x7fe0e4156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:54.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e2ffd640 1 --2- 172.21.15.22:0/2617766667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0e40b9170 0x7fe0e41495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:54.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e37fe640 1 -- 172.21.15.22:0/2617766667 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe0e40d1810 msgr2=0x7fe0e4156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:54.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e37fe640 1 --2- 172.21.15.22:0/2617766667 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe0e40d1810 0x7fe0e4156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e37fe640 1 -- 172.21.15.22:0/2617766667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0e40b9170 msgr2=0x7fe0e41495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:54.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e37fe640 1 --2- 172.21.15.22:0/2617766667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0e40b9170 0x7fe0e41495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:54.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e37fe640 1 -- 172.21.15.22:0/2617766667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e41567a0 con 0x7fe0e415e600 2024-03-29T04:17:54.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e37fe640 1 --2- 172.21.15.22:0/2617766667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e415e600 0x7fe0e4156b90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00c700 tx=0x7fe0dc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:54.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0cbfff640 1 -- 172.21.15.22:0/2617766667 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc018070 con 0x7fe0e415e600 2024-03-29T04:17:54.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0cbfff640 1 -- 172.21.15.22:0/2617766667 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0dc00ce10 con 0x7fe0e415e600 2024-03-29T04:17:54.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e9a42640 1 -- 172.21.15.22:0/2617766667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e40c8970 con 0x7fe0e415e600 2024-03-29T04:17:54.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0cbfff640 1 -- 172.21.15.22:0/2617766667 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc00a880 con 0x7fe0e415e600 2024-03-29T04:17:54.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e9a42640 1 -- 172.21.15.22:0/2617766667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e40c8f30 con 0x7fe0e415e600 2024-03-29T04:17:54.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0e9a42640 1 -- 172.21.15.22:0/2617766667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0e4068890 con 0x7fe0e415e600 2024-03-29T04:17:54.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0cbfff640 1 -- 172.21.15.22:0/2617766667 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe0dc01b070 con 0x7fe0e415e600 2024-03-29T04:17:54.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0cbfff640 1 --2- 172.21.15.22:0/2617766667 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe0d0067fc0 0x7fe0d006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:54.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.992+0000 7fe0cbfff640 1 -- 172.21.15.22:0/2617766667 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe0dc085d90 con 0x7fe0e415e600 2024-03-29T04:17:54.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.996+0000 7fe0e2ffd640 1 --2- 172.21.15.22:0/2617766667 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe0d0067fc0 0x7fe0d006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:54.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:54.996+0000 7fe0e2ffd640 1 --2- 172.21.15.22:0/2617766667 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe0d0067fc0 0x7fe0d006a480 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4149ea0 tx=0x7fe0d4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:54.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.000+0000 7fe0cbfff640 1 -- 172.21.15.22:0/2617766667 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fe0dc056290 con 0x7fe0e415e600 2024-03-29T04:17:55.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.156+0000 7fe0e9a42640 1 -- 172.21.15.22:0/2617766667 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe0e40b9f70 con 0x7fe0d0067fc0 2024-03-29T04:17:55.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.180+0000 7fe0cbfff640 1 -- 172.21.15.22:0/2617766667 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332558 (secure 0 0 0) 0x7fe0e40b9f70 con 0x7fe0d0067fc0 2024-03-29T04:17:55.163 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:55.166 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-29T04:17:55.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.184+0000 7fe0e9a42640 1 -- 172.21.15.22:0/2617766667 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe0d0067fc0 msgr2=0x7fe0d006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:55.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.184+0000 7fe0e9a42640 1 --2- 172.21.15.22:0/2617766667 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe0d0067fc0 0x7fe0d006a480 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4149ea0 tx=0x7fe0d4009040 comp rx=0 tx=0).stop 2024-03-29T04:17:55.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.184+0000 7fe0e9a42640 1 -- 172.21.15.22:0/2617766667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e415e600 msgr2=0x7fe0e4156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:55.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.184+0000 7fe0e9a42640 1 --2- 172.21.15.22:0/2617766667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e415e600 0x7fe0e4156b90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00c700 tx=0x7fe0dc00cbc0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.184+0000 7fe0e9a42640 1 -- 172.21.15.22:0/2617766667 shutdown_connections 2024-03-29T04:17:55.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.184+0000 7fe0e9a42640 1 --2- 172.21.15.22:0/2617766667 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe0d0067fc0 0x7fe0d006a480 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.184+0000 7fe0e9a42640 1 --2- 172.21.15.22:0/2617766667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e415e600 0x7fe0e4156b90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.184+0000 7fe0e9a42640 1 --2- 172.21.15.22:0/2617766667 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe0e40d1810 0x7fe0e4156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.184+0000 7fe0e9a42640 1 --2- 172.21.15.22:0/2617766667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0e40b9170 0x7fe0e41495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.184+0000 7fe0e9a42640 1 -- 172.21.15.22:0/2617766667 >> 172.21.15.22:0/2617766667 conn(0x7fe0e40b6230 msgr2=0x7fe0e40d26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:55.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.184+0000 7fe0e9a42640 1 -- 172.21.15.22:0/2617766667 shutdown_connections 2024-03-29T04:17:55.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.184+0000 7fe0e9a42640 1 -- 172.21.15.22:0/2617766667 wait complete. 2024-03-29T04:17:55.185 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-03-29T04:17:53.697360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236208,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738512,"statfs":{"total":773094113280,"available":772852236288,"internally_reserved":0,"allocated":3481600,"data_stored":2086528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.043496"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653655+0000","last_change":"2024-03-29T04:17:35.587951+0000","last_active":"2024-03-29T04:17:42.653655+0000","last_peered":"2024-03-29T04:17:42.653655+0000","last_clean":"2024-03-29T04:17:42.653655+0000","last_became_active":"2024-03-29T04:17:35.587644+0000","last_became_peered":"2024-03-29T04:17:35.587644+0000","last_unstale":"2024-03-29T04:17:42.653655+0000","last_undegraded":"2024-03-29T04:17:42.653655+0000","last_fullsized":"2024-03-29T04:17:42.653655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:07.631047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650468+0000","last_change":"2024-03-29T04:17:36.604695+0000","last_active":"2024-03-29T04:17:42.650468+0000","last_peered":"2024-03-29T04:17:42.650468+0000","last_clean":"2024-03-29T04:17:42.650468+0000","last_became_active":"2024-03-29T04:17:36.602172+0000","last_became_peered":"2024-03-29T04:17:36.602172+0000","last_unstale":"2024-03-29T04:17:42.650468+0000","last_undegraded":"2024-03-29T04:17:42.650468+0000","last_fullsized":"2024-03-29T04:17:42.650468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:30.729030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649719+0000","last_change":"2024-03-29T04:17:41.168150+0000","last_active":"2024-03-29T04:17:42.649719+0000","last_peered":"2024-03-29T04:17:42.649719+0000","last_clean":"2024-03-29T04:17:42.649719+0000","last_became_active":"2024-03-29T04:17:41.166854+0000","last_became_peered":"2024-03-29T04:17:41.166854+0000","last_unstale":"2024-03-29T04:17:42.649719+0000","last_undegraded":"2024-03-29T04:17:42.649719+0000","last_fullsized":"2024-03-29T04:17:42.649719+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:01:07.401841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650673+0000","last_change":"2024-03-29T04:17:35.575675+0000","last_active":"2024-03-29T04:17:42.650673+0000","last_peered":"2024-03-29T04:17:42.650673+0000","last_clean":"2024-03-29T04:17:42.650673+0000","last_became_active":"2024-03-29T04:17:35.575189+0000","last_became_peered":"2024-03-29T04:17:35.575189+0000","last_unstale":"2024-03-29T04:17:42.650673+0000","last_undegraded":"2024-03-29T04:17:42.650673+0000","last_fullsized":"2024-03-29T04:17:42.650673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:55:50.253366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653534+0000","last_change":"2024-03-29T04:17:36.598621+0000","last_active":"2024-03-29T04:17:42.653534+0000","last_peered":"2024-03-29T04:17:42.653534+0000","last_clean":"2024-03-29T04:17:42.653534+0000","last_became_active":"2024-03-29T04:17:36.596823+0000","last_became_peered":"2024-03-29T04:17:36.596823+0000","last_unstale":"2024-03-29T04:17:42.653534+0000","last_undegraded":"2024-03-29T04:17:42.653534+0000","last_fullsized":"2024-03-29T04:17:42.653534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:11.037419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656537+0000","last_change":"2024-03-29T04:17:40.663898+0000","last_active":"2024-03-29T04:17:42.656537+0000","last_peered":"2024-03-29T04:17:42.656537+0000","last_clean":"2024-03-29T04:17:42.656537+0000","last_became_active":"2024-03-29T04:17:40.663387+0000","last_became_peered":"2024-03-29T04:17:40.663387+0000","last_unstale":"2024-03-29T04:17:42.656537+0000","last_undegraded":"2024-03-29T04:17:42.656537+0000","last_fullsized":"2024-03-29T04:17:42.656537+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:46.993432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.635633+0000","last_change":"2024-03-29T04:17:35.604714+0000","last_active":"2024-03-29T04:17:44.635633+0000","last_peered":"2024-03-29T04:17:44.635633+0000","last_clean":"2024-03-29T04:17:44.635633+0000","last_became_active":"2024-03-29T04:17:35.604080+0000","last_became_peered":"2024-03-29T04:17:35.604080+0000","last_unstale":"2024-03-29T04:17:44.635633+0000","last_undegraded":"2024-03-29T04:17:44.635633+0000","last_fullsized":"2024-03-29T04:17:44.635633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:12:36.590426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553394+0000","last_change":"2024-03-29T04:17:36.615673+0000","last_active":"2024-03-29T04:17:43.553394+0000","last_peered":"2024-03-29T04:17:43.553394+0000","last_clean":"2024-03-29T04:17:43.553394+0000","last_became_active":"2024-03-29T04:17:36.615348+0000","last_became_peered":"2024-03-29T04:17:36.615348+0000","last_unstale":"2024-03-29T04:17:43.553394+0000","last_undegraded":"2024-03-29T04:17:43.553394+0000","last_fullsized":"2024-03-29T04:17:43.553394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:30.729030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656960+0000","last_change":"2024-03-29T04:17:40.653544+0000","last_active":"2024-03-29T04:17:42.656960+0000","last_peered":"2024-03-29T04:17:42.656960+0000","last_clean":"2024-03-29T04:17:42.656960+0000","last_became_active":"2024-03-29T04:17:40.652980+0000","last_became_peered":"2024-03-29T04:17:40.652980+0000","last_unstale":"2024-03-29T04:17:42.656960+0000","last_undegraded":"2024-03-29T04:17:42.656960+0000","last_fullsized":"2024-03-29T04:17:42.656960+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:25:02.717791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634331+0000","last_change":"2024-03-29T04:17:35.604849+0000","last_active":"2024-03-29T04:17:44.634331+0000","last_peered":"2024-03-29T04:17:44.634331+0000","last_clean":"2024-03-29T04:17:44.634331+0000","last_became_active":"2024-03-29T04:17:35.604004+0000","last_became_peered":"2024-03-29T04:17:35.604004+0000","last_unstale":"2024-03-29T04:17:44.634331+0000","last_undegraded":"2024-03-29T04:17:44.634331+0000","last_fullsized":"2024-03-29T04:17:44.634331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:50:11.912084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552984+0000","last_change":"2024-03-29T04:17:36.592055+0000","last_active":"2024-03-29T04:17:43.552984+0000","last_peered":"2024-03-29T04:17:43.552984+0000","last_clean":"2024-03-29T04:17:43.552984+0000","last_became_active":"2024-03-29T04:17:36.588805+0000","last_became_peered":"2024-03-29T04:17:36.588805+0000","last_unstale":"2024-03-29T04:17:43.552984+0000","last_undegraded":"2024-03-29T04:17:43.552984+0000","last_fullsized":"2024-03-29T04:17:43.552984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:13.677092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656938+0000","last_change":"2024-03-29T04:17:40.651131+0000","last_active":"2024-03-29T04:17:42.656938+0000","last_peered":"2024-03-29T04:17:42.656938+0000","last_clean":"2024-03-29T04:17:42.656938+0000","last_became_active":"2024-03-29T04:17:40.650319+0000","last_became_peered":"2024-03-29T04:17:40.650319+0000","last_unstale":"2024-03-29T04:17:42.656938+0000","last_undegraded":"2024-03-29T04:17:42.656938+0000","last_fullsized":"2024-03-29T04:17:42.656938+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:42:25.113757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.269694+0000","last_change":"2024-03-29T04:17:36.620761+0000","last_active":"2024-03-29T04:17:44.269694+0000","last_peered":"2024-03-29T04:17:44.269694+0000","last_clean":"2024-03-29T04:17:44.269694+0000","last_became_active":"2024-03-29T04:17:36.620483+0000","last_became_peered":"2024-03-29T04:17:36.620483+0000","last_unstale":"2024-03-29T04:17:44.269694+0000","last_undegraded":"2024-03-29T04:17:44.269694+0000","last_fullsized":"2024-03-29T04:17:44.269694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:10:00.802551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634192+0000","last_change":"2024-03-29T04:17:35.602583+0000","last_active":"2024-03-29T04:17:44.634192+0000","last_peered":"2024-03-29T04:17:44.634192+0000","last_clean":"2024-03-29T04:17:44.634192+0000","last_became_active":"2024-03-29T04:17:35.601902+0000","last_became_peered":"2024-03-29T04:17:35.601902+0000","last_unstale":"2024-03-29T04:17:44.634192+0000","last_undegraded":"2024-03-29T04:17:44.634192+0000","last_fullsized":"2024-03-29T04:17:44.634192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:43:57.713086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.658621+0000","last_change":"2024-03-29T04:17:40.661925+0000","last_active":"2024-03-29T04:17:42.658621+0000","last_peered":"2024-03-29T04:17:42.658621+0000","last_clean":"2024-03-29T04:17:42.658621+0000","last_became_active":"2024-03-29T04:17:40.661028+0000","last_became_peered":"2024-03-29T04:17:40.661028+0000","last_unstale":"2024-03-29T04:17:42.658621+0000","last_undegraded":"2024-03-29T04:17:42.658621+0000","last_fullsized":"2024-03-29T04:17:42.658621+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:33:27.957344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650137+0000","last_change":"2024-03-29T04:17:36.620569+0000","last_active":"2024-03-29T04:17:42.650137+0000","last_peered":"2024-03-29T04:17:42.650137+0000","last_clean":"2024-03-29T04:17:42.650137+0000","last_became_active":"2024-03-29T04:17:36.620079+0000","last_became_peered":"2024-03-29T04:17:36.620079+0000","last_unstale":"2024-03-29T04:17:42.650137+0000","last_undegraded":"2024-03-29T04:17:42.650137+0000","last_fullsized":"2024-03-29T04:17:42.650137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:41.872399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.469887+0000","last_change":"2024-03-29T04:17:35.577167+0000","last_active":"2024-03-29T04:17:44.469887+0000","last_peered":"2024-03-29T04:17:44.469887+0000","last_clean":"2024-03-29T04:17:44.469887+0000","last_became_active":"2024-03-29T04:17:35.576145+0000","last_became_peered":"2024-03-29T04:17:35.576145+0000","last_unstale":"2024-03-29T04:17:44.469887+0000","last_undegraded":"2024-03-29T04:17:44.469887+0000","last_fullsized":"2024-03-29T04:17:44.469887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:11:43.663978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.657612+0000","last_change":"2024-03-29T04:17:40.664962+0000","last_active":"2024-03-29T04:17:42.657612+0000","last_peered":"2024-03-29T04:17:42.657612+0000","last_clean":"2024-03-29T04:17:42.657612+0000","last_became_active":"2024-03-29T04:17:40.664470+0000","last_became_peered":"2024-03-29T04:17:40.664470+0000","last_unstale":"2024-03-29T04:17:42.657612+0000","last_undegraded":"2024-03-29T04:17:42.657612+0000","last_fullsized":"2024-03-29T04:17:42.657612+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:43:44.093178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650038+0000","last_change":"2024-03-29T04:17:36.617517+0000","last_active":"2024-03-29T04:17:42.650038+0000","last_peered":"2024-03-29T04:17:42.650038+0000","last_clean":"2024-03-29T04:17:42.650038+0000","last_became_active":"2024-03-29T04:17:36.616399+0000","last_became_peered":"2024-03-29T04:17:36.616399+0000","last_unstale":"2024-03-29T04:17:42.650038+0000","last_undegraded":"2024-03-29T04:17:42.650038+0000","last_fullsized":"2024-03-29T04:17:42.650038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:24:47.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470278+0000","last_change":"2024-03-29T04:17:35.576729+0000","last_active":"2024-03-29T04:17:44.470278+0000","last_peered":"2024-03-29T04:17:44.470278+0000","last_clean":"2024-03-29T04:17:44.470278+0000","last_became_active":"2024-03-29T04:17:35.576446+0000","last_became_peered":"2024-03-29T04:17:35.576446+0000","last_unstale":"2024-03-29T04:17:44.470278+0000","last_undegraded":"2024-03-29T04:17:44.470278+0000","last_fullsized":"2024-03-29T04:17:44.470278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:53.508449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470815+0000","last_change":"2024-03-29T04:17:41.170866+0000","last_active":"2024-03-29T04:17:44.470815+0000","last_peered":"2024-03-29T04:17:44.470815+0000","last_clean":"2024-03-29T04:17:44.470815+0000","last_became_active":"2024-03-29T04:17:41.169521+0000","last_became_peered":"2024-03-29T04:17:41.169521+0000","last_unstale":"2024-03-29T04:17:44.470815+0000","last_undegraded":"2024-03-29T04:17:44.470815+0000","last_fullsized":"2024-03-29T04:17:44.470815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:33:27.957344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.657095+0000","last_change":"2024-03-29T04:17:36.624392+0000","last_active":"2024-03-29T04:17:42.657095+0000","last_peered":"2024-03-29T04:17:42.657095+0000","last_clean":"2024-03-29T04:17:42.657095+0000","last_became_active":"2024-03-29T04:17:36.612708+0000","last_became_peered":"2024-03-29T04:17:36.612708+0000","last_unstale":"2024-03-29T04:17:42.657095+0000","last_undegraded":"2024-03-29T04:17:42.657095+0000","last_fullsized":"2024-03-29T04:17:42.657095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:59.355649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634144+0000","last_change":"2024-03-29T04:17:35.596985+0000","last_active":"2024-03-29T04:17:44.634144+0000","last_peered":"2024-03-29T04:17:44.634144+0000","last_clean":"2024-03-29T04:17:44.634144+0000","last_became_active":"2024-03-29T04:17:35.595917+0000","last_became_peered":"2024-03-29T04:17:35.595917+0000","last_unstale":"2024-03-29T04:17:44.634144+0000","last_undegraded":"2024-03-29T04:17:44.634144+0000","last_fullsized":"2024-03-29T04:17:44.634144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:24:46.122034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.658384+0000","last_change":"2024-03-29T04:17:40.664618+0000","last_active":"2024-03-29T04:17:42.658384+0000","last_peered":"2024-03-29T04:17:42.658384+0000","last_clean":"2024-03-29T04:17:42.658384+0000","last_became_active":"2024-03-29T04:17:40.664144+0000","last_became_peered":"2024-03-29T04:17:40.664144+0000","last_unstale":"2024-03-29T04:17:42.658384+0000","last_undegraded":"2024-03-29T04:17:42.658384+0000","last_fullsized":"2024-03-29T04:17:42.658384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:59:47.099511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653827+0000","last_change":"2024-03-29T04:17:36.596925+0000","last_active":"2024-03-29T04:17:42.653827+0000","last_peered":"2024-03-29T04:17:42.653827+0000","last_clean":"2024-03-29T04:17:42.653827+0000","last_became_active":"2024-03-29T04:17:36.596307+0000","last_became_peered":"2024-03-29T04:17:36.596307+0000","last_unstale":"2024-03-29T04:17:42.653827+0000","last_undegraded":"2024-03-29T04:17:42.653827+0000","last_fullsized":"2024-03-29T04:17:42.653827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:39.635039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.271188+0000","last_change":"2024-03-29T04:17:35.598885+0000","last_active":"2024-03-29T04:17:44.271188+0000","last_peered":"2024-03-29T04:17:44.271188+0000","last_clean":"2024-03-29T04:17:44.271188+0000","last_became_active":"2024-03-29T04:17:35.598155+0000","last_became_peered":"2024-03-29T04:17:35.598155+0000","last_unstale":"2024-03-29T04:17:44.271188+0000","last_undegraded":"2024-03-29T04:17:44.271188+0000","last_fullsized":"2024-03-29T04:17:44.271188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:58.346822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656759+0000","last_change":"2024-03-29T04:17:41.164185+0000","last_active":"2024-03-29T04:17:42.656759+0000","last_peered":"2024-03-29T04:17:42.656759+0000","last_clean":"2024-03-29T04:17:42.656759+0000","last_became_active":"2024-03-29T04:17:41.163368+0000","last_became_peered":"2024-03-29T04:17:41.163368+0000","last_unstale":"2024-03-29T04:17:42.656759+0000","last_undegraded":"2024-03-29T04:17:42.656759+0000","last_fullsized":"2024-03-29T04:17:42.656759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:04:15.384842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650724+0000","last_change":"2024-03-29T04:17:36.611583+0000","last_active":"2024-03-29T04:17:42.650724+0000","last_peered":"2024-03-29T04:17:42.650724+0000","last_clean":"2024-03-29T04:17:42.650724+0000","last_became_active":"2024-03-29T04:17:36.611204+0000","last_became_peered":"2024-03-29T04:17:36.611204+0000","last_unstale":"2024-03-29T04:17:42.650724+0000","last_undegraded":"2024-03-29T04:17:42.650724+0000","last_fullsized":"2024-03-29T04:17:42.650724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:13.677092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470861+0000","last_change":"2024-03-29T04:17:35.574541+0000","last_active":"2024-03-29T04:17:44.470861+0000","last_peered":"2024-03-29T04:17:44.470861+0000","last_clean":"2024-03-29T04:17:44.470861+0000","last_became_active":"2024-03-29T04:17:35.573297+0000","last_became_peered":"2024-03-29T04:17:35.573297+0000","last_unstale":"2024-03-29T04:17:44.470861+0000","last_undegraded":"2024-03-29T04:17:44.470861+0000","last_fullsized":"2024-03-29T04:17:44.470861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:58.346822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649700+0000","last_change":"2024-03-29T04:17:41.165155+0000","last_active":"2024-03-29T04:17:42.649700+0000","last_peered":"2024-03-29T04:17:42.649700+0000","last_clean":"2024-03-29T04:17:42.649700+0000","last_became_active":"2024-03-29T04:17:41.163801+0000","last_became_peered":"2024-03-29T04:17:41.163801+0000","last_unstale":"2024-03-29T04:17:42.649700+0000","last_undegraded":"2024-03-29T04:17:42.649700+0000","last_fullsized":"2024-03-29T04:17:42.649700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:05:37.595239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470679+0000","last_change":"2024-03-29T04:17:36.590406+0000","last_active":"2024-03-29T04:17:44.470679+0000","last_peered":"2024-03-29T04:17:44.470679+0000","last_clean":"2024-03-29T04:17:44.470679+0000","last_became_active":"2024-03-29T04:17:36.589975+0000","last_became_peered":"2024-03-29T04:17:36.589975+0000","last_unstale":"2024-03-29T04:17:44.470679+0000","last_undegraded":"2024-03-29T04:17:44.470679+0000","last_fullsized":"2024-03-29T04:17:44.470679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:39.635039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634555+0000","last_change":"2024-03-29T04:17:35.602939+0000","last_active":"2024-03-29T04:17:44.634555+0000","last_peered":"2024-03-29T04:17:44.634555+0000","last_clean":"2024-03-29T04:17:44.634555+0000","last_became_active":"2024-03-29T04:17:35.602250+0000","last_became_peered":"2024-03-29T04:17:35.602250+0000","last_unstale":"2024-03-29T04:17:44.634555+0000","last_undegraded":"2024-03-29T04:17:44.634555+0000","last_fullsized":"2024-03-29T04:17:44.634555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:16:13.431728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653145+0000","last_change":"2024-03-29T04:17:41.165561+0000","last_active":"2024-03-29T04:17:42.653145+0000","last_peered":"2024-03-29T04:17:42.653145+0000","last_clean":"2024-03-29T04:17:42.653145+0000","last_became_active":"2024-03-29T04:17:41.163703+0000","last_became_peered":"2024-03-29T04:17:41.163703+0000","last_unstale":"2024-03-29T04:17:42.653145+0000","last_undegraded":"2024-03-29T04:17:42.653145+0000","last_fullsized":"2024-03-29T04:17:42.653145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:10:04.139549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553157+0000","last_change":"2024-03-29T04:17:36.602865+0000","last_active":"2024-03-29T04:17:43.553157+0000","last_peered":"2024-03-29T04:17:43.553157+0000","last_clean":"2024-03-29T04:17:43.553157+0000","last_became_active":"2024-03-29T04:17:36.602578+0000","last_became_peered":"2024-03-29T04:17:36.602578+0000","last_unstale":"2024-03-29T04:17:43.553157+0000","last_undegraded":"2024-03-29T04:17:43.553157+0000","last_fullsized":"2024-03-29T04:17:43.553157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:55:11.413735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.469877+0000","last_change":"2024-03-29T04:17:35.560382+0000","last_active":"2024-03-29T04:17:44.469877+0000","last_peered":"2024-03-29T04:17:44.469877+0000","last_clean":"2024-03-29T04:17:44.469877+0000","last_became_active":"2024-03-29T04:17:35.560122+0000","last_became_peered":"2024-03-29T04:17:35.560122+0000","last_unstale":"2024-03-29T04:17:44.469877+0000","last_undegraded":"2024-03-29T04:17:44.469877+0000","last_fullsized":"2024-03-29T04:17:44.469877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:52:18.910727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470397+0000","last_change":"2024-03-29T04:17:40.663870+0000","last_active":"2024-03-29T04:17:44.470397+0000","last_peered":"2024-03-29T04:17:44.470397+0000","last_clean":"2024-03-29T04:17:44.470397+0000","last_became_active":"2024-03-29T04:17:40.662957+0000","last_became_peered":"2024-03-29T04:17:40.662957+0000","last_unstale":"2024-03-29T04:17:44.470397+0000","last_undegraded":"2024-03-29T04:17:44.470397+0000","last_fullsized":"2024-03-29T04:17:44.470397+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:10:04.139549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.657408+0000","last_change":"2024-03-29T04:17:36.624632+0000","last_active":"2024-03-29T04:17:42.657408+0000","last_peered":"2024-03-29T04:17:42.657408+0000","last_clean":"2024-03-29T04:17:42.657408+0000","last_became_active":"2024-03-29T04:17:36.621308+0000","last_became_peered":"2024-03-29T04:17:36.621308+0000","last_unstale":"2024-03-29T04:17:42.657408+0000","last_undegraded":"2024-03-29T04:17:42.657408+0000","last_fullsized":"2024-03-29T04:17:42.657408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:11:44.672805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553434+0000","last_change":"2024-03-29T04:17:35.590306+0000","last_active":"2024-03-29T04:17:43.553434+0000","last_peered":"2024-03-29T04:17:43.553434+0000","last_clean":"2024-03-29T04:17:43.553434+0000","last_became_active":"2024-03-29T04:17:35.582422+0000","last_became_peered":"2024-03-29T04:17:35.582422+0000","last_unstale":"2024-03-29T04:17:43.553434+0000","last_undegraded":"2024-03-29T04:17:43.553434+0000","last_fullsized":"2024-03-29T04:17:43.553434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:58.346822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470758+0000","last_change":"2024-03-29T04:17:40.661919+0000","last_active":"2024-03-29T04:17:44.470758+0000","last_peered":"2024-03-29T04:17:44.470758+0000","last_clean":"2024-03-29T04:17:44.470758+0000","last_became_active":"2024-03-29T04:17:40.661251+0000","last_became_peered":"2024-03-29T04:17:40.661251+0000","last_unstale":"2024-03-29T04:17:44.470758+0000","last_undegraded":"2024-03-29T04:17:44.470758+0000","last_fullsized":"2024-03-29T04:17:44.470758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:59:47.099511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470257+0000","last_change":"2024-03-29T04:17:36.590933+0000","last_active":"2024-03-29T04:17:44.470257+0000","last_peered":"2024-03-29T04:17:44.470257+0000","last_clean":"2024-03-29T04:17:44.470257+0000","last_became_active":"2024-03-29T04:17:36.590535+0000","last_became_peered":"2024-03-29T04:17:36.590535+0000","last_unstale":"2024-03-29T04:17:44.470257+0000","last_undegraded":"2024-03-29T04:17:44.470257+0000","last_fullsized":"2024-03-29T04:17:44.470257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:41.872399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578021+0000","last_change":"2024-03-29T04:17:35.565257+0000","last_active":"2024-03-29T04:17:43.578021+0000","last_peered":"2024-03-29T04:17:43.578021+0000","last_clean":"2024-03-29T04:17:43.578021+0000","last_became_active":"2024-03-29T04:17:35.564547+0000","last_became_peered":"2024-03-29T04:17:35.564547+0000","last_unstale":"2024-03-29T04:17:43.578021+0000","last_undegraded":"2024-03-29T04:17:43.578021+0000","last_fullsized":"2024-03-29T04:17:43.578021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:04:44.150824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470921+0000","last_change":"2024-03-29T04:17:40.657201+0000","last_active":"2024-03-29T04:17:44.470921+0000","last_peered":"2024-03-29T04:17:44.470921+0000","last_clean":"2024-03-29T04:17:44.470921+0000","last_became_active":"2024-03-29T04:17:40.655333+0000","last_became_peered":"2024-03-29T04:17:40.655333+0000","last_unstale":"2024-03-29T04:17:44.470921+0000","last_undegraded":"2024-03-29T04:17:44.470921+0000","last_fullsized":"2024-03-29T04:17:44.470921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:37:53.228170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.652829+0000","last_change":"2024-03-29T04:17:36.625393+0000","last_active":"2024-03-29T04:17:42.652829+0000","last_peered":"2024-03-29T04:17:42.652829+0000","last_clean":"2024-03-29T04:17:42.652829+0000","last_became_active":"2024-03-29T04:17:36.625053+0000","last_became_peered":"2024-03-29T04:17:36.625053+0000","last_unstale":"2024-03-29T04:17:42.652829+0000","last_undegraded":"2024-03-29T04:17:42.652829+0000","last_fullsized":"2024-03-29T04:17:42.652829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:16:58.574477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.635249+0000","last_change":"2024-03-29T04:17:35.597695+0000","last_active":"2024-03-29T04:17:44.635249+0000","last_peered":"2024-03-29T04:17:44.635249+0000","last_clean":"2024-03-29T04:17:44.635249+0000","last_became_active":"2024-03-29T04:17:35.596065+0000","last_became_peered":"2024-03-29T04:17:35.596065+0000","last_unstale":"2024-03-29T04:17:44.635249+0000","last_undegraded":"2024-03-29T04:17:44.635249+0000","last_fullsized":"2024-03-29T04:17:44.635249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:11:43.663978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653676+0000","last_change":"2024-03-29T04:17:41.169719+0000","last_active":"2024-03-29T04:17:42.653676+0000","last_peered":"2024-03-29T04:17:42.653676+0000","last_clean":"2024-03-29T04:17:42.653676+0000","last_became_active":"2024-03-29T04:17:41.168241+0000","last_became_peered":"2024-03-29T04:17:41.168241+0000","last_unstale":"2024-03-29T04:17:42.653676+0000","last_undegraded":"2024-03-29T04:17:42.653676+0000","last_fullsized":"2024-03-29T04:17:42.653676+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:37:53.228170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649994+0000","last_change":"2024-03-29T04:17:36.599562+0000","last_active":"2024-03-29T04:17:42.649994+0000","last_peered":"2024-03-29T04:17:42.649994+0000","last_clean":"2024-03-29T04:17:42.649994+0000","last_became_active":"2024-03-29T04:17:36.597078+0000","last_became_peered":"2024-03-29T04:17:36.597078+0000","last_unstale":"2024-03-29T04:17:42.649994+0000","last_undegraded":"2024-03-29T04:17:42.649994+0000","last_fullsized":"2024-03-29T04:17:42.649994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:43:27.568676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553439+0000","last_change":"2024-03-29T04:17:35.589469+0000","last_active":"2024-03-29T04:17:43.553439+0000","last_peered":"2024-03-29T04:17:43.553439+0000","last_clean":"2024-03-29T04:17:43.553439+0000","last_became_active":"2024-03-29T04:17:35.572474+0000","last_became_peered":"2024-03-29T04:17:35.572474+0000","last_unstale":"2024-03-29T04:17:43.553439+0000","last_undegraded":"2024-03-29T04:17:43.553439+0000","last_fullsized":"2024-03-29T04:17:43.553439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:55:50.253366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.635415+0000","last_change":"2024-03-29T04:17:25.361826+0000","last_active":"2024-03-29T04:17:44.635415+0000","last_peered":"2024-03-29T04:17:44.635415+0000","last_clean":"2024-03-29T04:17:44.635415+0000","last_became_active":"2024-03-29T04:17:25.361259+0000","last_became_peered":"2024-03-29T04:17:25.361259+0000","last_unstale":"2024-03-29T04:17:44.635415+0000","last_undegraded":"2024-03-29T04:17:44.635415+0000","last_fullsized":"2024-03-29T04:17:44.635415+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_clean_scrub_stamp":"2024-03-29T04:17:24.334639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:40.027873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270077+0000","last_change":"2024-03-29T04:17:41.167132+0000","last_active":"2024-03-29T04:17:44.270077+0000","last_peered":"2024-03-29T04:17:44.270077+0000","last_clean":"2024-03-29T04:17:44.270077+0000","last_became_active":"2024-03-29T04:17:41.166660+0000","last_became_peered":"2024-03-29T04:17:41.166660+0000","last_unstale":"2024-03-29T04:17:44.270077+0000","last_undegraded":"2024-03-29T04:17:44.270077+0000","last_fullsized":"2024-03-29T04:17:44.270077+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:42:25.113757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634556+0000","last_change":"2024-03-29T04:17:36.612535+0000","last_active":"2024-03-29T04:17:44.634556+0000","last_peered":"2024-03-29T04:17:44.634556+0000","last_clean":"2024-03-29T04:17:44.634556+0000","last_became_active":"2024-03-29T04:17:36.610106+0000","last_became_peered":"2024-03-29T04:17:36.610106+0000","last_unstale":"2024-03-29T04:17:44.634556+0000","last_undegraded":"2024-03-29T04:17:44.634556+0000","last_fullsized":"2024-03-29T04:17:44.634556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:24:58.639606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578939+0000","last_change":"2024-03-29T04:17:35.600209+0000","last_active":"2024-03-29T04:17:43.578939+0000","last_peered":"2024-03-29T04:17:43.578939+0000","last_clean":"2024-03-29T04:17:43.578939+0000","last_became_active":"2024-03-29T04:17:35.599686+0000","last_became_peered":"2024-03-29T04:17:35.599686+0000","last_unstale":"2024-03-29T04:17:43.578939+0000","last_undegraded":"2024-03-29T04:17:43.578939+0000","last_fullsized":"2024-03-29T04:17:43.578939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:55:50.253366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.269503+0000","last_change":"2024-03-29T04:17:41.165901+0000","last_active":"2024-03-29T04:17:44.269503+0000","last_peered":"2024-03-29T04:17:44.269503+0000","last_clean":"2024-03-29T04:17:44.269503+0000","last_became_active":"2024-03-29T04:17:41.165453+0000","last_became_peered":"2024-03-29T04:17:41.165453+0000","last_unstale":"2024-03-29T04:17:44.269503+0000","last_undegraded":"2024-03-29T04:17:44.269503+0000","last_fullsized":"2024-03-29T04:17:44.269503+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:04:15.384842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553553+0000","last_change":"2024-03-29T04:17:36.596650+0000","last_active":"2024-03-29T04:17:43.553553+0000","last_peered":"2024-03-29T04:17:43.553553+0000","last_clean":"2024-03-29T04:17:43.553553+0000","last_became_active":"2024-03-29T04:17:36.596458+0000","last_became_peered":"2024-03-29T04:17:36.596458+0000","last_unstale":"2024-03-29T04:17:43.553553+0000","last_undegraded":"2024-03-29T04:17:43.553553+0000","last_fullsized":"2024-03-29T04:17:43.553553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:42:46.422245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.469894+0000","last_change":"2024-03-29T04:17:35.554693+0000","last_active":"2024-03-29T04:17:44.469894+0000","last_peered":"2024-03-29T04:17:44.469894+0000","last_clean":"2024-03-29T04:17:44.469894+0000","last_became_active":"2024-03-29T04:17:35.554347+0000","last_became_peered":"2024-03-29T04:17:35.554347+0000","last_unstale":"2024-03-29T04:17:44.469894+0000","last_undegraded":"2024-03-29T04:17:44.469894+0000","last_fullsized":"2024-03-29T04:17:44.469894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:24:46.122034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653319+0000","last_change":"2024-03-29T04:17:40.656032+0000","last_active":"2024-03-29T04:17:42.653319+0000","last_peered":"2024-03-29T04:17:42.653319+0000","last_clean":"2024-03-29T04:17:42.653319+0000","last_became_active":"2024-03-29T04:17:40.654947+0000","last_became_peered":"2024-03-29T04:17:40.654947+0000","last_unstale":"2024-03-29T04:17:42.653319+0000","last_undegraded":"2024-03-29T04:17:42.653319+0000","last_fullsized":"2024-03-29T04:17:42.653319+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:59:47.099511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634965+0000","last_change":"2024-03-29T04:17:36.627875+0000","last_active":"2024-03-29T04:17:44.634965+0000","last_peered":"2024-03-29T04:17:44.634965+0000","last_clean":"2024-03-29T04:17:44.634965+0000","last_became_active":"2024-03-29T04:17:36.627498+0000","last_became_peered":"2024-03-29T04:17:36.627498+0000","last_unstale":"2024-03-29T04:17:44.634965+0000","last_undegraded":"2024-03-29T04:17:44.634965+0000","last_fullsized":"2024-03-29T04:17:44.634965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:10:00.061365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578092+0000","last_change":"2024-03-29T04:17:35.568135+0000","last_active":"2024-03-29T04:17:43.578092+0000","last_peered":"2024-03-29T04:17:43.578092+0000","last_clean":"2024-03-29T04:17:43.578092+0000","last_became_active":"2024-03-29T04:17:35.567800+0000","last_became_peered":"2024-03-29T04:17:35.567800+0000","last_unstale":"2024-03-29T04:17:43.578092+0000","last_undegraded":"2024-03-29T04:17:43.578092+0000","last_fullsized":"2024-03-29T04:17:43.578092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:23.283694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552529+0000","last_change":"2024-03-29T04:17:41.166538+0000","last_active":"2024-03-29T04:17:43.552529+0000","last_peered":"2024-03-29T04:17:43.552529+0000","last_clean":"2024-03-29T04:17:43.552529+0000","last_became_active":"2024-03-29T04:17:41.165914+0000","last_became_peered":"2024-03-29T04:17:41.165914+0000","last_unstale":"2024-03-29T04:17:43.552529+0000","last_undegraded":"2024-03-29T04:17:43.552529+0000","last_fullsized":"2024-03-29T04:17:43.552529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:37:53.228170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634358+0000","last_change":"2024-03-29T04:17:36.626068+0000","last_active":"2024-03-29T04:17:44.634358+0000","last_peered":"2024-03-29T04:17:44.634358+0000","last_clean":"2024-03-29T04:17:44.634358+0000","last_became_active":"2024-03-29T04:17:36.625776+0000","last_became_peered":"2024-03-29T04:17:36.625776+0000","last_unstale":"2024-03-29T04:17:44.634358+0000","last_undegraded":"2024-03-29T04:17:44.634358+0000","last_fullsized":"2024-03-29T04:17:44.634358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:55:11.413735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.658456+0000","last_change":"2024-03-29T04:17:35.573245+0000","last_active":"2024-03-29T04:17:42.658456+0000","last_peered":"2024-03-29T04:17:42.658456+0000","last_clean":"2024-03-29T04:17:42.658456+0000","last_became_active":"2024-03-29T04:17:35.572324+0000","last_became_peered":"2024-03-29T04:17:35.572324+0000","last_unstale":"2024-03-29T04:17:42.658456+0000","last_undegraded":"2024-03-29T04:17:42.658456+0000","last_fullsized":"2024-03-29T04:17:42.658456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:07:32.265670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656484+0000","last_change":"2024-03-29T04:17:40.653790+0000","last_active":"2024-03-29T04:17:42.656484+0000","last_peered":"2024-03-29T04:17:42.656484+0000","last_clean":"2024-03-29T04:17:42.656484+0000","last_became_active":"2024-03-29T04:17:40.653115+0000","last_became_peered":"2024-03-29T04:17:40.653115+0000","last_unstale":"2024-03-29T04:17:42.656484+0000","last_undegraded":"2024-03-29T04:17:42.656484+0000","last_fullsized":"2024-03-29T04:17:42.656484+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:55:15.491920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634122+0000","last_change":"2024-03-29T04:17:36.625368+0000","last_active":"2024-03-29T04:17:44.634122+0000","last_peered":"2024-03-29T04:17:44.634122+0000","last_clean":"2024-03-29T04:17:44.634122+0000","last_became_active":"2024-03-29T04:17:36.625073+0000","last_became_peered":"2024-03-29T04:17:36.625073+0000","last_unstale":"2024-03-29T04:17:44.634122+0000","last_undegraded":"2024-03-29T04:17:44.634122+0000","last_fullsized":"2024-03-29T04:17:44.634122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:13.677092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578401+0000","last_change":"2024-03-29T04:17:35.564073+0000","last_active":"2024-03-29T04:17:43.578401+0000","last_peered":"2024-03-29T04:17:43.578401+0000","last_clean":"2024-03-29T04:17:43.578401+0000","last_became_active":"2024-03-29T04:17:35.563521+0000","last_became_peered":"2024-03-29T04:17:35.563521+0000","last_unstale":"2024-03-29T04:17:43.578401+0000","last_undegraded":"2024-03-29T04:17:43.578401+0000","last_fullsized":"2024-03-29T04:17:43.578401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:53.508449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653476+0000","last_change":"2024-03-29T04:17:40.648424+0000","last_active":"2024-03-29T04:17:42.653476+0000","last_peered":"2024-03-29T04:17:42.653476+0000","last_clean":"2024-03-29T04:17:42.653476+0000","last_became_active":"2024-03-29T04:17:40.647665+0000","last_became_peered":"2024-03-29T04:17:40.647665+0000","last_unstale":"2024-03-29T04:17:42.653476+0000","last_undegraded":"2024-03-29T04:17:42.653476+0000","last_fullsized":"2024-03-29T04:17:42.653476+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:33:27.957344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.269744+0000","last_change":"2024-03-29T04:17:36.614419+0000","last_active":"2024-03-29T04:17:44.269744+0000","last_peered":"2024-03-29T04:17:44.269744+0000","last_clean":"2024-03-29T04:17:44.269744+0000","last_became_active":"2024-03-29T04:17:36.614052+0000","last_became_peered":"2024-03-29T04:17:36.614052+0000","last_unstale":"2024-03-29T04:17:44.269744+0000","last_undegraded":"2024-03-29T04:17:44.269744+0000","last_fullsized":"2024-03-29T04:17:44.269744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:41:31.910133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.268906+0000","last_change":"2024-03-29T04:17:35.576576+0000","last_active":"2024-03-29T04:17:44.268906+0000","last_peered":"2024-03-29T04:17:44.268906+0000","last_clean":"2024-03-29T04:17:44.268906+0000","last_became_active":"2024-03-29T04:17:35.575156+0000","last_became_peered":"2024-03-29T04:17:35.575156+0000","last_unstale":"2024-03-29T04:17:44.268906+0000","last_undegraded":"2024-03-29T04:17:44.268906+0000","last_fullsized":"2024-03-29T04:17:44.268906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:52:18.910727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650517+0000","last_change":"2024-03-29T04:17:41.165455+0000","last_active":"2024-03-29T04:17:42.650517+0000","last_peered":"2024-03-29T04:17:42.650517+0000","last_clean":"2024-03-29T04:17:42.650517+0000","last_became_active":"2024-03-29T04:17:41.164305+0000","last_became_peered":"2024-03-29T04:17:41.164305+0000","last_unstale":"2024-03-29T04:17:42.650517+0000","last_undegraded":"2024-03-29T04:17:42.650517+0000","last_fullsized":"2024-03-29T04:17:42.650517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:58:14.948478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634370+0000","last_change":"2024-03-29T04:17:36.626514+0000","last_active":"2024-03-29T04:17:44.634370+0000","last_peered":"2024-03-29T04:17:44.634370+0000","last_clean":"2024-03-29T04:17:44.634370+0000","last_became_active":"2024-03-29T04:17:36.626184+0000","last_became_peered":"2024-03-29T04:17:36.626184+0000","last_unstale":"2024-03-29T04:17:44.634370+0000","last_undegraded":"2024-03-29T04:17:44.634370+0000","last_fullsized":"2024-03-29T04:17:44.634370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:42:46.422245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553262+0000","last_change":"2024-03-29T04:17:35.599574+0000","last_active":"2024-03-29T04:17:43.553262+0000","last_peered":"2024-03-29T04:17:43.553262+0000","last_clean":"2024-03-29T04:17:43.553262+0000","last_became_active":"2024-03-29T04:17:35.589264+0000","last_became_peered":"2024-03-29T04:17:35.589264+0000","last_unstale":"2024-03-29T04:17:43.553262+0000","last_undegraded":"2024-03-29T04:17:43.553262+0000","last_fullsized":"2024-03-29T04:17:43.553262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:52:18.910727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650586+0000","last_change":"2024-03-29T04:17:41.164863+0000","last_active":"2024-03-29T04:17:42.650586+0000","last_peered":"2024-03-29T04:17:42.650586+0000","last_clean":"2024-03-29T04:17:42.650586+0000","last_became_active":"2024-03-29T04:17:41.163756+0000","last_became_peered":"2024-03-29T04:17:41.163756+0000","last_unstale":"2024-03-29T04:17:42.650586+0000","last_undegraded":"2024-03-29T04:17:42.650586+0000","last_fullsized":"2024-03-29T04:17:42.650586+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:54.890778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653149+0000","last_change":"2024-03-29T04:17:36.611832+0000","last_active":"2024-03-29T04:17:42.653149+0000","last_peered":"2024-03-29T04:17:42.653149+0000","last_clean":"2024-03-29T04:17:42.653149+0000","last_became_active":"2024-03-29T04:17:36.611106+0000","last_became_peered":"2024-03-29T04:17:36.611106+0000","last_unstale":"2024-03-29T04:17:42.653149+0000","last_undegraded":"2024-03-29T04:17:42.653149+0000","last_fullsized":"2024-03-29T04:17:42.653149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:10:00.802551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653387+0000","last_change":"2024-03-29T04:17:35.569650+0000","last_active":"2024-03-29T04:17:42.653387+0000","last_peered":"2024-03-29T04:17:42.653387+0000","last_clean":"2024-03-29T04:17:42.653387+0000","last_became_active":"2024-03-29T04:17:35.568995+0000","last_became_peered":"2024-03-29T04:17:35.568995+0000","last_unstale":"2024-03-29T04:17:42.653387+0000","last_undegraded":"2024-03-29T04:17:42.653387+0000","last_fullsized":"2024-03-29T04:17:42.653387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:55:50.253366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552920+0000","last_change":"2024-03-29T04:17:41.166364+0000","last_active":"2024-03-29T04:17:43.552920+0000","last_peered":"2024-03-29T04:17:43.552920+0000","last_clean":"2024-03-29T04:17:43.552920+0000","last_became_active":"2024-03-29T04:17:41.165953+0000","last_became_peered":"2024-03-29T04:17:41.165953+0000","last_unstale":"2024-03-29T04:17:43.552920+0000","last_undegraded":"2024-03-29T04:17:43.552920+0000","last_fullsized":"2024-03-29T04:17:43.552920+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:58.692538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634707+0000","last_change":"2024-03-29T04:17:36.610409+0000","last_active":"2024-03-29T04:17:44.634707+0000","last_peered":"2024-03-29T04:17:44.634707+0000","last_clean":"2024-03-29T04:17:44.634707+0000","last_became_active":"2024-03-29T04:17:36.609783+0000","last_became_peered":"2024-03-29T04:17:36.609783+0000","last_unstale":"2024-03-29T04:17:44.634707+0000","last_undegraded":"2024-03-29T04:17:44.634707+0000","last_fullsized":"2024-03-29T04:17:44.634707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:46:38.204638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:47.106160+0000","last_change":"2024-03-29T04:17:35.576067+0000","last_active":"2024-03-29T04:17:47.106160+0000","last_peered":"2024-03-29T04:17:47.106160+0000","last_clean":"2024-03-29T04:17:47.106160+0000","last_became_active":"2024-03-29T04:17:35.573099+0000","last_became_peered":"2024-03-29T04:17:35.573099+0000","last_unstale":"2024-03-29T04:17:47.106160+0000","last_undegraded":"2024-03-29T04:17:47.106160+0000","last_fullsized":"2024-03-29T04:17:47.106160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:53.508449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470529+0000","last_change":"2024-03-29T04:17:40.658773+0000","last_active":"2024-03-29T04:17:44.470529+0000","last_peered":"2024-03-29T04:17:44.470529+0000","last_clean":"2024-03-29T04:17:44.470529+0000","last_became_active":"2024-03-29T04:17:40.658277+0000","last_became_peered":"2024-03-29T04:17:40.658277+0000","last_unstale":"2024-03-29T04:17:44.470529+0000","last_undegraded":"2024-03-29T04:17:44.470529+0000","last_fullsized":"2024-03-29T04:17:44.470529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:43:44.093178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.658046+0000","last_change":"2024-03-29T04:17:36.620386+0000","last_active":"2024-03-29T04:17:42.658046+0000","last_peered":"2024-03-29T04:17:42.658046+0000","last_clean":"2024-03-29T04:17:42.658046+0000","last_became_active":"2024-03-29T04:17:36.620093+0000","last_became_peered":"2024-03-29T04:17:36.620093+0000","last_unstale":"2024-03-29T04:17:42.658046+0000","last_undegraded":"2024-03-29T04:17:42.658046+0000","last_fullsized":"2024-03-29T04:17:42.658046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:12:37.599253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.469929+0000","last_change":"2024-03-29T04:17:35.592723+0000","last_active":"2024-03-29T04:17:44.469929+0000","last_peered":"2024-03-29T04:17:44.469929+0000","last_clean":"2024-03-29T04:17:44.469929+0000","last_became_active":"2024-03-29T04:17:35.592409+0000","last_became_peered":"2024-03-29T04:17:35.592409+0000","last_unstale":"2024-03-29T04:17:44.469929+0000","last_undegraded":"2024-03-29T04:17:44.469929+0000","last_fullsized":"2024-03-29T04:17:44.469929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:12:36.590426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.657265+0000","last_change":"2024-03-29T04:17:40.654909+0000","last_active":"2024-03-29T04:17:42.657265+0000","last_peered":"2024-03-29T04:17:42.657265+0000","last_clean":"2024-03-29T04:17:42.657265+0000","last_became_active":"2024-03-29T04:17:40.654420+0000","last_became_peered":"2024-03-29T04:17:40.654420+0000","last_unstale":"2024-03-29T04:17:42.657265+0000","last_undegraded":"2024-03-29T04:17:42.657265+0000","last_fullsized":"2024-03-29T04:17:42.657265+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:10:04.139549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653614+0000","last_change":"2024-03-29T04:17:36.599274+0000","last_active":"2024-03-29T04:17:42.653614+0000","last_peered":"2024-03-29T04:17:42.653614+0000","last_clean":"2024-03-29T04:17:42.653614+0000","last_became_active":"2024-03-29T04:17:36.597822+0000","last_became_peered":"2024-03-29T04:17:36.597822+0000","last_unstale":"2024-03-29T04:17:42.653614+0000","last_undegraded":"2024-03-29T04:17:42.653614+0000","last_fullsized":"2024-03-29T04:17:42.653614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:24:47.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553100+0000","last_change":"2024-03-29T04:17:35.592120+0000","last_active":"2024-03-29T04:17:43.553100+0000","last_peered":"2024-03-29T04:17:43.553100+0000","last_clean":"2024-03-29T04:17:43.553100+0000","last_became_active":"2024-03-29T04:17:35.590699+0000","last_became_peered":"2024-03-29T04:17:35.590699+0000","last_unstale":"2024-03-29T04:17:43.553100+0000","last_undegraded":"2024-03-29T04:17:43.553100+0000","last_fullsized":"2024-03-29T04:17:43.553100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:53.508449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649946+0000","last_change":"2024-03-29T04:17:40.661266+0000","last_active":"2024-03-29T04:17:42.649946+0000","last_peered":"2024-03-29T04:17:42.649946+0000","last_clean":"2024-03-29T04:17:42.649946+0000","last_became_active":"2024-03-29T04:17:40.660780+0000","last_became_peered":"2024-03-29T04:17:40.660780+0000","last_unstale":"2024-03-29T04:17:42.649946+0000","last_undegraded":"2024-03-29T04:17:42.649946+0000","last_fullsized":"2024-03-29T04:17:42.649946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:02.859957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552978+0000","last_change":"2024-03-29T04:17:36.610408+0000","last_active":"2024-03-29T04:17:43.552978+0000","last_peered":"2024-03-29T04:17:43.552978+0000","last_clean":"2024-03-29T04:17:43.552978+0000","last_became_active":"2024-03-29T04:17:36.610111+0000","last_became_peered":"2024-03-29T04:17:36.610111+0000","last_unstale":"2024-03-29T04:17:43.552978+0000","last_undegraded":"2024-03-29T04:17:43.552978+0000","last_fullsized":"2024-03-29T04:17:43.552978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:46:38.204638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.269316+0000","last_change":"2024-03-29T04:17:35.601002+0000","last_active":"2024-03-29T04:17:44.269316+0000","last_peered":"2024-03-29T04:17:44.269316+0000","last_clean":"2024-03-29T04:17:44.269316+0000","last_became_active":"2024-03-29T04:17:35.600276+0000","last_became_peered":"2024-03-29T04:17:35.600276+0000","last_unstale":"2024-03-29T04:17:44.269316+0000","last_undegraded":"2024-03-29T04:17:44.269316+0000","last_fullsized":"2024-03-29T04:17:44.269316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:55:50.253366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649873+0000","last_change":"2024-03-29T04:17:41.164850+0000","last_active":"2024-03-29T04:17:42.649873+0000","last_peered":"2024-03-29T04:17:42.649873+0000","last_clean":"2024-03-29T04:17:42.649873+0000","last_became_active":"2024-03-29T04:17:41.163894+0000","last_became_peered":"2024-03-29T04:17:41.163894+0000","last_unstale":"2024-03-29T04:17:42.649873+0000","last_undegraded":"2024-03-29T04:17:42.649873+0000","last_fullsized":"2024-03-29T04:17:42.649873+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:03:39.448893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653276+0000","last_change":"2024-03-29T04:17:41.171200+0000","last_active":"2024-03-29T04:17:42.653276+0000","last_peered":"2024-03-29T04:17:42.653276+0000","last_clean":"2024-03-29T04:17:42.653276+0000","last_became_active":"2024-03-29T04:17:41.169774+0000","last_became_peered":"2024-03-29T04:17:41.169774+0000","last_unstale":"2024-03-29T04:17:42.653276+0000","last_undegraded":"2024-03-29T04:17:42.653276+0000","last_fullsized":"2024-03-29T04:17:42.653276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:43:44.093178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470127+0000","last_change":"2024-03-29T04:17:36.598385+0000","last_active":"2024-03-29T04:17:44.470127+0000","last_peered":"2024-03-29T04:17:44.470127+0000","last_clean":"2024-03-29T04:17:44.470127+0000","last_became_active":"2024-03-29T04:17:36.597989+0000","last_became_peered":"2024-03-29T04:17:36.597989+0000","last_unstale":"2024-03-29T04:17:44.470127+0000","last_undegraded":"2024-03-29T04:17:44.470127+0000","last_fullsized":"2024-03-29T04:17:44.470127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:11.037419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553421+0000","last_change":"2024-03-29T04:17:35.591199+0000","last_active":"2024-03-29T04:17:43.553421+0000","last_peered":"2024-03-29T04:17:43.553421+0000","last_clean":"2024-03-29T04:17:43.553421+0000","last_became_active":"2024-03-29T04:17:35.590444+0000","last_became_peered":"2024-03-29T04:17:35.590444+0000","last_unstale":"2024-03-29T04:17:43.553421+0000","last_undegraded":"2024-03-29T04:17:43.553421+0000","last_fullsized":"2024-03-29T04:17:43.553421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:23.283694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270972+0000","last_change":"2024-03-29T04:17:40.661020+0000","last_active":"2024-03-29T04:17:44.270972+0000","last_peered":"2024-03-29T04:17:44.270972+0000","last_clean":"2024-03-29T04:17:44.270972+0000","last_became_active":"2024-03-29T04:17:40.660552+0000","last_became_peered":"2024-03-29T04:17:40.660552+0000","last_unstale":"2024-03-29T04:17:44.270972+0000","last_undegraded":"2024-03-29T04:17:44.270972+0000","last_fullsized":"2024-03-29T04:17:44.270972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:25:02.717791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649896+0000","last_change":"2024-03-29T04:17:36.619765+0000","last_active":"2024-03-29T04:17:42.649896+0000","last_peered":"2024-03-29T04:17:42.649896+0000","last_clean":"2024-03-29T04:17:42.649896+0000","last_became_active":"2024-03-29T04:17:36.619507+0000","last_became_peered":"2024-03-29T04:17:36.619507+0000","last_unstale":"2024-03-29T04:17:42.649896+0000","last_undegraded":"2024-03-29T04:17:42.649896+0000","last_fullsized":"2024-03-29T04:17:42.649896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:39.635039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649952+0000","last_change":"2024-03-29T04:17:35.575983+0000","last_active":"2024-03-29T04:17:42.649952+0000","last_peered":"2024-03-29T04:17:42.649952+0000","last_clean":"2024-03-29T04:17:42.649952+0000","last_became_active":"2024-03-29T04:17:35.575381+0000","last_became_peered":"2024-03-29T04:17:35.575381+0000","last_unstale":"2024-03-29T04:17:42.649952+0000","last_undegraded":"2024-03-29T04:17:42.649952+0000","last_fullsized":"2024-03-29T04:17:42.649952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:07.631047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.268935+0000","last_change":"2024-03-29T04:17:35.577005+0000","last_active":"2024-03-29T04:17:44.268935+0000","last_peered":"2024-03-29T04:17:44.268935+0000","last_clean":"2024-03-29T04:17:44.268935+0000","last_became_active":"2024-03-29T04:17:35.574501+0000","last_became_peered":"2024-03-29T04:17:35.574501+0000","last_unstale":"2024-03-29T04:17:44.268935+0000","last_undegraded":"2024-03-29T04:17:44.268935+0000","last_fullsized":"2024-03-29T04:17:44.268935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:23.283694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653382+0000","last_change":"2024-03-29T04:17:36.615300+0000","last_active":"2024-03-29T04:17:42.653382+0000","last_peered":"2024-03-29T04:17:42.653382+0000","last_clean":"2024-03-29T04:17:42.653382+0000","last_became_active":"2024-03-29T04:17:36.614767+0000","last_became_peered":"2024-03-29T04:17:36.614767+0000","last_unstale":"2024-03-29T04:17:42.653382+0000","last_undegraded":"2024-03-29T04:17:42.653382+0000","last_fullsized":"2024-03-29T04:17:42.653382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:30.598140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578719+0000","last_change":"2024-03-29T04:17:40.666173+0000","last_active":"2024-03-29T04:17:43.578719+0000","last_peered":"2024-03-29T04:17:43.578719+0000","last_clean":"2024-03-29T04:17:43.578719+0000","last_became_active":"2024-03-29T04:17:40.665205+0000","last_became_peered":"2024-03-29T04:17:40.665205+0000","last_unstale":"2024-03-29T04:17:43.578719+0000","last_undegraded":"2024-03-29T04:17:43.578719+0000","last_fullsized":"2024-03-29T04:17:43.578719+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:10:04.880736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650203+0000","last_change":"2024-03-29T04:17:35.597727+0000","last_active":"2024-03-29T04:17:42.650203+0000","last_peered":"2024-03-29T04:17:42.650203+0000","last_clean":"2024-03-29T04:17:42.650203+0000","last_became_active":"2024-03-29T04:17:35.597476+0000","last_became_peered":"2024-03-29T04:17:35.597476+0000","last_unstale":"2024-03-29T04:17:42.650203+0000","last_undegraded":"2024-03-29T04:17:42.650203+0000","last_fullsized":"2024-03-29T04:17:42.650203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:04:44.150824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656845+0000","last_change":"2024-03-29T04:17:36.593245+0000","last_active":"2024-03-29T04:17:42.656845+0000","last_peered":"2024-03-29T04:17:42.656845+0000","last_clean":"2024-03-29T04:17:42.656845+0000","last_became_active":"2024-03-29T04:17:36.592508+0000","last_became_peered":"2024-03-29T04:17:36.592508+0000","last_unstale":"2024-03-29T04:17:42.656845+0000","last_undegraded":"2024-03-29T04:17:42.656845+0000","last_fullsized":"2024-03-29T04:17:42.656845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:52:19.919554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650486+0000","last_change":"2024-03-29T04:17:40.664734+0000","last_active":"2024-03-29T04:17:42.650486+0000","last_peered":"2024-03-29T04:17:42.650486+0000","last_clean":"2024-03-29T04:17:42.650486+0000","last_became_active":"2024-03-29T04:17:40.664326+0000","last_became_peered":"2024-03-29T04:17:40.664326+0000","last_unstale":"2024-03-29T04:17:42.650486+0000","last_undegraded":"2024-03-29T04:17:42.650486+0000","last_fullsized":"2024-03-29T04:17:42.650486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:03.774120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.269324+0000","last_change":"2024-03-29T04:17:36.618652+0000","last_active":"2024-03-29T04:17:44.269324+0000","last_peered":"2024-03-29T04:17:44.269324+0000","last_clean":"2024-03-29T04:17:44.269324+0000","last_became_active":"2024-03-29T04:17:36.618384+0000","last_became_peered":"2024-03-29T04:17:36.618384+0000","last_unstale":"2024-03-29T04:17:44.269324+0000","last_undegraded":"2024-03-29T04:17:44.269324+0000","last_fullsized":"2024-03-29T04:17:44.269324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:30.598140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578474+0000","last_change":"2024-03-29T04:17:36.606149+0000","last_active":"2024-03-29T04:17:43.578474+0000","last_peered":"2024-03-29T04:17:43.578474+0000","last_clean":"2024-03-29T04:17:43.578474+0000","last_became_active":"2024-03-29T04:17:36.605253+0000","last_became_peered":"2024-03-29T04:17:36.605253+0000","last_unstale":"2024-03-29T04:17:43.578474+0000","last_undegraded":"2024-03-29T04:17:43.578474+0000","last_fullsized":"2024-03-29T04:17:43.578474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:24:47.130861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553107+0000","last_change":"2024-03-29T04:17:36.602472+0000","last_active":"2024-03-29T04:17:43.553107+0000","last_peered":"2024-03-29T04:17:43.553107+0000","last_clean":"2024-03-29T04:17:43.553107+0000","last_became_active":"2024-03-29T04:17:36.601990+0000","last_became_peered":"2024-03-29T04:17:36.601990+0000","last_unstale":"2024-03-29T04:17:43.553107+0000","last_undegraded":"2024-03-29T04:17:43.553107+0000","last_fullsized":"2024-03-29T04:17:43.553107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:38:20.641332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653238+0000","last_change":"2024-03-29T04:17:36.607285+0000","last_active":"2024-03-29T04:17:42.653238+0000","last_peered":"2024-03-29T04:17:42.653238+0000","last_clean":"2024-03-29T04:17:42.653238+0000","last_became_active":"2024-03-29T04:17:36.606419+0000","last_became_peered":"2024-03-29T04:17:36.606419+0000","last_unstale":"2024-03-29T04:17:42.653238+0000","last_undegraded":"2024-03-29T04:17:42.653238+0000","last_fullsized":"2024-03-29T04:17:42.653238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:41.872399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650775+0000","last_change":"2024-03-29T04:17:36.627762+0000","last_active":"2024-03-29T04:17:42.650775+0000","last_peered":"2024-03-29T04:17:42.650775+0000","last_clean":"2024-03-29T04:17:42.650775+0000","last_became_active":"2024-03-29T04:17:36.627476+0000","last_became_peered":"2024-03-29T04:17:36.627476+0000","last_unstale":"2024-03-29T04:17:42.650775+0000","last_undegraded":"2024-03-29T04:17:42.650775+0000","last_fullsized":"2024-03-29T04:17:42.650775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:38:20.641332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470447+0000","last_change":"2024-03-29T04:17:36.597795+0000","last_active":"2024-03-29T04:17:44.470447+0000","last_peered":"2024-03-29T04:17:44.470447+0000","last_clean":"2024-03-29T04:17:44.470447+0000","last_became_active":"2024-03-29T04:17:36.597490+0000","last_became_peered":"2024-03-29T04:17:36.597490+0000","last_unstale":"2024-03-29T04:17:44.470447+0000","last_undegraded":"2024-03-29T04:17:44.470447+0000","last_fullsized":"2024-03-29T04:17:44.470447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:43:27.568676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270661+0000","last_change":"2024-03-29T04:17:36.609886+0000","last_active":"2024-03-29T04:17:44.270661+0000","last_peered":"2024-03-29T04:17:44.270661+0000","last_clean":"2024-03-29T04:17:44.270661+0000","last_became_active":"2024-03-29T04:17:36.609354+0000","last_became_peered":"2024-03-29T04:17:36.609354+0000","last_unstale":"2024-03-29T04:17:44.270661+0000","last_undegraded":"2024-03-29T04:17:44.270661+0000","last_fullsized":"2024-03-29T04:17:44.270661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:24:47.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650587+0000","last_change":"2024-03-29T04:17:36.627964+0000","last_active":"2024-03-29T04:17:42.650587+0000","last_peered":"2024-03-29T04:17:42.650587+0000","last_clean":"2024-03-29T04:17:42.650587+0000","last_became_active":"2024-03-29T04:17:36.627606+0000","last_became_peered":"2024-03-29T04:17:36.627606+0000","last_unstale":"2024-03-29T04:17:42.650587+0000","last_undegraded":"2024-03-29T04:17:42.650587+0000","last_fullsized":"2024-03-29T04:17:42.650587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:44.877184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552870+0000","last_change":"2024-03-29T04:17:36.617738+0000","last_active":"2024-03-29T04:17:43.552870+0000","last_peered":"2024-03-29T04:17:43.552870+0000","last_clean":"2024-03-29T04:17:43.552870+0000","last_became_active":"2024-03-29T04:17:36.617532+0000","last_became_peered":"2024-03-29T04:17:36.617532+0000","last_unstale":"2024-03-29T04:17:43.552870+0000","last_undegraded":"2024-03-29T04:17:43.552870+0000","last_fullsized":"2024-03-29T04:17:43.552870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:41.872399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649836+0000","last_change":"2024-03-29T04:17:36.627025+0000","last_active":"2024-03-29T04:17:42.649836+0000","last_peered":"2024-03-29T04:17:42.649836+0000","last_clean":"2024-03-29T04:17:42.649836+0000","last_became_active":"2024-03-29T04:17:36.626671+0000","last_became_peered":"2024-03-29T04:17:36.626671+0000","last_unstale":"2024-03-29T04:17:42.649836+0000","last_undegraded":"2024-03-29T04:17:42.649836+0000","last_fullsized":"2024-03-29T04:17:42.649836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:31:36.783083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552909+0000","last_change":"2024-03-29T04:17:36.609953+0000","last_active":"2024-03-29T04:17:43.552909+0000","last_peered":"2024-03-29T04:17:43.552909+0000","last_clean":"2024-03-29T04:17:43.552909+0000","last_became_active":"2024-03-29T04:17:36.609455+0000","last_became_peered":"2024-03-29T04:17:36.609455+0000","last_unstale":"2024-03-29T04:17:43.552909+0000","last_undegraded":"2024-03-29T04:17:43.552909+0000","last_fullsized":"2024-03-29T04:17:43.552909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:31:36.783083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653087+0000","last_change":"2024-03-29T04:17:36.618490+0000","last_active":"2024-03-29T04:17:42.653087+0000","last_peered":"2024-03-29T04:17:42.653087+0000","last_clean":"2024-03-29T04:17:42.653087+0000","last_became_active":"2024-03-29T04:17:36.618012+0000","last_became_peered":"2024-03-29T04:17:36.618012+0000","last_unstale":"2024-03-29T04:17:42.653087+0000","last_undegraded":"2024-03-29T04:17:42.653087+0000","last_fullsized":"2024-03-29T04:17:42.653087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:43:27.568676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270104+0000","last_change":"2024-03-29T04:17:36.621359+0000","last_active":"2024-03-29T04:17:44.270104+0000","last_peered":"2024-03-29T04:17:44.270104+0000","last_clean":"2024-03-29T04:17:44.270104+0000","last_became_active":"2024-03-29T04:17:36.620689+0000","last_became_peered":"2024-03-29T04:17:36.620689+0000","last_unstale":"2024-03-29T04:17:44.270104+0000","last_undegraded":"2024-03-29T04:17:44.270104+0000","last_fullsized":"2024-03-29T04:17:44.270104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:43:27.568676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634352+0000","last_change":"2024-03-29T04:17:36.610675+0000","last_active":"2024-03-29T04:17:44.634352+0000","last_peered":"2024-03-29T04:17:44.634352+0000","last_clean":"2024-03-29T04:17:44.634352+0000","last_became_active":"2024-03-29T04:17:36.609797+0000","last_became_peered":"2024-03-29T04:17:36.609797+0000","last_unstale":"2024-03-29T04:17:44.634352+0000","last_undegraded":"2024-03-29T04:17:44.634352+0000","last_fullsized":"2024-03-29T04:17:44.634352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:04:11.306658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.579363+0000","last_change":"2024-03-29T04:17:36.605354+0000","last_active":"2024-03-29T04:17:43.579363+0000","last_peered":"2024-03-29T04:17:43.579363+0000","last_clean":"2024-03-29T04:17:43.579363+0000","last_became_active":"2024-03-29T04:17:36.605028+0000","last_became_peered":"2024-03-29T04:17:36.605028+0000","last_unstale":"2024-03-29T04:17:43.579363+0000","last_undegraded":"2024-03-29T04:17:43.579363+0000","last_fullsized":"2024-03-29T04:17:43.579363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:12:37.599253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553747+0000","last_change":"2024-03-29T04:17:36.611310+0000","last_active":"2024-03-29T04:17:43.553747+0000","last_peered":"2024-03-29T04:17:43.553747+0000","last_clean":"2024-03-29T04:17:43.553747+0000","last_became_active":"2024-03-29T04:17:36.611020+0000","last_became_peered":"2024-03-29T04:17:36.611020+0000","last_unstale":"2024-03-29T04:17:43.553747+0000","last_undegraded":"2024-03-29T04:17:43.553747+0000","last_fullsized":"2024-03-29T04:17:43.553747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:42.915247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552533+0000","last_change":"2024-03-29T04:17:36.618624+0000","last_active":"2024-03-29T04:17:43.552533+0000","last_peered":"2024-03-29T04:17:43.552533+0000","last_clean":"2024-03-29T04:17:43.552533+0000","last_became_active":"2024-03-29T04:17:36.618282+0000","last_became_peered":"2024-03-29T04:17:36.618282+0000","last_unstale":"2024-03-29T04:17:43.552533+0000","last_undegraded":"2024-03-29T04:17:43.552533+0000","last_fullsized":"2024-03-29T04:17:43.552533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:24:47.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470271+0000","last_change":"2024-03-29T04:17:36.587187+0000","last_active":"2024-03-29T04:17:44.470271+0000","last_peered":"2024-03-29T04:17:44.470271+0000","last_clean":"2024-03-29T04:17:44.470271+0000","last_became_active":"2024-03-29T04:17:36.586740+0000","last_became_peered":"2024-03-29T04:17:36.586740+0000","last_unstale":"2024-03-29T04:17:44.470271+0000","last_undegraded":"2024-03-29T04:17:44.470271+0000","last_fullsized":"2024-03-29T04:17:44.470271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:24:47.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653115+0000","last_change":"2024-03-29T04:17:36.615925+0000","last_active":"2024-03-29T04:17:42.653115+0000","last_peered":"2024-03-29T04:17:42.653115+0000","last_clean":"2024-03-29T04:17:42.653115+0000","last_became_active":"2024-03-29T04:17:36.615143+0000","last_became_peered":"2024-03-29T04:17:36.615143+0000","last_unstale":"2024-03-29T04:17:42.653115+0000","last_undegraded":"2024-03-29T04:17:42.653115+0000","last_fullsized":"2024-03-29T04:17:42.653115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:50:12.920911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650032+0000","last_change":"2024-03-29T04:17:36.605090+0000","last_active":"2024-03-29T04:17:42.650032+0000","last_peered":"2024-03-29T04:17:42.650032+0000","last_clean":"2024-03-29T04:17:42.650032+0000","last_became_active":"2024-03-29T04:17:36.604669+0000","last_became_peered":"2024-03-29T04:17:36.604669+0000","last_unstale":"2024-03-29T04:17:42.650032+0000","last_undegraded":"2024-03-29T04:17:42.650032+0000","last_fullsized":"2024-03-29T04:17:42.650032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:30.598140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.657981+0000","last_change":"2024-03-29T04:17:36.614472+0000","last_active":"2024-03-29T04:17:42.657981+0000","last_peered":"2024-03-29T04:17:42.657981+0000","last_clean":"2024-03-29T04:17:42.657981+0000","last_became_active":"2024-03-29T04:17:36.613881+0000","last_became_peered":"2024-03-29T04:17:36.613881+0000","last_unstale":"2024-03-29T04:17:42.657981+0000","last_undegraded":"2024-03-29T04:17:42.657981+0000","last_fullsized":"2024-03-29T04:17:42.657981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:24:47.130861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270375+0000","last_change":"2024-03-29T04:17:36.600125+0000","last_active":"2024-03-29T04:17:44.270375+0000","last_peered":"2024-03-29T04:17:44.270375+0000","last_clean":"2024-03-29T04:17:44.270375+0000","last_became_active":"2024-03-29T04:17:36.599450+0000","last_became_peered":"2024-03-29T04:17:36.599450+0000","last_unstale":"2024-03-29T04:17:44.270375+0000","last_undegraded":"2024-03-29T04:17:44.270375+0000","last_fullsized":"2024-03-29T04:17:44.270375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:11.037419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270626+0000","last_change":"2024-03-29T04:17:36.600177+0000","last_active":"2024-03-29T04:17:44.270626+0000","last_peered":"2024-03-29T04:17:44.270626+0000","last_clean":"2024-03-29T04:17:44.270626+0000","last_became_active":"2024-03-29T04:17:36.599696+0000","last_became_peered":"2024-03-29T04:17:36.599696+0000","last_unstale":"2024-03-29T04:17:44.270626+0000","last_undegraded":"2024-03-29T04:17:44.270626+0000","last_fullsized":"2024-03-29T04:17:44.270626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:39.635039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.635042+0000","last_change":"2024-03-29T04:17:36.610038+0000","last_active":"2024-03-29T04:17:44.635042+0000","last_peered":"2024-03-29T04:17:44.635042+0000","last_clean":"2024-03-29T04:17:44.635042+0000","last_became_active":"2024-03-29T04:17:36.609175+0000","last_became_peered":"2024-03-29T04:17:36.609175+0000","last_unstale":"2024-03-29T04:17:44.635042+0000","last_undegraded":"2024-03-29T04:17:44.635042+0000","last_fullsized":"2024-03-29T04:17:44.635042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:42:21.035572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649728+0000","last_change":"2024-03-29T04:17:36.616578+0000","last_active":"2024-03-29T04:17:42.649728+0000","last_peered":"2024-03-29T04:17:42.649728+0000","last_clean":"2024-03-29T04:17:42.649728+0000","last_became_active":"2024-03-29T04:17:36.612656+0000","last_became_peered":"2024-03-29T04:17:36.612656+0000","last_unstale":"2024-03-29T04:17:42.649728+0000","last_undegraded":"2024-03-29T04:17:42.649728+0000","last_fullsized":"2024-03-29T04:17:42.649728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:11.037419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634140+0000","last_change":"2024-03-29T04:17:36.618859+0000","last_active":"2024-03-29T04:17:44.634140+0000","last_peered":"2024-03-29T04:17:44.634140+0000","last_clean":"2024-03-29T04:17:44.634140+0000","last_became_active":"2024-03-29T04:17:36.618475+0000","last_became_peered":"2024-03-29T04:17:36.618475+0000","last_unstale":"2024-03-29T04:17:44.634140+0000","last_undegraded":"2024-03-29T04:17:44.634140+0000","last_fullsized":"2024-03-29T04:17:44.634140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:30.729030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552858+0000","last_change":"2024-03-29T04:17:36.614982+0000","last_active":"2024-03-29T04:17:43.552858+0000","last_peered":"2024-03-29T04:17:43.552858+0000","last_clean":"2024-03-29T04:17:43.552858+0000","last_became_active":"2024-03-29T04:17:36.614701+0000","last_became_peered":"2024-03-29T04:17:36.614701+0000","last_unstale":"2024-03-29T04:17:43.552858+0000","last_undegraded":"2024-03-29T04:17:43.552858+0000","last_fullsized":"2024-03-29T04:17:43.552858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:30.598140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634536+0000","last_change":"2024-03-29T04:17:36.608699+0000","last_active":"2024-03-29T04:17:44.634536+0000","last_peered":"2024-03-29T04:17:44.634536+0000","last_clean":"2024-03-29T04:17:44.634536+0000","last_became_active":"2024-03-29T04:17:36.608428+0000","last_became_peered":"2024-03-29T04:17:36.608428+0000","last_unstale":"2024-03-29T04:17:44.634536+0000","last_undegraded":"2024-03-29T04:17:44.634536+0000","last_fullsized":"2024-03-29T04:17:44.634536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:44.877184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553129+0000","last_change":"2024-03-29T04:17:36.596870+0000","last_active":"2024-03-29T04:17:43.553129+0000","last_peered":"2024-03-29T04:17:43.553129+0000","last_clean":"2024-03-29T04:17:43.553129+0000","last_became_active":"2024-03-29T04:17:36.596557+0000","last_became_peered":"2024-03-29T04:17:36.596557+0000","last_unstale":"2024-03-29T04:17:43.553129+0000","last_undegraded":"2024-03-29T04:17:43.553129+0000","last_fullsized":"2024-03-29T04:17:43.553129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:44.877184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653260+0000","last_change":"2024-03-29T04:17:36.593793+0000","last_active":"2024-03-29T04:17:42.653260+0000","last_peered":"2024-03-29T04:17:42.653260+0000","last_clean":"2024-03-29T04:17:42.653260+0000","last_became_active":"2024-03-29T04:17:36.592632+0000","last_became_peered":"2024-03-29T04:17:36.592632+0000","last_unstale":"2024-03-29T04:17:42.653260+0000","last_undegraded":"2024-03-29T04:17:42.653260+0000","last_fullsized":"2024-03-29T04:17:42.653260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:41:31.910133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470745+0000","last_change":"2024-03-29T04:17:36.593148+0000","last_active":"2024-03-29T04:17:44.470745+0000","last_peered":"2024-03-29T04:17:44.470745+0000","last_clean":"2024-03-29T04:17:44.470745+0000","last_became_active":"2024-03-29T04:17:36.592754+0000","last_became_peered":"2024-03-29T04:17:36.592754+0000","last_unstale":"2024-03-29T04:17:44.470745+0000","last_undegraded":"2024-03-29T04:17:44.470745+0000","last_fullsized":"2024-03-29T04:17:44.470745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:30.598140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.635518+0000","last_change":"2024-03-29T04:17:36.615787+0000","last_active":"2024-03-29T04:17:44.635518+0000","last_peered":"2024-03-29T04:17:44.635518+0000","last_clean":"2024-03-29T04:17:44.635518+0000","last_became_active":"2024-03-29T04:17:36.615476+0000","last_became_peered":"2024-03-29T04:17:36.615476+0000","last_unstale":"2024-03-29T04:17:44.635518+0000","last_undegraded":"2024-03-29T04:17:44.635518+0000","last_fullsized":"2024-03-29T04:17:44.635518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:42.915247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":876544,"data_stored":703014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":112154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":112190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":278528,"data_stored":112064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":286720,"data_stored":112224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":884736,"data_stored":703070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:17:55.191 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-29T04:17:55.191 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-29T04:17:55.191 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-29T04:17:55.191 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-29T04:17:55.192 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-29T04:17:55.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.372+0000 7f86d36c0640 1 Processor -- start 2024-03-29T04:17:55.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.372+0000 7f86d36c0640 1 -- start start 2024-03-29T04:17:55.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.376+0000 7f86d36c0640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f86cc0b2ef0 0x7f86cc144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:55.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.376+0000 7f86d36c0640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f86cc15f7a0 0x7f86cc06c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:55.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.376+0000 7f86d36c0640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86cc0b4ba0 con 0x7f86cc144270 2024-03-29T04:17:55.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.376+0000 7f86d36c0640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f86cc144fa0 con 0x7f86cc0b2ef0 2024-03-29T04:17:55.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.376+0000 7f86d36c0640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f86cc0b33f0 con 0x7f86cc15f7a0 2024-03-29T04:17:55.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.376+0000 7f86d0c5b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f86cc0b2ef0 0x7f86cc144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:55.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.376+0000 7f86d0c5b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f86cc0b2ef0 0x7f86cc144650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51240/0 (socket says 172.21.15.22:51240) 2024-03-29T04:17:55.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.376+0000 7f86d0c5b640 1 -- 172.21.15.22:0/63866972 learned_addr learned my addr 172.21.15.22:0/63866972 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:55.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.376+0000 7f86d1c5d640 1 --2- 172.21.15.22:0/63866972 >> v2:172.21.15.151:3300/0 conn(0x7f86cc15f7a0 0x7f86cc06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:55.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.376+0000 7f86d0c5b640 1 -- 172.21.15.22:0/63866972 >> v2:172.21.15.151:3300/0 conn(0x7f86cc15f7a0 msgr2=0x7f86cc06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:55.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.376+0000 7f86d0c5b640 1 --2- 172.21.15.22:0/63866972 >> v2:172.21.15.151:3300/0 conn(0x7f86cc15f7a0 0x7f86cc06c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.376+0000 7f86d0c5b640 1 -- 172.21.15.22:0/63866972 >> v1:172.21.15.151:6789/0 conn(0x7f86cc144270 legacy=0x7f86cc0b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:55.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.376+0000 7f86d0c5b640 1 -- 172.21.15.22:0/63866972 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86cc0b36d0 con 0x7f86cc0b2ef0 2024-03-29T04:17:55.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.376+0000 7f86d0c5b640 1 --2- 172.21.15.22:0/63866972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86cc0b2ef0 0x7f86cc144650 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0b5430 tx=0x7f86bc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5424e6153468dfa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:55.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.376+0000 7f86c3fff640 1 -- 172.21.15.22:0/63866972 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0ab600 con 0x7f86cc0b2ef0 2024-03-29T04:17:55.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.376+0000 7f86c3fff640 1 -- 172.21.15.22:0/63866972 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86bc0b0070 con 0x7f86cc0b2ef0 2024-03-29T04:17:55.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86c3fff640 1 -- 172.21.15.22:0/63866972 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0ab9c0 con 0x7f86cc0b2ef0 2024-03-29T04:17:55.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d36c0640 1 -- 172.21.15.22:0/63866972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86cc0b2ef0 msgr2=0x7f86cc144650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:55.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d36c0640 1 --2- 172.21.15.22:0/63866972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86cc0b2ef0 0x7f86cc144650 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0b5430 tx=0x7f86bc01c4f0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d36c0640 1 -- 172.21.15.22:0/63866972 shutdown_connections 2024-03-29T04:17:55.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d36c0640 1 --2- 172.21.15.22:0/63866972 >> v2:172.21.15.151:3300/0 conn(0x7f86cc15f7a0 0x7f86cc06c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d36c0640 1 --2- 172.21.15.22:0/63866972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86cc0b2ef0 0x7f86cc144650 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d36c0640 1 -- 172.21.15.22:0/63866972 >> 172.21.15.22:0/63866972 conn(0x7f86cc0cfee0 msgr2=0x7f86cc0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:55.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d36c0640 1 -- 172.21.15.22:0/63866972 shutdown_connections 2024-03-29T04:17:55.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d36c0640 1 -- 172.21.15.22:0/63866972 wait complete. 2024-03-29T04:17:55.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d36c0640 1 Processor -- start 2024-03-29T04:17:55.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d36c0640 1 -- start start 2024-03-29T04:17:55.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d36c0640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86cc0b2ef0 0x7f86cc13d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:55.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d36c0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86cc144270 0x7f86cc1564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:55.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d36c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86cc15f7a0 0x7f86cc1f5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:55.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d36c0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f86cc0cdd10 con 0x7f86cc15f7a0 2024-03-29T04:17:55.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d36c0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f86cc15fbb0 con 0x7f86cc144270 2024-03-29T04:17:55.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d36c0640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f86cc0cde90 con 0x7f86cc0b2ef0 2024-03-29T04:17:55.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d1c5d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86cc15f7a0 0x7f86cc1f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:55.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d1c5d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86cc15f7a0 0x7f86cc1f5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51252/0 (socket says 172.21.15.22:51252) 2024-03-29T04:17:55.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d1c5d640 1 -- 172.21.15.22:0/429659616 learned_addr learned my addr 172.21.15.22:0/429659616 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:55.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d0c5b640 1 --2- 172.21.15.22:0/429659616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86cc144270 0x7f86cc1564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:55.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d145c640 1 --2- 172.21.15.22:0/429659616 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86cc0b2ef0 0x7f86cc13d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:55.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d1c5d640 1 -- 172.21.15.22:0/429659616 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86cc0b2ef0 msgr2=0x7f86cc13d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:55.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d1c5d640 1 --2- 172.21.15.22:0/429659616 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86cc0b2ef0 0x7f86cc13d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d1c5d640 1 -- 172.21.15.22:0/429659616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86cc144270 msgr2=0x7f86cc1564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:55.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d1c5d640 1 --2- 172.21.15.22:0/429659616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86cc144270 0x7f86cc1564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d1c5d640 1 -- 172.21.15.22:0/429659616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86cc138f30 con 0x7f86cc15f7a0 2024-03-29T04:17:55.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d145c640 1 --2- 172.21.15.22:0/429659616 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86cc0b2ef0 0x7f86cc13d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:55.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d1c5d640 1 --2- 172.21.15.22:0/429659616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86cc15f7a0 0x7f86cc1f5730 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f86c4007e70 tx=0x7f86c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:55.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86c1ffb640 1 -- 172.21.15.22:0/429659616 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4014420 con 0x7f86cc15f7a0 2024-03-29T04:17:55.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86c1ffb640 1 -- 172.21.15.22:0/429659616 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c4019070 con 0x7f86cc15f7a0 2024-03-29T04:17:55.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d36c0640 1 -- 172.21.15.22:0/429659616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86cc139210 con 0x7f86cc15f7a0 2024-03-29T04:17:55.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86c1ffb640 1 -- 172.21.15.22:0/429659616 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4014820 con 0x7f86cc15f7a0 2024-03-29T04:17:55.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.380+0000 7f86d36c0640 1 -- 172.21.15.22:0/429659616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86cc1397d0 con 0x7f86cc15f7a0 2024-03-29T04:17:55.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.384+0000 7f86d36c0640 1 -- 172.21.15.22:0/429659616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86cc068890 con 0x7f86cc15f7a0 2024-03-29T04:17:55.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.384+0000 7f86c1ffb640 1 -- 172.21.15.22:0/429659616 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f86c40149c0 con 0x7f86cc15f7a0 2024-03-29T04:17:55.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.384+0000 7f86c1ffb640 1 --2- 172.21.15.22:0/429659616 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f86b0067fc0 0x7f86b006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:55.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.384+0000 7f86c1ffb640 1 -- 172.21.15.22:0/429659616 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f86c40871c0 con 0x7f86cc15f7a0 2024-03-29T04:17:55.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.384+0000 7f86d145c640 1 --2- 172.21.15.22:0/429659616 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f86b0067fc0 0x7f86b006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:55.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.384+0000 7f86d145c640 1 --2- 172.21.15.22:0/429659616 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f86b0067fc0 0x7f86b006a480 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f86cc142300 tx=0x7f86c8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:55.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.388+0000 7f86c1ffb640 1 -- 172.21.15.22:0/429659616 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f86c4057590 con 0x7f86cc15f7a0 2024-03-29T04:17:55.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.592+0000 7f86d36c0640 1 -- 172.21.15.22:0/429659616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f86cc1427f0 con 0x7f86cc15f7a0 2024-03-29T04:17:55.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.600+0000 7f86c1ffb640 1 -- 172.21.15.22:0/429659616 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+167688 (secure 0 0 0) 0x7f86c4056e90 con 0x7f86cc15f7a0 2024-03-29T04:17:55.583 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:55.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.604+0000 7f86d36c0640 1 -- 172.21.15.22:0/429659616 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f86b0067fc0 msgr2=0x7f86b006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:55.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.604+0000 7f86d36c0640 1 --2- 172.21.15.22:0/429659616 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f86b0067fc0 0x7f86b006a480 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f86cc142300 tx=0x7f86c8009040 comp rx=0 tx=0).stop 2024-03-29T04:17:55.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.604+0000 7f86d36c0640 1 -- 172.21.15.22:0/429659616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86cc15f7a0 msgr2=0x7f86cc1f5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:55.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.604+0000 7f86d36c0640 1 --2- 172.21.15.22:0/429659616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86cc15f7a0 0x7f86cc1f5730 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f86c4007e70 tx=0x7f86c400e4b0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.604+0000 7f86d36c0640 1 -- 172.21.15.22:0/429659616 shutdown_connections 2024-03-29T04:17:55.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.604+0000 7f86d36c0640 1 --2- 172.21.15.22:0/429659616 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f86b0067fc0 0x7f86b006a480 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.604+0000 7f86d36c0640 1 --2- 172.21.15.22:0/429659616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86cc15f7a0 0x7f86cc1f5730 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.604+0000 7f86d36c0640 1 --2- 172.21.15.22:0/429659616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86cc144270 0x7f86cc1564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.604+0000 7f86d36c0640 1 --2- 172.21.15.22:0/429659616 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86cc0b2ef0 0x7f86cc13d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.604+0000 7f86d36c0640 1 -- 172.21.15.22:0/429659616 >> 172.21.15.22:0/429659616 conn(0x7f86cc0cfee0 msgr2=0x7f86cc072eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:55.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.604+0000 7f86d36c0640 1 -- 172.21.15.22:0/429659616 shutdown_connections 2024-03-29T04:17:55.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.604+0000 7f86d36c0640 1 -- 172.21.15.22:0/429659616 wait complete. 2024-03-29T04:17:55.602 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":5,"flags":0,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1191011275},{"type":"v1","addr":"172.21.15.151:6833","nonce":1191011275}]},"active_addr":"172.21.15.151:6833/1191011275","active_change":"2024-03-29T04:17:15.676014+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4098,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.151:0","nonce":3838227462}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.151:0","nonce":53051212}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.151:0","nonce":2135199696}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.151:0","nonce":3331673929}]}]} 2024-03-29T04:17:55.606 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-29T04:17:55.606 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-29T04:17:55.606 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:17:55.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.752+0000 7f1ff3863640 1 Processor -- start 2024-03-29T04:17:55.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.752+0000 7f1ff3863640 1 -- start start 2024-03-29T04:17:55.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.752+0000 7f1ff3863640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1fec0aff30 0x7f1fec0b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:55.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.752+0000 7f1ff3863640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1fec0b4a50 0x7f1fec06e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:55.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.752+0000 7f1ff3863640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fec0b37c0 con 0x7f1fec06e2d0 2024-03-29T04:17:55.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.752+0000 7f1ff3863640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f1fec0b4080 con 0x7f1fec0b4a50 2024-03-29T04:17:55.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.752+0000 7f1ff3863640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1fec0b3a30 con 0x7f1fec0aff30 2024-03-29T04:17:55.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.752+0000 7f1ff1e00640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1fec0b4a50 0x7f1fec06e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:55.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.752+0000 7f1ff1e00640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1fec0b4a50 0x7f1fec06e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51268/0 (socket says 172.21.15.22:51268) 2024-03-29T04:17:55.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.752+0000 7f1ff1e00640 1 -- 172.21.15.22:0/1531379350 learned_addr learned my addr 172.21.15.22:0/1531379350 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:55.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.752+0000 7f1ff0dfe640 1 --2- 172.21.15.22:0/1531379350 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1fec0aff30 0x7f1fec0b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:55.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.752+0000 7f1ff1e00640 1 -- 172.21.15.22:0/1531379350 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1fec0aff30 msgr2=0x7f1fec0b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:55.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.752+0000 7f1ff1e00640 1 --2- 172.21.15.22:0/1531379350 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1fec0aff30 0x7f1fec0b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.752+0000 7f1ff1e00640 1 -- 172.21.15.22:0/1531379350 >> v1:172.21.15.151:6789/0 conn(0x7f1fec06e2d0 legacy=0x7f1fec06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:55.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.752+0000 7f1ff1e00640 1 -- 172.21.15.22:0/1531379350 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fec0b3d10 con 0x7f1fec0b4a50 2024-03-29T04:17:55.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff1e00640 1 --2- 172.21.15.22:0/1531379350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fec0b4a50 0x7f1fec06e6b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400bb10 tx=0x7f1fe401ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbdbb5d83b942e90 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:55.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1fe3fff640 1 -- 172.21.15.22:0/1531379350 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe40b4070 con 0x7f1fec0b4a50 2024-03-29T04:17:55.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1fe3fff640 1 -- 172.21.15.22:0/1531379350 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe40a7c70 con 0x7f1fec0b4a50 2024-03-29T04:17:55.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1fe3fff640 1 -- 172.21.15.22:0/1531379350 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe40af660 con 0x7f1fec0b4a50 2024-03-29T04:17:55.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff3863640 1 -- 172.21.15.22:0/1531379350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fec0b4a50 msgr2=0x7f1fec06e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:55.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff3863640 1 --2- 172.21.15.22:0/1531379350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fec0b4a50 0x7f1fec06e6b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400bb10 tx=0x7f1fe401ec00 comp rx=0 tx=0).stop 2024-03-29T04:17:55.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff3863640 1 -- 172.21.15.22:0/1531379350 shutdown_connections 2024-03-29T04:17:55.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff3863640 1 --2- 172.21.15.22:0/1531379350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fec0b4a50 0x7f1fec06e6b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff3863640 1 --2- 172.21.15.22:0/1531379350 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1fec0aff30 0x7f1fec0b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff3863640 1 -- 172.21.15.22:0/1531379350 >> 172.21.15.22:0/1531379350 conn(0x7f1fec0b2b30 msgr2=0x7f1fec0d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:55.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff3863640 1 -- 172.21.15.22:0/1531379350 shutdown_connections 2024-03-29T04:17:55.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff3863640 1 -- 172.21.15.22:0/1531379350 wait complete. 2024-03-29T04:17:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff3863640 1 Processor -- start 2024-03-29T04:17:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff3863640 1 -- start start 2024-03-29T04:17:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff3863640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1fec06e2d0 0x7f1fec0cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff3863640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fec0aff30 0x7f1fec1f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff3863640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fec0b4a50 0x7f1fec153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff3863640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec158650 con 0x7f1fec0b4a50 2024-03-29T04:17:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff3863640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec06f000 con 0x7f1fec0aff30 2024-03-29T04:17:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff3863640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1fec1587d0 con 0x7f1fec06e2d0 2024-03-29T04:17:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fec0b4a50 0x7f1fec153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fec0b4a50 0x7f1fec153120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51272/0 (socket says 172.21.15.22:51272) 2024-03-29T04:17:55.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff1e00640 1 -- 172.21.15.22:0/3968360836 learned_addr learned my addr 172.21.15.22:0/3968360836 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:55.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff0dfe640 1 --2- 172.21.15.22:0/3968360836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fec0aff30 0x7f1fec1f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:55.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff15ff640 1 --2- 172.21.15.22:0/3968360836 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1fec06e2d0 0x7f1fec0cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:55.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff1e00640 1 -- 172.21.15.22:0/3968360836 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1fec06e2d0 msgr2=0x7f1fec0cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:55.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff1e00640 1 --2- 172.21.15.22:0/3968360836 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1fec06e2d0 0x7f1fec0cd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff1e00640 1 -- 172.21.15.22:0/3968360836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fec0aff30 msgr2=0x7f1fec1f7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:55.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff1e00640 1 --2- 172.21.15.22:0/3968360836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fec0aff30 0x7f1fec1f7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff1e00640 1 -- 172.21.15.22:0/3968360836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fec0ce100 con 0x7f1fec0b4a50 2024-03-29T04:17:55.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff0dfe640 1 --2- 172.21.15.22:0/3968360836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fec0aff30 0x7f1fec1f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:55.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff1e00640 1 --2- 172.21.15.22:0/3968360836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fec0b4a50 0x7f1fec153120 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400bb10 tx=0x7f1fe401b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:55.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1fe1ffb640 1 -- 172.21.15.22:0/3968360836 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe40b4070 con 0x7f1fec0b4a50 2024-03-29T04:17:55.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1fe1ffb640 1 -- 172.21.15.22:0/3968360836 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe40b6070 con 0x7f1fec0b4a50 2024-03-29T04:17:55.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1ff3863640 1 -- 172.21.15.22:0/3968360836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fec0ce3e0 con 0x7f1fec0b4a50 2024-03-29T04:17:55.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.756+0000 7f1fe1ffb640 1 -- 172.21.15.22:0/3968360836 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe40bb6d0 con 0x7f1fec0b4a50 2024-03-29T04:17:55.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.760+0000 7f1ff3863640 1 -- 172.21.15.22:0/3968360836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fec0ce9a0 con 0x7f1fec0b4a50 2024-03-29T04:17:55.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.760+0000 7f1ff3863640 1 -- 172.21.15.22:0/3968360836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fec068890 con 0x7f1fec0b4a50 2024-03-29T04:17:55.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.760+0000 7f1fe1ffb640 1 -- 172.21.15.22:0/3968360836 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1fe40a7b30 con 0x7f1fec0b4a50 2024-03-29T04:17:55.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.760+0000 7f1fe1ffb640 1 --2- 172.21.15.22:0/3968360836 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1fd0067fc0 0x7f1fd006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:55.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.760+0000 7f1fe1ffb640 1 -- 172.21.15.22:0/3968360836 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1fe4120ff0 con 0x7f1fec0b4a50 2024-03-29T04:17:55.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.760+0000 7f1ff15ff640 1 --2- 172.21.15.22:0/3968360836 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1fd0067fc0 0x7f1fd006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:55.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.760+0000 7f1ff15ff640 1 --2- 172.21.15.22:0/3968360836 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1fd0067fc0 0x7f1fd006a480 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1fec0aecd0 tx=0x7f1fe8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:55.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.764+0000 7f1fe1ffb640 1 -- 172.21.15.22:0/3968360836 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1fe40a7e20 con 0x7f1fec0b4a50 2024-03-29T04:17:55.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.928+0000 7f1ff3863640 1 -- 172.21.15.22:0/3968360836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fec06dbd0 con 0x7f1fec0b4a50 2024-03-29T04:17:55.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.932+0000 7f1fe1ffb640 1 -- 172.21.15.22:0/3968360836 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18514 (secure 0 0 0) 0x7f1fe4018820 con 0x7f1fec0b4a50 2024-03-29T04:17:55.913 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:55.913 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:17:42.645632+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:17:55.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.932+0000 7f1ff3863640 1 -- 172.21.15.22:0/3968360836 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1fd0067fc0 msgr2=0x7f1fd006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:55.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.932+0000 7f1ff3863640 1 --2- 172.21.15.22:0/3968360836 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1fd0067fc0 0x7f1fd006a480 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1fec0aecd0 tx=0x7f1fe8009040 comp rx=0 tx=0).stop 2024-03-29T04:17:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.932+0000 7f1ff3863640 1 -- 172.21.15.22:0/3968360836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fec0b4a50 msgr2=0x7f1fec153120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.932+0000 7f1ff3863640 1 --2- 172.21.15.22:0/3968360836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fec0b4a50 0x7f1fec153120 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400bb10 tx=0x7f1fe401b9a0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.936+0000 7f1ff3863640 1 -- 172.21.15.22:0/3968360836 shutdown_connections 2024-03-29T04:17:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.936+0000 7f1ff3863640 1 --2- 172.21.15.22:0/3968360836 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1fd0067fc0 0x7f1fd006a480 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.936+0000 7f1ff3863640 1 --2- 172.21.15.22:0/3968360836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fec0b4a50 0x7f1fec153120 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.936+0000 7f1ff3863640 1 --2- 172.21.15.22:0/3968360836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fec0aff30 0x7f1fec1f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.936+0000 7f1ff3863640 1 --2- 172.21.15.22:0/3968360836 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1fec06e2d0 0x7f1fec0cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.936+0000 7f1ff3863640 1 -- 172.21.15.22:0/3968360836 >> 172.21.15.22:0/3968360836 conn(0x7f1fec0b2b30 msgr2=0x7f1fec06bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.936+0000 7f1ff3863640 1 -- 172.21.15.22:0/3968360836 shutdown_connections 2024-03-29T04:17:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:55.936+0000 7f1ff3863640 1 -- 172.21.15.22:0/3968360836 wait complete. 2024-03-29T04:17:55.934 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-29T04:17:55.934 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:17:56.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f40555ee640 1 Processor -- start 2024-03-29T04:17:56.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f40555ee640 1 -- start start 2024-03-29T04:17:56.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f40555ee640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f40500b3f50 0x7f4050169310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f40555ee640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40500b4500 con 0x7f40500cef80 2024-03-29T04:17:56.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f40555ee640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40500b47e0 con 0x7f40500b55a0 2024-03-29T04:17:56.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f40555ee640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f40500b4ac0 con 0x7f40500b3f50 2024-03-29T04:17:56.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404effd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f40500cef80 0x7f40500cf360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44140/0 (socket says 172.21.15.22:44140) 2024-03-29T04:17:56.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404effd640 1 -- 172.21.15.22:0/797687562 learned_addr learned my addr 172.21.15.22:0/797687562 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:56.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404dffb640 1 -- 172.21.15.22:0/797687562 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1413408548 0 0) 0x7f40500b4500 con 0x7f40500cef80 2024-03-29T04:17:56.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404dffb640 1 -- 172.21.15.22:0/797687562 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f403c003530 con 0x7f40500cef80 2024-03-29T04:17:56.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404f7fe640 1 --2- 172.21.15.22:0/797687562 >> v2:172.21.15.151:3300/0 conn(0x7f40500b3f50 0x7f4050169310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404dffb640 1 -- 172.21.15.22:0/797687562 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2574027305 0 0) 0x7f403c003530 con 0x7f40500cef80 2024-03-29T04:17:56.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404dffb640 1 -- 172.21.15.22:0/797687562 >> v2:172.21.15.151:3300/0 conn(0x7f40500b3f50 msgr2=0x7f4050169310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404dffb640 1 --2- 172.21.15.22:0/797687562 >> v2:172.21.15.151:3300/0 conn(0x7f40500b3f50 0x7f4050169310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404dffb640 1 -- 172.21.15.22:0/797687562 >> v1:172.21.15.151:6789/0 conn(0x7f40500b55a0 legacy=0x7f40500b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404dffb640 1 -- 172.21.15.22:0/797687562 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40500b4da0 con 0x7f40500cef80 2024-03-29T04:17:56.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404dffb640 1 -- 172.21.15.22:0/797687562 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f4040001a00 con 0x7f40500cef80 2024-03-29T04:17:56.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404f7fe640 1 --2- 172.21.15.22:0/797687562 >> v2:172.21.15.151:3300/0 conn(0x7f40500b3f50 0x7f4050169310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:17:56.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404dffb640 1 -- 172.21.15.22:0/797687562 >> v1:172.21.15.22:6789/0 conn(0x7f40500cef80 legacy=0x7f40500cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404dffb640 1 --2- 172.21.15.22:0/797687562 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f403c0039c0 0x7f403c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404dffb640 1 --2- 172.21.15.22:0/797687562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f403c004810 0x7f403c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404dffb640 1 --2- 172.21.15.22:0/797687562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f403c0051d0 0x7f403c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404dffb640 1 -- 172.21.15.22:0/797687562 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f40500b4ac0 con 0x7f403c0051d0 2024-03-29T04:17:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404dffb640 1 -- 172.21.15.22:0/797687562 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f403c00c120 con 0x7f403c004810 2024-03-29T04:17:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404dffb640 1 -- 172.21.15.22:0/797687562 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f403c00c320 con 0x7f403c0039c0 2024-03-29T04:17:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404f7fe640 1 --2- 172.21.15.22:0/797687562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f403c0051d0 0x7f403c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404e7fc640 1 --2- 172.21.15.22:0/797687562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f403c004810 0x7f403c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404effd640 1 --2- 172.21.15.22:0/797687562 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f403c0039c0 0x7f403c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404f7fe640 1 -- 172.21.15.22:0/797687562 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f403c0039c0 msgr2=0x7f403c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404f7fe640 1 --2- 172.21.15.22:0/797687562 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f403c0039c0 0x7f403c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404f7fe640 1 -- 172.21.15.22:0/797687562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f403c004810 msgr2=0x7f403c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404f7fe640 1 --2- 172.21.15.22:0/797687562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f403c004810 0x7f403c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.072+0000 7f404f7fe640 1 -- 172.21.15.22:0/797687562 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c003530 con 0x7f403c0051d0 2024-03-29T04:17:56.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.076+0000 7f404f7fe640 1 -- 172.21.15.22:0/797687562 reap_dead start 2024-03-29T04:17:56.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.076+0000 7f404f7fe640 1 --2- 172.21.15.22:0/797687562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f403c0051d0 0x7f403c00b9d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f40480098d0 tx=0x7f404801c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16c308a8620a0e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.076+0000 7f404dffb640 1 -- 172.21.15.22:0/797687562 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404800f040 con 0x7f403c0051d0 2024-03-29T04:17:56.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.076+0000 7f404dffb640 1 -- 172.21.15.22:0/797687562 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40480a6660 con 0x7f403c0051d0 2024-03-29T04:17:56.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.076+0000 7f404dffb640 1 -- 172.21.15.22:0/797687562 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4048006e40 con 0x7f403c0051d0 2024-03-29T04:17:56.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.076+0000 7f40555ee640 1 -- 172.21.15.22:0/797687562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f403c0051d0 msgr2=0x7f403c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.076+0000 7f40555ee640 1 --2- 172.21.15.22:0/797687562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f403c0051d0 0x7f403c00b9d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f40480098d0 tx=0x7f404801c650 comp rx=0 tx=0).stop 2024-03-29T04:17:56.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.076+0000 7f40555ee640 1 -- 172.21.15.22:0/797687562 shutdown_connections 2024-03-29T04:17:56.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.076+0000 7f40555ee640 1 --2- 172.21.15.22:0/797687562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f403c0051d0 0x7f403c00b9d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.076+0000 7f40555ee640 1 -- 172.21.15.22:0/797687562 >> 172.21.15.22:0/797687562 conn(0x7f40500d4310 msgr2=0x7f40500d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.076+0000 7f40555ee640 1 -- 172.21.15.22:0/797687562 shutdown_connections 2024-03-29T04:17:56.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.076+0000 7f40555ee640 1 -- 172.21.15.22:0/797687562 wait complete. 2024-03-29T04:17:56.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.076+0000 7f40555ee640 1 Processor -- start 2024-03-29T04:17:56.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.076+0000 7f40555ee640 1 -- start start 2024-03-29T04:17:56.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f40555ee640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f403c0051d0 0x7f405015fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f40555ee640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f40501f7aa0 0x7f405015ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f40555ee640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f405015efd0 0x7f40501f4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f40555ee640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f40500b3840 con 0x7f403c0051d0 2024-03-29T04:17:56.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f40555ee640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f40500b4380 con 0x7f405015efd0 2024-03-29T04:17:56.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f40555ee640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f40500b39c0 con 0x7f40501f7aa0 2024-03-29T04:17:56.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f404effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f403c0051d0 0x7f405015fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f404effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f403c0051d0 0x7f405015fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51290/0 (socket says 172.21.15.22:51290) 2024-03-29T04:17:56.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f404effd640 1 -- 172.21.15.22:0/3688316135 learned_addr learned my addr 172.21.15.22:0/3688316135 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f404f7fe640 1 --2- 172.21.15.22:0/3688316135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f405015efd0 0x7f40501f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f404e7fc640 1 --2- 172.21.15.22:0/3688316135 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f40501f7aa0 0x7f405015ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f404effd640 1 -- 172.21.15.22:0/3688316135 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f40501f7aa0 msgr2=0x7f405015ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f404effd640 1 --2- 172.21.15.22:0/3688316135 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f40501f7aa0 0x7f405015ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f404effd640 1 -- 172.21.15.22:0/3688316135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f405015efd0 msgr2=0x7f40501f4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f404effd640 1 --2- 172.21.15.22:0/3688316135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f405015efd0 0x7f40501f4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f404effd640 1 -- 172.21.15.22:0/3688316135 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40501f4de0 con 0x7f403c0051d0 2024-03-29T04:17:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f404f7fe640 1 --2- 172.21.15.22:0/3688316135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f405015efd0 0x7f40501f4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:17:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f404effd640 1 --2- 172.21.15.22:0/3688316135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f403c0051d0 0x7f405015fbb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4040007a40 tx=0x7f404000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f40477fe640 1 -- 172.21.15.22:0/3688316135 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404000edf0 con 0x7f403c0051d0 2024-03-29T04:17:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f40477fe640 1 -- 172.21.15.22:0/3688316135 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4040018070 con 0x7f403c0051d0 2024-03-29T04:17:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f40477fe640 1 -- 172.21.15.22:0/3688316135 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4040013660 con 0x7f403c0051d0 2024-03-29T04:17:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f40555ee640 1 -- 172.21.15.22:0/3688316135 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40501f50c0 con 0x7f403c0051d0 2024-03-29T04:17:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f40555ee640 1 -- 172.21.15.22:0/3688316135 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40501f4870 con 0x7f403c0051d0 2024-03-29T04:17:56.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f40555ee640 1 -- 172.21.15.22:0/3688316135 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4050068890 con 0x7f403c0051d0 2024-03-29T04:17:56.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f40477fe640 1 -- 172.21.15.22:0/3688316135 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f4040013800 con 0x7f403c0051d0 2024-03-29T04:17:56.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f40477fe640 1 --2- 172.21.15.22:0/3688316135 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4030067fc0 0x7f403006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f40477fe640 1 -- 172.21.15.22:0/3688316135 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f4040086d70 con 0x7f403c0051d0 2024-03-29T04:17:56.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.080+0000 7f404e7fc640 1 --2- 172.21.15.22:0/3688316135 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4030067fc0 0x7f403006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.084+0000 7f404e7fc640 1 --2- 172.21.15.22:0/3688316135 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4030067fc0 0x7f403006a480 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40500d3d40 tx=0x7f4038009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.084+0000 7f40477fe640 1 -- 172.21.15.22:0/3688316135 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f4040057270 con 0x7f403c0051d0 2024-03-29T04:17:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.244+0000 7f40555ee640 1 -- 172.21.15.22:0/3688316135 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40500b3620 con 0x7f403c0051d0 2024-03-29T04:17:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.244+0000 7f40477fe640 1 -- 172.21.15.22:0/3688316135 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18514 (secure 0 0 0) 0x7f4040056b70 con 0x7f403c0051d0 2024-03-29T04:17:56.227 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:56.227 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:17:42.645632+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:17:56.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.248+0000 7f40555ee640 1 -- 172.21.15.22:0/3688316135 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4030067fc0 msgr2=0x7f403006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.248+0000 7f40555ee640 1 --2- 172.21.15.22:0/3688316135 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4030067fc0 0x7f403006a480 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40500d3d40 tx=0x7f4038009040 comp rx=0 tx=0).stop 2024-03-29T04:17:56.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.248+0000 7f40555ee640 1 -- 172.21.15.22:0/3688316135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f403c0051d0 msgr2=0x7f405015fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.248+0000 7f40555ee640 1 --2- 172.21.15.22:0/3688316135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f403c0051d0 0x7f405015fbb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4040007a40 tx=0x7f404000e530 comp rx=0 tx=0).stop 2024-03-29T04:17:56.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.248+0000 7f40555ee640 1 -- 172.21.15.22:0/3688316135 shutdown_connections 2024-03-29T04:17:56.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.248+0000 7f40555ee640 1 --2- 172.21.15.22:0/3688316135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f405015efd0 0x7f40501f4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.248+0000 7f40555ee640 1 --2- 172.21.15.22:0/3688316135 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f40501f7aa0 0x7f405015ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.248+0000 7f40555ee640 1 --2- 172.21.15.22:0/3688316135 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4030067fc0 0x7f403006a480 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.248+0000 7f40555ee640 1 --2- 172.21.15.22:0/3688316135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f403c0051d0 0x7f405015fbb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.248+0000 7f40555ee640 1 -- 172.21.15.22:0/3688316135 >> 172.21.15.22:0/3688316135 conn(0x7f40500d4310 msgr2=0x7f405015dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.248+0000 7f40555ee640 1 -- 172.21.15.22:0/3688316135 shutdown_connections 2024-03-29T04:17:56.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.248+0000 7f40555ee640 1 -- 172.21.15.22:0/3688316135 wait complete. 2024-03-29T04:17:56.246 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-29T04:17:56.246 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-29T04:17:56.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-29T04:17:56.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-29T04:17:56.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-29T04:17:56.248 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-29T04:17:56.248 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-29T04:17:56.249 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-29T04:17:56.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.444+0000 7f61695aa640 1 Processor -- start 2024-03-29T04:17:56.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f61695aa640 1 -- start start 2024-03-29T04:17:56.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f61695aa640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f615c054b60 0x7f615c055360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f61695aa640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f615c0558a0 0x7f615c0366b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f61695aa640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f615c0e5aa0 con 0x7f615c0e1320 2024-03-29T04:17:56.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f61695aa640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f615c051c30 con 0x7f615c054b60 2024-03-29T04:17:56.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f61627fc640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f615c054b60 0x7f615c055360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f61695aa640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f615c051f10 con 0x7f615c0558a0 2024-03-29T04:17:56.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f6162ffd640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f615c0e1320 0x7f615c052710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.22:60030/0 (socket says 172.21.15.22:60030) 2024-03-29T04:17:56.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f61627fc640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f615c054b60 0x7f615c055360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:46314/0 (socket says 172.21.15.22:46314) 2024-03-29T04:17:56.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f6162ffd640 1 -- 172.21.15.22:0/2632792009 learned_addr learned my addr 172.21.15.22:0/2632792009 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:56.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f61637fe640 1 --2- 172.21.15.22:0/2632792009 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f615c0558a0 0x7f615c0366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f7839e61640 1 Processor -- start 2024-03-29T04:17:56.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f7839e61640 1 -- start start 2024-03-29T04:17:56.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f0fcc45d640 1 Processor -- start 2024-03-29T04:17:56.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f0fcc45d640 1 -- start start 2024-03-29T04:17:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f4525a90640 1 Processor -- start 2024-03-29T04:17:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f4525a90640 1 -- start start 2024-03-29T04:17:56.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f0fcc45d640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fbc054230 0x7f0fbc04f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f0fcc45d640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f0fbc054610 0x7f0fbc04f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f0fcc45d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0fbc104850 0x7f0fbc104c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f0fcc45d640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f0fbc053ae0 con 0x7f0fbc104850 2024-03-29T04:17:56.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f0fcc45d640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f0fbc037b30 con 0x7f0fbc054610 2024-03-29T04:17:56.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f0fcc45d640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0fbc053800 con 0x7f0fbc054230 2024-03-29T04:17:56.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f7839e61640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f783408b300 0x7f783408b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f7839e61640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f78340b4080 0x7f78340b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f7839e61640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f783416b650 con 0x7f783406d670 2024-03-29T04:17:56.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f7839e61640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f783408bc20 con 0x7f783408b300 2024-03-29T04:17:56.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f7839e61640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f783416b8c0 con 0x7f78340b4080 2024-03-29T04:17:56.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f0fca9fa640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0fbc104850 0x7f0fbc104c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f0fca9fa640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0fbc104850 0x7f0fbc104c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51300/0 (socket says 172.21.15.22:51300) 2024-03-29T04:17:56.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f0fca9fa640 1 -- 172.21.15.22:0/4085821311 learned_addr learned my addr 172.21.15.22:0/4085821311 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f7833fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f783406d670 0x7f783406dae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44146/0 (socket says 172.21.15.22:44146) 2024-03-29T04:17:56.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f7833fff640 1 -- 172.21.15.22:0/701762535 learned_addr learned my addr 172.21.15.22:0/701762535 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:56.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f7832ffd640 1 --2- 172.21.15.22:0/701762535 >> v2:172.21.15.151:3300/0 conn(0x7f78340b4080 0x7f78340b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f0fc99f8640 1 --2- 172.21.15.22:0/4085821311 >> v2:172.21.15.151:3300/0 conn(0x7f0fbc054610 0x7f0fbc04f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f0fca1f9640 1 --2- 172.21.15.22:0/4085821311 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fbc054230 0x7f0fbc04f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f61627fc640 1 -- 172.21.15.22:0/2632792009 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f615c0558a0 msgr2=0x7f615c0366b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f61627fc640 1 --2- 172.21.15.22:0/2632792009 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f615c0558a0 0x7f615c0366b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f61627fc640 1 -- 172.21.15.22:0/2632792009 >> v1:172.21.15.151:6789/0 conn(0x7f615c0e1320 legacy=0x7f615c052710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f61627fc640 1 -- 172.21.15.22:0/2632792009 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615c0521f0 con 0x7f615c054b60 2024-03-29T04:17:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f148ffff640 1 Processor -- start 2024-03-29T04:17:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f0fca9fa640 1 -- 172.21.15.22:0/4085821311 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fbc054230 msgr2=0x7f0fbc04f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f0fca9fa640 1 --2- 172.21.15.22:0/4085821311 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fbc054230 0x7f0fbc04f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f0fca9fa640 1 -- 172.21.15.22:0/4085821311 >> v2:172.21.15.151:3300/0 conn(0x7f0fbc054610 msgr2=0x7f0fbc04f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f0fca9fa640 1 --2- 172.21.15.22:0/4085821311 >> v2:172.21.15.151:3300/0 conn(0x7f0fbc054610 0x7f0fbc04f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f0fca9fa640 1 -- 172.21.15.22:0/4085821311 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc03a550 con 0x7f0fbc104850 2024-03-29T04:17:56.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f148ffff640 1 -- start start 2024-03-29T04:17:56.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f78337fe640 1 --2- 172.21.15.22:0/701762535 >> v2:172.21.15.22:3300/0 conn(0x7f783408b300 0x7f783408b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f78327fc640 1 -- 172.21.15.22:0/701762535 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3674516045 0 0) 0x7f783416b650 con 0x7f783406d670 2024-03-29T04:17:56.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.448+0000 7f78327fc640 1 -- 172.21.15.22:0/701762535 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f781c003450 con 0x7f783406d670 2024-03-29T04:17:56.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f0fc99f8640 1 --2- 172.21.15.22:0/4085821311 >> v2:172.21.15.151:3300/0 conn(0x7f0fbc054610 0x7f0fbc04f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:17:56.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f78327fc640 1 -- 172.21.15.22:0/701762535 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 875625002 0 0) 0x7f781c003450 con 0x7f783406d670 2024-03-29T04:17:56.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f78327fc640 1 -- 172.21.15.22:0/701762535 >> v2:172.21.15.151:3300/0 conn(0x7f78340b4080 msgr2=0x7f78340b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f78327fc640 1 --2- 172.21.15.22:0/701762535 >> v2:172.21.15.151:3300/0 conn(0x7f78340b4080 0x7f78340b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f78327fc640 1 -- 172.21.15.22:0/701762535 >> v2:172.21.15.22:3300/0 conn(0x7f783408b300 msgr2=0x7f783408b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f867cbf7640 1 Processor -- start 2024-03-29T04:17:56.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f78327fc640 1 --2- 172.21.15.22:0/701762535 >> v2:172.21.15.22:3300/0 conn(0x7f783408b300 0x7f783408b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f78327fc640 1 -- 172.21.15.22:0/701762535 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783416bba0 con 0x7f783406d670 2024-03-29T04:17:56.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f78327fc640 1 -- 172.21.15.22:0/701762535 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f7828001680 con 0x7f783406d670 2024-03-29T04:17:56.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f78327fc640 1 -- 172.21.15.22:0/701762535 >> v1:172.21.15.22:6789/0 conn(0x7f783406d670 legacy=0x7f783406dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f78327fc640 1 --2- 172.21.15.22:0/701762535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f781c003dc0 0x7f781c0041e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f78327fc640 1 --2- 172.21.15.22:0/701762535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f781c004720 0x7f781c00d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f78327fc640 1 --2- 172.21.15.22:0/701762535 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f781c00da40 0x7f781c00fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f78327fc640 1 -- 172.21.15.22:0/701762535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f783408bc20 con 0x7f781c004720 2024-03-29T04:17:56.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f78327fc640 1 -- 172.21.15.22:0/701762535 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f783416b8c0 con 0x7f781c003dc0 2024-03-29T04:17:56.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f78327fc640 1 -- 172.21.15.22:0/701762535 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f781c009170 con 0x7f781c00da40 2024-03-29T04:17:56.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f7832ffd640 1 --2- 172.21.15.22:0/701762535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f781c004720 0x7f781c00d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f4525a90640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f45180e32e0 0x7f451803e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f7833fff640 1 --2- 172.21.15.22:0/701762535 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f781c00da40 0x7f781c00fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f148ffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f149015e580 0x7f14900b9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f4525a90640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f4518053c20 0x7f45180540b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f4525a90640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4518057df0 con 0x7f45180e6c40 2024-03-29T04:17:56.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f61627fc640 1 --2- 172.21.15.22:0/2632792009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f615c054b60 0x7f615c055360 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f61640512a0 tx=0x7f6164097b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a4e229e964f8723 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f4525a90640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f451803bfd0 con 0x7f45180e32e0 2024-03-29T04:17:56.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f4525a90640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f45180580d0 con 0x7f4518053c20 2024-03-29T04:17:56.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f148ffff640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f14900cf5a0 0x7f149006bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f148ffff640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14900d3610 con 0x7f14900d2e70 2024-03-29T04:17:56.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f148ffff640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f14900d38f0 con 0x7f149015e580 2024-03-29T04:17:56.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f148ffff640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f14900d3bd0 con 0x7f14900cf5a0 2024-03-29T04:17:56.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f6161ffb640 1 -- 172.21.15.22:0/2632792009 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164120800 con 0x7f615c054b60 2024-03-29T04:17:56.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f6161ffb640 1 -- 172.21.15.22:0/2632792009 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61641209e0 con 0x7f615c054b60 2024-03-29T04:17:56.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f6161ffb640 1 -- 172.21.15.22:0/2632792009 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164120e50 con 0x7f615c054b60 2024-03-29T04:17:56.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f451effd640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f4518053c20 0x7f45180540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f451effd640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f4518053c20 0x7f45180540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:46350/0 (socket says 172.21.15.22:46350) 2024-03-29T04:17:56.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f451effd640 1 -- 172.21.15.22:0/900023729 learned_addr learned my addr 172.21.15.22:0/900023729 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f867cbf7640 1 -- start start 2024-03-29T04:17:56.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f867cbf7640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86700df4d0 0x7f867003acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f867cbf7640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f86700df8b0 0x7f8670034750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f867cbf7640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8670053530 con 0x7f8670038500 2024-03-29T04:17:56.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f867cbf7640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f8670053810 con 0x7f86700df8b0 2024-03-29T04:17:56.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f867cbf7640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8670053af0 con 0x7f86700df4d0 2024-03-29T04:17:56.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f61695aa640 1 -- 172.21.15.22:0/2632792009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f615c054b60 msgr2=0x7f615c055360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f61695aa640 1 --2- 172.21.15.22:0/2632792009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f615c054b60 0x7f615c055360 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f61640512a0 tx=0x7f6164097b80 comp rx=0 tx=0).stop 2024-03-29T04:17:56.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f148f7fe640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f14900cf5a0 0x7f149006bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f148f7fe640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f14900cf5a0 0x7f149006bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:46334/0 (socket says 172.21.15.22:46334) 2024-03-29T04:17:56.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f148f7fe640 1 -- 172.21.15.22:0/2204987611 learned_addr learned my addr 172.21.15.22:0/2204987611 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f148f7fe640 1 -- 172.21.15.22:0/2204987611 >> v2:172.21.15.22:3300/0 conn(0x7f149015e580 msgr2=0x7f14900b9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f148f7fe640 1 --2- 172.21.15.22:0/2204987611 >> v2:172.21.15.22:3300/0 conn(0x7f149015e580 0x7f14900b9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f148f7fe640 1 -- 172.21.15.22:0/2204987611 >> v1:172.21.15.22:6789/0 conn(0x7f14900d2e70 legacy=0x7f14900cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f148f7fe640 1 -- 172.21.15.22:0/2204987611 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149006b7e0 con 0x7f14900cf5a0 2024-03-29T04:17:56.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f61695aa640 1 -- 172.21.15.22:0/2632792009 shutdown_connections 2024-03-29T04:17:56.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f867659c640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86700df4d0 0x7f867003acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f61695aa640 1 --2- 172.21.15.22:0/2632792009 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f615c0558a0 0x7f615c0366b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f61695aa640 1 --2- 172.21.15.22:0/2632792009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f615c054b60 0x7f615c055360 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f61695aa640 1 -- 172.21.15.22:0/2632792009 >> 172.21.15.22:0/2632792009 conn(0x7f615c020050 msgr2=0x7f615c056d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f867659c640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86700df4d0 0x7f867003acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.22:52842/0 (socket says 172.21.15.22:52842) 2024-03-29T04:17:56.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f867659c640 1 -- 172.21.15.22:0/2940890269 learned_addr learned my addr 172.21.15.22:0/2940890269 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f8675d9b640 1 --2- 172.21.15.22:0/2940890269 >> v2:172.21.15.151:3300/0 conn(0x7f86700df8b0 0x7f8670034750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f61695aa640 1 -- 172.21.15.22:0/2632792009 shutdown_connections 2024-03-29T04:17:56.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f61695aa640 1 -- 172.21.15.22:0/2632792009 wait complete. 2024-03-29T04:17:56.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f7832ffd640 1 -- 172.21.15.22:0/701762535 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f781c00da40 msgr2=0x7f781c00fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f7832ffd640 1 --2- 172.21.15.22:0/701762535 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f781c00da40 0x7f781c00fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f7832ffd640 1 -- 172.21.15.22:0/701762535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f781c003dc0 msgr2=0x7f781c0041e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:17:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f7832ffd640 1 --2- 172.21.15.22:0/701762535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f781c003dc0 0x7f781c0041e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f7832ffd640 1 -- 172.21.15.22:0/701762535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783416bba0 con 0x7f781c004720 2024-03-29T04:17:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f7833fff640 1 -- 172.21.15.22:0/701762535 reap_dead start 2024-03-29T04:17:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f78337fe640 1 --2- 172.21.15.22:0/701762535 >> v2:172.21.15.22:3300/0 conn(0x7f783408b300 0x7f783408b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f0fca9fa640 1 --2- 172.21.15.22:0/4085821311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0fbc104850 0x7f0fbc104c80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc052490 tx=0x7f0fc4086c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336daae47c270b9a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f7832ffd640 1 --2- 172.21.15.22:0/701762535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f781c004720 0x7f781c00d500 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f782c009fc0 tx=0x7f782c01cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62d212ec10a733d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f451effd640 1 -- 172.21.15.22:0/900023729 >> v2:172.21.15.22:3300/0 conn(0x7f45180e32e0 msgr2=0x7f451803e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f451effd640 1 --2- 172.21.15.22:0/900023729 >> v2:172.21.15.22:3300/0 conn(0x7f45180e32e0 0x7f451803e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f451effd640 1 -- 172.21.15.22:0/900023729 >> v1:172.21.15.22:6789/0 conn(0x7f45180e6c40 legacy=0x7f45180e7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f451effd640 1 -- 172.21.15.22:0/900023729 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45180583b0 con 0x7f4518053c20 2024-03-29T04:17:56.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f78327fc640 1 -- 172.21.15.22:0/701762535 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c0b2070 con 0x7f781c004720 2024-03-29T04:17:56.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f78327fc640 1 -- 172.21.15.22:0/701762535 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f782c0a5bb0 con 0x7f781c004720 2024-03-29T04:17:56.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f78327fc640 1 -- 172.21.15.22:0/701762535 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c009c10 con 0x7f781c004720 2024-03-29T04:17:56.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f45566fc640 1 Processor -- start 2024-03-29T04:17:56.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f507bac2640 1 Processor -- start 2024-03-29T04:17:56.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f507bac2640 1 -- start start 2024-03-29T04:17:56.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f61695aa640 1 Processor -- start 2024-03-29T04:17:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f507bac2640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f507415b020 0x7f507406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f507bac2640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f50740b5060 0x7f50740cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f507bac2640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50740b5670 con 0x7f507406ff20 2024-03-29T04:17:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f507bac2640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f5074070360 con 0x7f50740b5060 2024-03-29T04:17:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f507bac2640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f50740b5950 con 0x7f507415b020 2024-03-29T04:17:56.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f867659c640 1 -- 172.21.15.22:0/2940890269 >> v2:172.21.15.151:3300/0 conn(0x7f86700df8b0 msgr2=0x7f8670034750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f867659c640 1 --2- 172.21.15.22:0/2940890269 >> v2:172.21.15.151:3300/0 conn(0x7f86700df8b0 0x7f8670034750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f148f7fe640 1 --2- 172.21.15.22:0/2204987611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14900cf5a0 0x7f149006bd00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f14900ceaf0 tx=0x7f14880251e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b57d2ced842f4c81 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f867659c640 1 -- 172.21.15.22:0/2940890269 >> v1:172.21.15.151:6789/0 conn(0x7f8670038500 legacy=0x7f86700545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.452+0000 7f867659c640 1 -- 172.21.15.22:0/2940890269 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8670053dd0 con 0x7f86700df4d0 2024-03-29T04:17:56.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f0fc91f7640 1 -- 172.21.15.22:0/4085821311 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc410f930 con 0x7f0fbc104850 2024-03-29T04:17:56.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f0fc91f7640 1 -- 172.21.15.22:0/4085821311 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fc410fb10 con 0x7f0fbc104850 2024-03-29T04:17:56.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f0fc91f7640 1 -- 172.21.15.22:0/4085821311 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4117440 con 0x7f0fbc104850 2024-03-29T04:17:56.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f61695aa640 1 -- start start 2024-03-29T04:17:56.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f45566fc640 1 -- start start 2024-03-29T04:17:56.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f451effd640 1 --2- 172.21.15.22:0/900023729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4518053c20 0x7f45180540b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4520074a70 tx=0x7f4520097c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=564b44e4f604784f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f7839e61640 1 -- 172.21.15.22:0/701762535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f781c004720 msgr2=0x7f781c00d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f7839e61640 1 --2- 172.21.15.22:0/701762535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f781c004720 0x7f781c00d500 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f782c009fc0 tx=0x7f782c01cc50 comp rx=0 tx=0).stop 2024-03-29T04:17:56.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f451e7fc640 1 -- 172.21.15.22:0/900023729 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45201208b0 con 0x7f4518053c20 2024-03-29T04:17:56.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f451e7fc640 1 -- 172.21.15.22:0/900023729 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4520120a90 con 0x7f4518053c20 2024-03-29T04:17:56.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f451e7fc640 1 -- 172.21.15.22:0/900023729 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520128440 con 0x7f4518053c20 2024-03-29T04:17:56.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f7839e61640 1 -- 172.21.15.22:0/701762535 shutdown_connections 2024-03-29T04:17:56.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f7839e61640 1 --2- 172.21.15.22:0/701762535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f781c004720 0x7f781c00d500 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f7839e61640 1 -- 172.21.15.22:0/701762535 >> 172.21.15.22:0/701762535 conn(0x7f783406c570 msgr2=0x7f783406c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f7839e61640 1 -- 172.21.15.22:0/701762535 shutdown_connections 2024-03-29T04:17:56.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f0fcc45d640 1 -- 172.21.15.22:0/4085821311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0fbc104850 msgr2=0x7f0fbc104c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f0fcc45d640 1 --2- 172.21.15.22:0/4085821311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0fbc104850 0x7f0fbc104c80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc052490 tx=0x7f0fc4086c00 comp rx=0 tx=0).stop 2024-03-29T04:17:56.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f0fcc45d640 1 -- 172.21.15.22:0/4085821311 shutdown_connections 2024-03-29T04:17:56.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f0fcc45d640 1 --2- 172.21.15.22:0/4085821311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0fbc104850 0x7f0fbc104c80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f0fcc45d640 1 --2- 172.21.15.22:0/4085821311 >> v2:172.21.15.151:3300/0 conn(0x7f0fbc054610 0x7f0fbc04f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f0fcc45d640 1 --2- 172.21.15.22:0/4085821311 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fbc054230 0x7f0fbc04f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f0fcc45d640 1 -- 172.21.15.22:0/4085821311 >> 172.21.15.22:0/4085821311 conn(0x7f0fbc009fc0 msgr2=0x7f0fbc00a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f0fcc45d640 1 -- 172.21.15.22:0/4085821311 shutdown_connections 2024-03-29T04:17:56.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f0fcc45d640 1 -- 172.21.15.22:0/4085821311 wait complete. 2024-03-29T04:17:56.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f148e7fc640 1 -- 172.21.15.22:0/2204987611 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14880b9070 con 0x7f14900cf5a0 2024-03-29T04:17:56.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f148e7fc640 1 -- 172.21.15.22:0/2204987611 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14880b4400 con 0x7f14900cf5a0 2024-03-29T04:17:56.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f148ffff640 1 -- 172.21.15.22:0/2204987611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14900cf5a0 msgr2=0x7f149006bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f148ffff640 1 --2- 172.21.15.22:0/2204987611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14900cf5a0 0x7f149006bd00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f14900ceaf0 tx=0x7f14880251e0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f148ffff640 1 -- 172.21.15.22:0/2204987611 shutdown_connections 2024-03-29T04:17:56.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f148ffff640 1 --2- 172.21.15.22:0/2204987611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14900cf5a0 0x7f149006bd00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f148ffff640 1 --2- 172.21.15.22:0/2204987611 >> v2:172.21.15.22:3300/0 conn(0x7f149015e580 0x7f14900b9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f148ffff640 1 -- 172.21.15.22:0/2204987611 >> 172.21.15.22:0/2204987611 conn(0x7f14900d40c0 msgr2=0x7f14900d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f507985e640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f507415b020 0x7f507406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f507985e640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f507415b020 0x7f507406f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.22:52854/0 (socket says 172.21.15.22:52854) 2024-03-29T04:17:56.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f507985e640 1 -- 172.21.15.22:0/2801705020 learned_addr learned my addr 172.21.15.22:0/2801705020 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f507a05f640 1 --2- 172.21.15.22:0/2801705020 >> v2:172.21.15.151:3300/0 conn(0x7f50740b5060 0x7f50740cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f867659c640 1 --2- 172.21.15.22:0/2940890269 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86700df4d0 0x7f867003acf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8668009870 tx=0x7f866801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab28211b028d1f8f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f7839e61640 1 -- 172.21.15.22:0/701762535 wait complete. 2024-03-29T04:17:56.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f148ffff640 1 -- 172.21.15.22:0/2204987611 shutdown_connections 2024-03-29T04:17:56.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f4525a90640 1 -- 172.21.15.22:0/900023729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4518053c20 msgr2=0x7f45180540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f4525a90640 1 --2- 172.21.15.22:0/900023729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4518053c20 0x7f45180540b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4520074a70 tx=0x7f4520097c30 comp rx=0 tx=0).stop 2024-03-29T04:17:56.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f61695aa640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f615c054b60 0x7f615c0da1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f61695aa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f615c0558a0 0x7f615c0db710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f61695aa640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f615c0e1320 0x7f615c04ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f61695aa640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f615c0ee190 con 0x7f615c0558a0 2024-03-29T04:17:56.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f61695aa640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f615c0ee010 con 0x7f615c0e1320 2024-03-29T04:17:56.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f61695aa640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f615c0ee310 con 0x7f615c054b60 2024-03-29T04:17:56.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f61627fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f615c0558a0 0x7f615c0db710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f61627fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f615c0558a0 0x7f615c0db710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51360/0 (socket says 172.21.15.22:51360) 2024-03-29T04:17:56.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f61627fc640 1 -- 172.21.15.22:0/1549931682 learned_addr learned my addr 172.21.15.22:0/1549931682 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f4525a90640 1 -- 172.21.15.22:0/900023729 shutdown_connections 2024-03-29T04:17:56.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f4525a90640 1 --2- 172.21.15.22:0/900023729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4518053c20 0x7f45180540b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f4525a90640 1 --2- 172.21.15.22:0/900023729 >> v2:172.21.15.22:3300/0 conn(0x7f45180e32e0 0x7f451803e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f4525a90640 1 -- 172.21.15.22:0/900023729 >> 172.21.15.22:0/900023729 conn(0x7f45180559f0 msgr2=0x7f4518055e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f867559a640 1 -- 172.21.15.22:0/2940890269 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86680ab5f0 con 0x7f86700df4d0 2024-03-29T04:17:56.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f867559a640 1 -- 172.21.15.22:0/2940890269 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86680b0070 con 0x7f86700df4d0 2024-03-29T04:17:56.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f867559a640 1 -- 172.21.15.22:0/2940890269 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86680ab9b0 con 0x7f86700df4d0 2024-03-29T04:17:56.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f4525a90640 1 -- 172.21.15.22:0/900023729 shutdown_connections 2024-03-29T04:17:56.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f61627fc640 1 -- 172.21.15.22:0/1549931682 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f615c054b60 msgr2=0x7f615c0da1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f61627fc640 1 --2- 172.21.15.22:0/1549931682 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f615c054b60 0x7f615c0da1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f61627fc640 1 -- 172.21.15.22:0/1549931682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f615c0e1320 msgr2=0x7f615c04ca80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T04:17:56.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f61627fc640 1 --2- 172.21.15.22:0/1549931682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f615c0e1320 0x7f615c04ca80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f61627fc640 1 -- 172.21.15.22:0/1549931682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615c0dc6c0 con 0x7f615c0558a0 2024-03-29T04:17:56.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f4525a90640 1 -- 172.21.15.22:0/900023729 wait complete. 2024-03-29T04:17:56.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f61627fc640 1 --2- 172.21.15.22:0/1549931682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f615c0558a0 0x7f615c0db710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f616402d950 tx=0x7f61640949c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f7839e61640 1 Processor -- start 2024-03-29T04:17:56.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f867cbf7640 1 -- 172.21.15.22:0/2940890269 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86700df4d0 msgr2=0x7f867003acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f867cbf7640 1 --2- 172.21.15.22:0/2940890269 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86700df4d0 0x7f867003acf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8668009870 tx=0x7f866801c560 comp rx=0 tx=0).stop 2024-03-29T04:17:56.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f867cbf7640 1 -- 172.21.15.22:0/2940890269 shutdown_connections 2024-03-29T04:17:56.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f867cbf7640 1 --2- 172.21.15.22:0/2940890269 >> v2:172.21.15.151:3300/0 conn(0x7f86700df8b0 0x7f8670034750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f867cbf7640 1 --2- 172.21.15.22:0/2940890269 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86700df4d0 0x7f867003acf0 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8668009870 tx=0x7f866801c560 comp rx=0 tx=0).stop 2024-03-29T04:17:56.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f867cbf7640 1 -- 172.21.15.22:0/2940890269 >> 172.21.15.22:0/2940890269 conn(0x7f8670055360 msgr2=0x7f8670055770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f7839e61640 1 -- start start 2024-03-29T04:17:56.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f867cbf7640 1 -- 172.21.15.22:0/2940890269 shutdown_connections 2024-03-29T04:17:56.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f45566fc640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f454804fa80 0x7f454804fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f4554ef9640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f45480df390 0x7f4548007190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44186/0 (socket says 172.21.15.22:44186) 2024-03-29T04:17:56.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f4554ef9640 1 -- 172.21.15.22:0/2497696221 learned_addr learned my addr 172.21.15.22:0/2497696221 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:56.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f45566fc640 1 -- 172.21.15.22:0/2497696221 --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4548054390 con 0x7f45480df390 2024-03-29T04:17:56.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f45566fc640 1 -- 172.21.15.22:0/2497696221 --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45480503a0 con 0x7f454804f5c0 2024-03-29T04:17:56.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f45566fc640 1 -- 172.21.15.22:0/2497696221 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4548005ea0 con 0x7f454804fa80 2024-03-29T04:17:56.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.456+0000 7f148ffff640 1 -- 172.21.15.22:0/2204987611 wait complete. 2024-03-29T04:17:56.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f0fcc45d640 1 Processor -- start 2024-03-29T04:17:56.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f148ffff640 1 Processor -- start 2024-03-29T04:17:56.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f614b7fe640 1 -- 172.21.15.22:0/1549931682 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164128710 con 0x7f615c0558a0 2024-03-29T04:17:56.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f507985e640 1 -- 172.21.15.22:0/2801705020 >> v2:172.21.15.151:3300/0 conn(0x7f50740b5060 msgr2=0x7f50740cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f507985e640 1 --2- 172.21.15.22:0/2801705020 >> v2:172.21.15.151:3300/0 conn(0x7f50740b5060 0x7f50740cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f507985e640 1 -- 172.21.15.22:0/2801705020 >> v1:172.21.15.151:6789/0 conn(0x7f507406ff20 legacy=0x7f50740b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f507985e640 1 -- 172.21.15.22:0/2801705020 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50740b5c30 con 0x7f507415b020 2024-03-29T04:17:56.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f614b7fe640 1 -- 172.21.15.22:0/1549931682 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61641288b0 con 0x7f615c0558a0 2024-03-29T04:17:56.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f614b7fe640 1 -- 172.21.15.22:0/1549931682 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164128ca0 con 0x7f615c0558a0 2024-03-29T04:17:56.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f61695aa640 1 -- 172.21.15.22:0/1549931682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615c0dc9a0 con 0x7f615c0558a0 2024-03-29T04:17:56.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f4525a90640 1 Processor -- start 2024-03-29T04:17:56.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f507a05f640 1 --2- 172.21.15.22:0/2801705020 >> v2:172.21.15.151:3300/0 conn(0x7f50740b5060 0x7f50740cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:17:56.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f61695aa640 1 -- 172.21.15.22:0/1549931682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f615c0dcf60 con 0x7f615c0558a0 2024-03-29T04:17:56.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f7839e61640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f781c004720 0x7f78341ffe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f7839e61640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7834157f40 0x7f78340afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f7839e61640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78340a94d0 0x7f78340a9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f7839e61640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f783416b3f0 con 0x7f781c004720 2024-03-29T04:17:56.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f7839e61640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f783416b270 con 0x7f78340a94d0 2024-03-29T04:17:56.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f7839e61640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f78340b3040 con 0x7f7834157f40 2024-03-29T04:17:56.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f7832ffd640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7834157f40 0x7f78340afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f7833fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78340a94d0 0x7f78340a9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f7832ffd640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7834157f40 0x7f78340afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.22:52868/0 (socket says 172.21.15.22:52868) 2024-03-29T04:17:56.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f7832ffd640 1 -- 172.21.15.22:0/2306056055 learned_addr learned my addr 172.21.15.22:0/2306056055 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f78337fe640 1 --2- 172.21.15.22:0/2306056055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f781c004720 0x7f78341ffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f148ffff640 1 -- start start 2024-03-29T04:17:56.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f148ffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14900cf5a0 0x7f149015f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f148ffff640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f14900d2e70 0x7f14900abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f148ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f149015e580 0x7f14900aad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f0fcc45d640 1 -- start start 2024-03-29T04:17:56.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f0fcc45d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fbc054230 0x7f0fbc1a32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f0fcc45d640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fbc054610 0x7f0fbc037cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f0fcc45d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0fbc104850 0x7f0fbc038200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f0fca1f9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fbc054230 0x7f0fbc1a32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f0fca1f9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fbc054230 0x7f0fbc1a32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:46384/0 (socket says 172.21.15.22:46384) 2024-03-29T04:17:56.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f0fca1f9640 1 -- 172.21.15.22:0/1804424869 learned_addr learned my addr 172.21.15.22:0/1804424869 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f0fcc45d640 1 -- 172.21.15.22:0/1804424869 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc053550 con 0x7f0fbc104850 2024-03-29T04:17:56.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f0fcc45d640 1 -- 172.21.15.22:0/1804424869 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc0533d0 con 0x7f0fbc054230 2024-03-29T04:17:56.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f0fc99f8640 1 --2- 172.21.15.22:0/1804424869 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fbc054610 0x7f0fbc037cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f4525a90640 1 -- start start 2024-03-29T04:17:56.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f4525a90640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4518053c20 0x7f45180e1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f4525a90640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45180e6c40 0x7f45180e5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f4525a90640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45180e6310 0x7f4518184190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f4525a90640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f45180f0690 con 0x7f45180e6c40 2024-03-29T04:17:56.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f4525a90640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f45180f0510 con 0x7f45180e6310 2024-03-29T04:17:56.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f4525a90640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f45180f0810 con 0x7f4518053c20 2024-03-29T04:17:56.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f148ffff640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f149006cde0 con 0x7f149015e580 2024-03-29T04:17:56.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f148ffff640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f149006cc60 con 0x7f14900cf5a0 2024-03-29T04:17:56.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f148effd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14900cf5a0 0x7f149015f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f0fcc45d640 1 -- 172.21.15.22:0/1804424869 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0fbc10e3a0 con 0x7f0fbc054610 2024-03-29T04:17:56.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f148ffff640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f149006cf60 con 0x7f14900d2e70 2024-03-29T04:17:56.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f148effd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14900cf5a0 0x7f149015f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:46400/0 (socket says 172.21.15.22:46400) 2024-03-29T04:17:56.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f148effd640 1 -- 172.21.15.22:0/2102162361 learned_addr learned my addr 172.21.15.22:0/2102162361 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f45556fa640 1 --2- 172.21.15.22:0/2497696221 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f454804fa80 0x7f454804fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f451f7fe640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4518053c20 0x7f45180e1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f451f7fe640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4518053c20 0x7f45180e1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.22:52886/0 (socket says 172.21.15.22:52886) 2024-03-29T04:17:56.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f451f7fe640 1 -- 172.21.15.22:0/3054928051 learned_addr learned my addr 172.21.15.22:0/3054928051 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f451ffff640 1 --2- 172.21.15.22:0/3054928051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45180e6310 0x7f4518184190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f867cbf7640 1 -- 172.21.15.22:0/2940890269 wait complete. 2024-03-29T04:17:56.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f4547fff640 1 -- 172.21.15.22:0/2497696221 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3747805671 0 0) 0x7f4548054390 con 0x7f45480df390 2024-03-29T04:17:56.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f867cbf7640 1 Processor -- start 2024-03-29T04:17:56.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f4547fff640 1 -- 172.21.15.22:0/2497696221 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f453c003530 con 0x7f45480df390 2024-03-29T04:17:56.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f4547fff640 1 -- 172.21.15.22:0/2497696221 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2088348280 0 0) 0x7f45480503a0 con 0x7f454804f5c0 2024-03-29T04:17:56.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f4547fff640 1 -- 172.21.15.22:0/2497696221 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4548054390 con 0x7f454804f5c0 2024-03-29T04:17:56.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f1487fff640 1 --2- 172.21.15.22:0/2102162361 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f14900d2e70 0x7f14900abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f7832ffd640 1 -- 172.21.15.22:0/2306056055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78340a94d0 msgr2=0x7f78340a9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f7832ffd640 1 --2- 172.21.15.22:0/2306056055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78340a94d0 0x7f78340a9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f7832ffd640 1 -- 172.21.15.22:0/2306056055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f781c004720 msgr2=0x7f78341ffe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f7832ffd640 1 --2- 172.21.15.22:0/2306056055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f781c004720 0x7f78341ffe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f7832ffd640 1 -- 172.21.15.22:0/2306056055 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78340b0840 con 0x7f7834157f40 2024-03-29T04:17:56.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f61695aa640 1 -- 172.21.15.22:0/1549931682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6144001040 con 0x7f615c0558a0 2024-03-29T04:17:56.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f614b7fe640 1 -- 172.21.15.22:0/1549931682 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f6164137060 con 0x7f615c0558a0 2024-03-29T04:17:56.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f148f7fe640 1 --2- 172.21.15.22:0/2102162361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f149015e580 0x7f14900aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f614b7fe640 1 --2- 172.21.15.22:0/1549931682 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6150067fc0 0x7f615006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f614b7fe640 1 -- 172.21.15.22:0/1549931682 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f616419ab10 con 0x7f615c0558a0 2024-03-29T04:17:56.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f45556fa640 1 -- 172.21.15.22:0/2497696221 >> v1:172.21.15.151:6789/0 conn(0x7f454804f5c0 legacy=0x7f4548038020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f45556fa640 1 -- 172.21.15.22:0/2497696221 >> v1:172.21.15.22:6789/0 conn(0x7f45480df390 legacy=0x7f4548007190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f45556fa640 1 -- 172.21.15.22:0/2497696221 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45480060f0 con 0x7f454804fa80 2024-03-29T04:17:56.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f7832ffd640 1 --2- 172.21.15.22:0/2306056055 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7834157f40 0x7f78340afed0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f782c01c710 tx=0x7f782c009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f148effd640 1 -- 172.21.15.22:0/2102162361 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f14900d2e70 msgr2=0x7f14900abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f148effd640 1 --2- 172.21.15.22:0/2102162361 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f14900d2e70 0x7f14900abdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f148effd640 1 -- 172.21.15.22:0/2102162361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f149015e580 msgr2=0x7f14900aad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f148effd640 1 --2- 172.21.15.22:0/2102162361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f149015e580 0x7f14900aad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f148effd640 1 -- 172.21.15.22:0/2102162361 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149015fb00 con 0x7f14900cf5a0 2024-03-29T04:17:56.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f0fca1f9640 1 -- 172.21.15.22:0/1804424869 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fbc054610 msgr2=0x7f0fbc037cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f0fca9fa640 1 --2- 172.21.15.22:0/1804424869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0fbc104850 0x7f0fbc038200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f0fca1f9640 1 --2- 172.21.15.22:0/1804424869 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0fbc054610 0x7f0fbc037cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f0fca1f9640 1 -- 172.21.15.22:0/1804424869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0fbc104850 msgr2=0x7f0fbc038200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f451f7fe640 1 -- 172.21.15.22:0/3054928051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45180e6310 msgr2=0x7f4518184190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f451f7fe640 1 --2- 172.21.15.22:0/3054928051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45180e6310 0x7f4518184190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f451f7fe640 1 -- 172.21.15.22:0/3054928051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45180e6c40 msgr2=0x7f45180e5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f451effd640 1 --2- 172.21.15.22:0/3054928051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45180e6c40 0x7f45180e5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f451f7fe640 1 --2- 172.21.15.22:0/3054928051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45180e6c40 0x7f45180e5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f451f7fe640 1 -- 172.21.15.22:0/3054928051 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45180e1c80 con 0x7f4518053c20 2024-03-29T04:17:56.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f451effd640 1 --2- 172.21.15.22:0/3054928051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45180e6c40 0x7f45180e5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:56.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.460+0000 7f867cbf7640 1 -- start start 2024-03-29T04:17:56.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f867cbf7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8670038500 0x7f86700d5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f867cbf7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86700df8b0 0x7f86700de5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f867cbf7640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f867004ede0 0x7f867004a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f867cbf7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8670007ec0 con 0x7f8670038500 2024-03-29T04:17:56.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f867cbf7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8670007d40 con 0x7f86700df8b0 2024-03-29T04:17:56.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f867cbf7640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8670008040 con 0x7f867004ede0 2024-03-29T04:17:56.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f867659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8670038500 0x7f86700d5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f867659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8670038500 0x7f86700d5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51408/0 (socket says 172.21.15.22:51408) 2024-03-29T04:17:56.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f867659c640 1 -- 172.21.15.22:0/436240616 learned_addr learned my addr 172.21.15.22:0/436240616 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f614b7fe640 1 --2- 172.21.15.22:0/1549931682 >> [v2:172.21.15.22:6800/3551960158,v1:172.21.15.22:6801/3551960158] conn(0x7f6150071630 0x7f6150073a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f614b7fe640 1 -- 172.21.15.22:0/1549931682 --> [v2:172.21.15.22:6800/3551960158,v1:172.21.15.22:6801/3551960158] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6164063080 con 0x7f6150071630 2024-03-29T04:17:56.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f148effd640 1 --2- 172.21.15.22:0/2102162361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14900cf5a0 0x7f149015f400 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f147800b550 tx=0x7f147800ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f8675d9b640 1 --2- 172.21.15.22:0/436240616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86700df8b0 0x7f86700de5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f0fca1f9640 1 --2- 172.21.15.22:0/1804424869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0fbc104850 0x7f0fbc038200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f0fca1f9640 1 -- 172.21.15.22:0/1804424869 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc0f3560 con 0x7f0fbc054230 2024-03-29T04:17:56.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f451f7fe640 1 --2- 172.21.15.22:0/3054928051 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4518053c20 0x7f45180e1740 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f451400b550 tx=0x7f451400ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f0fca9fa640 1 --2- 172.21.15.22:0/1804424869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0fbc104850 0x7f0fbc038200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:56.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f148cff9640 1 -- 172.21.15.22:0/2102162361 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478014070 con 0x7f14900cf5a0 2024-03-29T04:17:56.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f8676d9d640 1 --2- 172.21.15.22:0/436240616 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f867004ede0 0x7f867004a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f781bfff640 1 -- 172.21.15.22:0/2306056055 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c0ba070 con 0x7f7834157f40 2024-03-29T04:17:56.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f0fca1f9640 1 --2- 172.21.15.22:0/1804424869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fbc054230 0x7f0fbc1a32b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000d800 tx=0x7f0fc000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f867659c640 1 -- 172.21.15.22:0/436240616 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f867004ede0 msgr2=0x7f867004a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f867659c640 1 --2- 172.21.15.22:0/436240616 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f867004ede0 0x7f867004a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f867659c640 1 -- 172.21.15.22:0/436240616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86700df8b0 msgr2=0x7f86700de5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f867659c640 1 --2- 172.21.15.22:0/436240616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86700df8b0 0x7f86700de5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f867659c640 1 -- 172.21.15.22:0/436240616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8670184490 con 0x7f8670038500 2024-03-29T04:17:56.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f61637fe640 1 --2- 172.21.15.22:0/1549931682 >> [v2:172.21.15.22:6800/3551960158,v1:172.21.15.22:6801/3551960158] conn(0x7f6150071630 0x7f6150073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f6162ffd640 1 --2- 172.21.15.22:0/1549931682 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6150067fc0 0x7f615006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f148ffff640 1 -- 172.21.15.22:0/2102162361 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f149015fde0 con 0x7f14900cf5a0 2024-03-29T04:17:56.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f148ffff640 1 -- 172.21.15.22:0/2102162361 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14901603a0 con 0x7f14900cf5a0 2024-03-29T04:17:56.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f450bfff640 1 -- 172.21.15.22:0/3054928051 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4514014070 con 0x7f4518053c20 2024-03-29T04:17:56.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f148cff9640 1 -- 172.21.15.22:0/2102162361 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1478010870 con 0x7f14900cf5a0 2024-03-29T04:17:56.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f61637fe640 1 --2- 172.21.15.22:0/1549931682 >> [v2:172.21.15.22:6800/3551960158,v1:172.21.15.22:6801/3551960158] conn(0x7f6150071630 0x7f6150073a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f0fbaffd640 1 -- 172.21.15.22:0/1804424869 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0018030 con 0x7f0fbc054230 2024-03-29T04:17:56.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f0fbaffd640 1 -- 172.21.15.22:0/1804424869 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fc00096a0 con 0x7f0fbc054230 2024-03-29T04:17:56.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f867659c640 1 --2- 172.21.15.22:0/436240616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8670038500 0x7f86700d5860 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f86680ad040 tx=0x7f866801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f7839e61640 1 -- 172.21.15.22:0/2306056055 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78340b0b20 con 0x7f7834157f40 2024-03-29T04:17:56.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f7839e61640 1 -- 172.21.15.22:0/2306056055 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f78340b10e0 con 0x7f7834157f40 2024-03-29T04:17:56.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f8666ffd640 1 -- 172.21.15.22:0/436240616 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86680be3f0 con 0x7f8670038500 2024-03-29T04:17:56.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f8666ffd640 1 -- 172.21.15.22:0/436240616 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86680be5b0 con 0x7f8670038500 2024-03-29T04:17:56.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f8666ffd640 1 -- 172.21.15.22:0/436240616 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86680be9a0 con 0x7f8670038500 2024-03-29T04:17:56.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f614b7fe640 1 -- 172.21.15.22:0/1549931682 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6164166bb0 con 0x7f615c0558a0 2024-03-29T04:17:56.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f450bfff640 1 -- 172.21.15.22:0/3054928051 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4514010610 con 0x7f4518053c20 2024-03-29T04:17:56.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f4525a90640 1 -- 172.21.15.22:0/3054928051 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45180e06a0 con 0x7f4518053c20 2024-03-29T04:17:56.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f4525a90640 1 -- 172.21.15.22:0/3054928051 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f45180e0c60 con 0x7f4518053c20 2024-03-29T04:17:56.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f450bfff640 1 -- 172.21.15.22:0/3054928051 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45140109e0 con 0x7f4518053c20 2024-03-29T04:17:56.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f148cff9640 1 -- 172.21.15.22:0/2102162361 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478010c60 con 0x7f14900cf5a0 2024-03-29T04:17:56.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f7839e61640 1 -- 172.21.15.22:0/2306056055 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f78340686e0 con 0x7f7834157f40 2024-03-29T04:17:56.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f0fbaffd640 1 -- 172.21.15.22:0/1804424869 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0009870 con 0x7f0fbc054230 2024-03-29T04:17:56.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f867cbf7640 1 -- 172.21.15.22:0/436240616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8670184770 con 0x7f8670038500 2024-03-29T04:17:56.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f867cbf7640 1 -- 172.21.15.22:0/436240616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8670184d30 con 0x7f8670038500 2024-03-29T04:17:56.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f614b7fe640 1 -- 172.21.15.22:0/1549931682 <== osd.1 v2:172.21.15.22:6800/3551960158 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6164063080 con 0x7f6150071630 2024-03-29T04:17:56.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f4525a90640 1 -- 172.21.15.22:0/3054928051 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4500001040 con 0x7f4518053c20 2024-03-29T04:17:56.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f148ffff640 1 -- 172.21.15.22:0/2102162361 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1474001040 con 0x7f14900cf5a0 2024-03-29T04:17:56.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f0fcc45d640 1 -- 172.21.15.22:0/1804424869 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fbc0f3840 con 0x7f0fbc054230 2024-03-29T04:17:56.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f0fcc45d640 1 -- 172.21.15.22:0/1804424869 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fbc0f3e00 con 0x7f0fbc054230 2024-03-29T04:17:56.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f45556fa640 1 --2- 172.21.15.22:0/2497696221 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f454804fa80 0x7f454804fe60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f454c0090e0 tx=0x7f454c01d710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f37aaaa726cc9528 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f781bfff640 1 -- 172.21.15.22:0/2306056055 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f782c0ada20 con 0x7f7834157f40 2024-03-29T04:17:56.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f867cbf7640 1 -- 172.21.15.22:0/436240616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f86700057f0 con 0x7f8670038500 2024-03-29T04:17:56.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.464+0000 7f148cff9640 1 -- 172.21.15.22:0/2102162361 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1478017070 con 0x7f14900cf5a0 2024-03-29T04:17:56.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f0fcc45d640 1 -- 172.21.15.22:0/1804424869 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0fbc0f41e0 con 0x7f0fbc054230 2024-03-29T04:17:56.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f4547fff640 1 -- 172.21.15.22:0/2497696221 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c0a9650 con 0x7f454804fa80 2024-03-29T04:17:56.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f6162ffd640 1 --2- 172.21.15.22:0/1549931682 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6150067fc0 0x7f615006a480 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f615c184090 tx=0x7f6154007a50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f4547fff640 1 -- 172.21.15.22:0/2497696221 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f454c0a97f0 con 0x7f454804fa80 2024-03-29T04:17:56.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f148cff9640 1 --2- 172.21.15.22:0/2102162361 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1470067fc0 0x7f147006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f4547fff640 1 -- 172.21.15.22:0/2497696221 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c0a9650 con 0x7f454804fa80 2024-03-29T04:17:56.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f148cff9640 1 -- 172.21.15.22:0/2102162361 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1478086660 con 0x7f14900cf5a0 2024-03-29T04:17:56.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f781bfff640 1 -- 172.21.15.22:0/2306056055 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c0b2040 con 0x7f7834157f40 2024-03-29T04:17:56.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f1487fff640 1 --2- 172.21.15.22:0/2102162361 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1470067fc0 0x7f147006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f148cff9640 1 --2- 172.21.15.22:0/2102162361 >> [v2:172.21.15.151:6800/3763105575,v1:172.21.15.151:6802/3763105575] conn(0x7f1470071740 0x7f1470073b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f148cff9640 1 -- 172.21.15.22:0/2102162361 --> [v2:172.21.15.151:6800/3763105575,v1:172.21.15.151:6802/3763105575] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1478010870 con 0x7f1470071740 2024-03-29T04:17:56.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f148cff9640 1 -- 172.21.15.22:0/2102162361 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f147808d050 con 0x7f14900cf5a0 2024-03-29T04:17:56.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f148f7fe640 1 --2- 172.21.15.22:0/2102162361 >> [v2:172.21.15.151:6800/3763105575,v1:172.21.15.151:6802/3763105575] conn(0x7f1470071740 0x7f1470073b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f450bfff640 1 -- 172.21.15.22:0/3054928051 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f4514017070 con 0x7f4518053c20 2024-03-29T04:17:56.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f450bfff640 1 --2- 172.21.15.22:0/3054928051 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4504068010 0x7f450406a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f450bfff640 1 -- 172.21.15.22:0/3054928051 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f45140865d0 con 0x7f4518053c20 2024-03-29T04:17:56.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f450bfff640 1 --2- 172.21.15.22:0/3054928051 >> [v2:172.21.15.151:6801/2961229122,v1:172.21.15.151:6803/2961229122] conn(0x7f4504071790 0x7f4504073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f450bfff640 1 -- 172.21.15.22:0/3054928051 --> [v2:172.21.15.151:6801/2961229122,v1:172.21.15.151:6803/2961229122] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4514017e70 con 0x7f4504071790 2024-03-29T04:17:56.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f450bfff640 1 -- 172.21.15.22:0/3054928051 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4514086ab0 con 0x7f4518053c20 2024-03-29T04:17:56.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f781bfff640 1 -- 172.21.15.22:0/2306056055 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f782c0c8480 con 0x7f7834157f40 2024-03-29T04:17:56.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f451effd640 1 --2- 172.21.15.22:0/3054928051 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4504068010 0x7f450406a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f8666ffd640 1 -- 172.21.15.22:0/436240616 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f86680c8420 con 0x7f8670038500 2024-03-29T04:17:56.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f8666ffd640 1 --2- 172.21.15.22:0/436240616 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8658067ff0 0x7f865806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f451ffff640 1 --2- 172.21.15.22:0/3054928051 >> [v2:172.21.15.151:6801/2961229122,v1:172.21.15.151:6803/2961229122] conn(0x7f4504071790 0x7f4504073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f148f7fe640 1 --2- 172.21.15.22:0/2102162361 >> [v2:172.21.15.151:6800/3763105575,v1:172.21.15.151:6802/3763105575] conn(0x7f1470071740 0x7f1470073b80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f781bfff640 1 --2- 172.21.15.22:0/2306056055 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7820068090 0x7f782006a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f8666ffd640 1 -- 172.21.15.22:0/436240616 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8668120c00 con 0x7f8670038500 2024-03-29T04:17:56.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f8675d9b640 1 --2- 172.21.15.22:0/436240616 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8658067ff0 0x7f865806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f0fbaffd640 1 -- 172.21.15.22:0/1804424869 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f0fc001b070 con 0x7f0fbc054230 2024-03-29T04:17:56.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f0fbaffd640 1 --2- 172.21.15.22:0/1804424869 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0fa8067ff0 0x7f0fa806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f0fbaffd640 1 -- 172.21.15.22:0/1804424869 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0fc00871d0 con 0x7f0fbc054230 2024-03-29T04:17:56.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f0fbaffd640 1 --2- 172.21.15.22:0/1804424869 >> [v2:172.21.15.151:6824/2519745336,v1:172.21.15.151:6825/2519745336] conn(0x7f0fa80716e0 0x7f0fa8073b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f0fbaffd640 1 -- 172.21.15.22:0/1804424869 --> [v2:172.21.15.151:6824/2519745336,v1:172.21.15.151:6825/2519745336] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0fc0001d40 con 0x7f0fa80716e0 2024-03-29T04:17:56.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f0fbaffd640 1 -- 172.21.15.22:0/1804424869 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0fc0087580 con 0x7f0fbc054230 2024-03-29T04:17:56.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f0fc99f8640 1 --2- 172.21.15.22:0/1804424869 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0fa8067ff0 0x7f0fa806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f8666ffd640 1 --2- 172.21.15.22:0/436240616 >> [v2:172.21.15.22:6809/2216395437,v1:172.21.15.22:6811/2216395437] conn(0x7f8658071770 0x7f8658073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f78337fe640 1 --2- 172.21.15.22:0/2306056055 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7820068090 0x7f782006a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f781bfff640 1 -- 172.21.15.22:0/2306056055 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f782c1216f0 con 0x7f7834157f40 2024-03-29T04:17:56.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f451effd640 1 --2- 172.21.15.22:0/3054928051 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4504068010 0x7f450406a4d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4518058250 tx=0x7f4520122070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f1487fff640 1 --2- 172.21.15.22:0/2102162361 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1470067fc0 0x7f147006a480 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f14901f5d40 tx=0x7f1480009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f0fca9fa640 1 --2- 172.21.15.22:0/1804424869 >> [v2:172.21.15.151:6824/2519745336,v1:172.21.15.151:6825/2519745336] conn(0x7f0fa80716e0 0x7f0fa8073b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f451ffff640 1 --2- 172.21.15.22:0/3054928051 >> [v2:172.21.15.151:6801/2961229122,v1:172.21.15.151:6803/2961229122] conn(0x7f4504071790 0x7f4504073bd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f781bfff640 1 --2- 172.21.15.22:0/2306056055 >> [v2:172.21.15.151:6816/2643908371,v1:172.21.15.151:6817/2643908371] conn(0x7f7820071810 0x7f7820073c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f781bfff640 1 -- 172.21.15.22:0/2306056055 --> [v2:172.21.15.151:6816/2643908371,v1:172.21.15.151:6817/2643908371] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f782c00df90 con 0x7f7820071810 2024-03-29T04:17:56.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f781bfff640 1 -- 172.21.15.22:0/2306056055 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f782c121ae0 con 0x7f7834157f40 2024-03-29T04:17:56.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f7833fff640 1 --2- 172.21.15.22:0/2306056055 >> [v2:172.21.15.151:6816/2643908371,v1:172.21.15.151:6817/2643908371] conn(0x7f7820071810 0x7f7820073c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f0fca9fa640 1 --2- 172.21.15.22:0/1804424869 >> [v2:172.21.15.151:6824/2519745336,v1:172.21.15.151:6825/2519745336] conn(0x7f0fa80716e0 0x7f0fa8073b20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f78337fe640 1 --2- 172.21.15.22:0/2306056055 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7820068090 0x7f782006a550 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7834074a70 tx=0x7f7824008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f8676d9d640 1 --2- 172.21.15.22:0/436240616 >> [v2:172.21.15.22:6809/2216395437,v1:172.21.15.22:6811/2216395437] conn(0x7f8658071770 0x7f8658073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f8676d9d640 1 --2- 172.21.15.22:0/436240616 >> [v2:172.21.15.22:6809/2216395437,v1:172.21.15.22:6811/2216395437] conn(0x7f8658071770 0x7f8658073bb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f8666ffd640 1 -- 172.21.15.22:0/436240616 --> [v2:172.21.15.22:6809/2216395437,v1:172.21.15.22:6811/2216395437] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8668001d40 con 0x7f8658071770 2024-03-29T04:17:56.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f0fc99f8640 1 --2- 172.21.15.22:0/1804424869 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0fa8067ff0 0x7f0fa806a4b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc113280 tx=0x7f0fb4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f8675d9b640 1 --2- 172.21.15.22:0/436240616 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8658067ff0 0x7f865806a4b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8670039c20 tx=0x7f866c00a2a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f148cff9640 1 -- 172.21.15.22:0/2102162361 <== osd.5 v2:172.21.15.151:6800/3763105575 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1478010870 con 0x7f1470071740 2024-03-29T04:17:56.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f8666ffd640 1 -- 172.21.15.22:0/436240616 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8668120ff0 con 0x7f8670038500 2024-03-29T04:17:56.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f507985e640 1 --2- 172.21.15.22:0/2801705020 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f507415b020 0x7f507406f9e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f506c00bef0 tx=0x7f506c01f2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c92407fd94846696 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f45566fc640 1 -- 172.21.15.22:0/2497696221 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f454804fa80 msgr2=0x7f454804fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f45566fc640 1 --2- 172.21.15.22:0/2497696221 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f454804fa80 0x7f454804fe60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f454c0090e0 tx=0x7f454c01d710 comp rx=0 tx=0).stop 2024-03-29T04:17:56.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f45566fc640 1 -- 172.21.15.22:0/2497696221 shutdown_connections 2024-03-29T04:17:56.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f45566fc640 1 --2- 172.21.15.22:0/2497696221 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f454804fa80 0x7f454804fe60 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f454c0090e0 tx=0x7f454c01d710 comp rx=0 tx=0).stop 2024-03-29T04:17:56.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.468+0000 7f45566fc640 1 -- 172.21.15.22:0/2497696221 >> 172.21.15.22:0/2497696221 conn(0x7f4548009fc0 msgr2=0x7f454800a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f507885c640 1 -- 172.21.15.22:0/2801705020 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c0b4070 con 0x7f507415b020 2024-03-29T04:17:56.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f7833fff640 1 --2- 172.21.15.22:0/2306056055 >> [v2:172.21.15.151:6816/2643908371,v1:172.21.15.151:6817/2643908371] conn(0x7f7820071810 0x7f7820073c50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f507885c640 1 -- 172.21.15.22:0/2801705020 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f506c00b2d0 con 0x7f507415b020 2024-03-29T04:17:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f450bfff640 1 -- 172.21.15.22:0/3054928051 <== osd.7 v2:172.21.15.151:6801/2961229122 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4514017e70 con 0x7f4504071790 2024-03-29T04:17:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f507bac2640 1 -- 172.21.15.22:0/2801705020 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f507415b020 msgr2=0x7f507406f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f507885c640 1 -- 172.21.15.22:0/2801705020 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c0b4070 con 0x7f507415b020 2024-03-29T04:17:56.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f781bfff640 1 -- 172.21.15.22:0/2306056055 <== osd.4 v2:172.21.15.151:6816/2643908371 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f782c00df90 con 0x7f7820071810 2024-03-29T04:17:56.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f507bac2640 1 --2- 172.21.15.22:0/2801705020 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f507415b020 0x7f507406f9e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f506c00bef0 tx=0x7f506c01f2c0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f507bac2640 1 -- 172.21.15.22:0/2801705020 shutdown_connections 2024-03-29T04:17:56.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f507bac2640 1 --2- 172.21.15.22:0/2801705020 >> v2:172.21.15.151:3300/0 conn(0x7f50740b5060 0x7f50740cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f507bac2640 1 --2- 172.21.15.22:0/2801705020 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f507415b020 0x7f507406f9e0 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f506c00bef0 tx=0x7f506c01f2c0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f507bac2640 1 -- 172.21.15.22:0/2801705020 >> 172.21.15.22:0/2801705020 conn(0x7f507406e3b0 msgr2=0x7f507406e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f507bac2640 1 -- 172.21.15.22:0/2801705020 shutdown_connections 2024-03-29T04:17:56.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f507bac2640 1 -- 172.21.15.22:0/2801705020 wait complete. 2024-03-29T04:17:56.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f507bac2640 1 Processor -- start 2024-03-29T04:17:56.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f0fbaffd640 1 -- 172.21.15.22:0/1804424869 <== osd.6 v2:172.21.15.151:6824/2519745336 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0fc0001d40 con 0x7f0fa80716e0 2024-03-29T04:17:56.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.476+0000 7f507bac2640 1 -- start start 2024-03-29T04:17:56.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.476+0000 7f507bac2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f507406ff20 0x7f507414f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.476+0000 7f507bac2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50740b5060 0x7f5074150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.476+0000 7f507bac2640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5074150090 0x7f5074152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.476+0000 7f507bac2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f50740ce7b0 con 0x7f50740b5060 2024-03-29T04:17:56.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.476+0000 7f507bac2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f50740ce630 con 0x7f507406ff20 2024-03-29T04:17:56.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.476+0000 7f507bac2640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f507415b400 con 0x7f5074150090 2024-03-29T04:17:56.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.476+0000 7f507905d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50740b5060 0x7f5074150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.476+0000 7f507905d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50740b5060 0x7f5074150c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51414/0 (socket says 172.21.15.22:51414) 2024-03-29T04:17:56.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.476+0000 7f507905d640 1 -- 172.21.15.22:0/3863129983 learned_addr learned my addr 172.21.15.22:0/3863129983 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.476+0000 7f507a05f640 1 --2- 172.21.15.22:0/3863129983 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5074150090 0x7f5074152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.476+0000 7f507905d640 1 -- 172.21.15.22:0/3863129983 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5074150090 msgr2=0x7f5074152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.476+0000 7f507905d640 1 --2- 172.21.15.22:0/3863129983 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5074150090 0x7f5074152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.476+0000 7f507905d640 1 -- 172.21.15.22:0/3863129983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f507406ff20 msgr2=0x7f507414f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.476+0000 7f507905d640 1 --2- 172.21.15.22:0/3863129983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f507406ff20 0x7f507414f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.476+0000 7f507905d640 1 -- 172.21.15.22:0/3863129983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50741515e0 con 0x7f50740b5060 2024-03-29T04:17:56.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.476+0000 7f507905d640 1 --2- 172.21.15.22:0/3863129983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50740b5060 0x7f5074150c70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f507000c500 tx=0x7f507000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.476+0000 7f8666ffd640 1 -- 172.21.15.22:0/436240616 <== osd.2 v2:172.21.15.22:6809/2216395437 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8668001d40 con 0x7f8658071770 2024-03-29T04:17:56.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.472+0000 7f45566fc640 1 -- 172.21.15.22:0/2497696221 shutdown_connections 2024-03-29T04:17:56.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.480+0000 7f506a7fc640 1 -- 172.21.15.22:0/3863129983 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070016020 con 0x7f50740b5060 2024-03-29T04:17:56.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.480+0000 7f506a7fc640 1 -- 172.21.15.22:0/3863129983 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5070005220 con 0x7f50740b5060 2024-03-29T04:17:56.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.480+0000 7f506a7fc640 1 -- 172.21.15.22:0/3863129983 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070007030 con 0x7f50740b5060 2024-03-29T04:17:56.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.480+0000 7f507bac2640 1 -- 172.21.15.22:0/3863129983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50741518c0 con 0x7f50740b5060 2024-03-29T04:17:56.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.480+0000 7f507bac2640 1 -- 172.21.15.22:0/3863129983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50741547c0 con 0x7f50740b5060 2024-03-29T04:17:56.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.480+0000 7f45566fc640 1 -- 172.21.15.22:0/2497696221 wait complete. 2024-03-29T04:17:56.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.480+0000 7f45566fc640 1 Processor -- start 2024-03-29T04:17:56.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.480+0000 7f507bac2640 1 -- 172.21.15.22:0/3863129983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5074154ca0 con 0x7f50740b5060 2024-03-29T04:17:56.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.480+0000 7f45566fc640 1 -- start start 2024-03-29T04:17:56.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.480+0000 7f45566fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f454804f5c0 0x7f45480cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.480+0000 7f45566fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45480df390 0x7f45480cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.480+0000 7f45566fc640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f45480cfee0 0x7f45480d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.480+0000 7f45566fc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f45480388b0 con 0x7f454804f5c0 2024-03-29T04:17:56.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.480+0000 7f45566fc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4548038730 con 0x7f45480df390 2024-03-29T04:17:56.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.480+0000 7f45566fc640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4548038a30 con 0x7f45480cfee0 2024-03-29T04:17:56.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.480+0000 7f4554ef9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45480df390 0x7f45480cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.480+0000 7f4554ef9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45480df390 0x7f45480cf9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:46440/0 (socket says 172.21.15.22:46440) 2024-03-29T04:17:56.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.480+0000 7f4554ef9640 1 -- 172.21.15.22:0/505834212 learned_addr learned my addr 172.21.15.22:0/505834212 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f45556fa640 1 --2- 172.21.15.22:0/505834212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f454804f5c0 0x7f45480cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f4554ef9640 1 -- 172.21.15.22:0/505834212 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f45480cfee0 msgr2=0x7f45480d4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f4554ef9640 1 --2- 172.21.15.22:0/505834212 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f45480cfee0 0x7f45480d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f4554ef9640 1 -- 172.21.15.22:0/505834212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f454804f5c0 msgr2=0x7f45480cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.480+0000 7f506a7fc640 1 -- 172.21.15.22:0/3863129983 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f50700071d0 con 0x7f50740b5060 2024-03-29T04:17:56.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f506a7fc640 1 --2- 172.21.15.22:0/3863129983 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5058067fc0 0x7f505806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f506a7fc640 1 -- 172.21.15.22:0/3863129983 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5070085c50 con 0x7f50740b5060 2024-03-29T04:17:56.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f506a7fc640 1 --2- 172.21.15.22:0/3863129983 >> [v2:172.21.15.22:6808/2469357214,v1:172.21.15.22:6810/2469357214] conn(0x7f50580716c0 0x7f5058073b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f506a7fc640 1 -- 172.21.15.22:0/3863129983 --> [v2:172.21.15.22:6808/2469357214,v1:172.21.15.22:6810/2469357214] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5070015e50 con 0x7f50580716c0 2024-03-29T04:17:56.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f506a7fc640 1 -- 172.21.15.22:0/3863129983 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5070086010 con 0x7f50740b5060 2024-03-29T04:17:56.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f4554ef9640 1 --2- 172.21.15.22:0/505834212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f454804f5c0 0x7f45480cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f4554ef9640 1 -- 172.21.15.22:0/505834212 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45480d0670 con 0x7f45480df390 2024-03-29T04:17:56.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f45556fa640 1 --2- 172.21.15.22:0/505834212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f454804f5c0 0x7f45480cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:56.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f507a05f640 1 --2- 172.21.15.22:0/3863129983 >> [v2:172.21.15.22:6808/2469357214,v1:172.21.15.22:6810/2469357214] conn(0x7f50580716c0 0x7f5058073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f4554ef9640 1 --2- 172.21.15.22:0/505834212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45480df390 0x7f45480cf9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f454000c4d0 tx=0x7f454000c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f507a05f640 1 --2- 172.21.15.22:0/3863129983 >> [v2:172.21.15.22:6808/2469357214,v1:172.21.15.22:6810/2469357214] conn(0x7f50580716c0 0x7f5058073b00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f61695aa640 1 -- 172.21.15.22:0/1549931682 --> [v2:172.21.15.22:6800/3551960158,v1:172.21.15.22:6801/3551960158] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6144002de0 con 0x7f6150071630 2024-03-29T04:17:56.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f4545ffb640 1 -- 172.21.15.22:0/505834212 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4540017070 con 0x7f45480df390 2024-03-29T04:17:56.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f506a7fc640 1 -- 172.21.15.22:0/3863129983 <== osd.0 v2:172.21.15.22:6808/2469357214 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5070015e50 con 0x7f50580716c0 2024-03-29T04:17:56.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f614b7fe640 1 -- 172.21.15.22:0/1549931682 <== osd.1 v2:172.21.15.22:6800/3551960158 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6144002de0 con 0x7f6150071630 2024-03-29T04:17:56.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f45566fc640 1 -- 172.21.15.22:0/505834212 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45480d0950 con 0x7f45480df390 2024-03-29T04:17:56.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f45566fc640 1 -- 172.21.15.22:0/505834212 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45480d0f10 con 0x7f45480df390 2024-03-29T04:17:56.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f45566fc640 1 -- 172.21.15.22:0/505834212 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f45480053a0 con 0x7f45480df390 2024-03-29T04:17:56.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f4545ffb640 1 -- 172.21.15.22:0/505834212 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4540005cf0 con 0x7f45480df390 2024-03-29T04:17:56.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f61695aa640 1 -- 172.21.15.22:0/1549931682 >> [v2:172.21.15.22:6800/3551960158,v1:172.21.15.22:6801/3551960158] conn(0x7f6150071630 msgr2=0x7f6150073a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.484+0000 7f61695aa640 1 --2- 172.21.15.22:0/1549931682 >> [v2:172.21.15.22:6800/3551960158,v1:172.21.15.22:6801/3551960158] conn(0x7f6150071630 0x7f6150073a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.488+0000 7f4545ffb640 1 -- 172.21.15.22:0/505834212 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4540007270 con 0x7f45480df390 2024-03-29T04:17:56.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.488+0000 7f4545ffb640 1 -- 172.21.15.22:0/505834212 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f454001a070 con 0x7f45480df390 2024-03-29T04:17:56.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.488+0000 7f4545ffb640 1 --2- 172.21.15.22:0/505834212 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4534068010 0x7f453406a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.488+0000 7f4545ffb640 1 -- 172.21.15.22:0/505834212 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f45400865a0 con 0x7f45480df390 2024-03-29T04:17:56.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.488+0000 7f4545ffb640 1 --2- 172.21.15.22:0/505834212 >> [v2:172.21.15.22:6813/583588642,v1:172.21.15.22:6816/583588642] conn(0x7f4534071780 0x7f4534073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.488+0000 7f45556fa640 1 --2- 172.21.15.22:0/505834212 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4534068010 0x7f453406a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.488+0000 7f507985e640 1 --2- 172.21.15.22:0/3863129983 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5058067fc0 0x7f505806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.488+0000 7f4545ffb640 1 -- 172.21.15.22:0/505834212 --> [v2:172.21.15.22:6813/583588642,v1:172.21.15.22:6816/583588642] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4534074290 con 0x7f4534071780 2024-03-29T04:17:56.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.488+0000 7f4545ffb640 1 -- 172.21.15.22:0/505834212 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4540086a00 con 0x7f45480df390 2024-03-29T04:17:56.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.488+0000 7f45556fa640 1 --2- 172.21.15.22:0/505834212 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4534068010 0x7f453406a4d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f454c0b0000 tx=0x7f454c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.488+0000 7f507985e640 1 --2- 172.21.15.22:0/3863129983 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5058067fc0 0x7f505806a480 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f506c0b1000 tx=0x7f506c004a50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.492+0000 7f4555efb640 1 --2- 172.21.15.22:0/505834212 >> [v2:172.21.15.22:6813/583588642,v1:172.21.15.22:6816/583588642] conn(0x7f4534071780 0x7f4534073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.492+0000 7f61695aa640 1 -- 172.21.15.22:0/1549931682 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6150067fc0 msgr2=0x7f615006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.492+0000 7f4555efb640 1 --2- 172.21.15.22:0/505834212 >> [v2:172.21.15.22:6813/583588642,v1:172.21.15.22:6816/583588642] conn(0x7f4534071780 0x7f4534073bc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.492+0000 7f61695aa640 1 --2- 172.21.15.22:0/1549931682 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6150067fc0 0x7f615006a480 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f615c184090 tx=0x7f6154007a50 comp rx=0 tx=0).stop 2024-03-29T04:17:56.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.492+0000 7f61695aa640 1 -- 172.21.15.22:0/1549931682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f615c0558a0 msgr2=0x7f615c0db710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.492+0000 7f61695aa640 1 --2- 172.21.15.22:0/1549931682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f615c0558a0 0x7f615c0db710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f616402d950 tx=0x7f61640949c0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.492+0000 7f61637fe640 1 -- 172.21.15.22:0/1549931682 reap_dead start 2024-03-29T04:17:56.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.492+0000 7f61695aa640 1 -- 172.21.15.22:0/1549931682 shutdown_connections 2024-03-29T04:17:56.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.492+0000 7f61695aa640 1 -- 172.21.15.22:0/1549931682 >> 172.21.15.22:0/1549931682 conn(0x7f615c020050 msgr2=0x7f615c00db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.492+0000 7f4545ffb640 1 -- 172.21.15.22:0/505834212 <== osd.3 v2:172.21.15.22:6813/583588642 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4534074290 con 0x7f4534071780 2024-03-29T04:17:56.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.496+0000 7f61695aa640 1 -- 172.21.15.22:0/1549931682 shutdown_connections 2024-03-29T04:17:56.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.496+0000 7f61695aa640 1 -- 172.21.15.22:0/1549931682 wait complete. 2024-03-29T04:17:56.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.496+0000 7f0fcc45d640 1 -- 172.21.15.22:0/1804424869 --> [v2:172.21.15.151:6824/2519745336,v1:172.21.15.151:6825/2519745336] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0fbc1a3a70 con 0x7f0fa80716e0 2024-03-29T04:17:56.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.496+0000 7f0fbaffd640 1 -- 172.21.15.22:0/1804424869 <== osd.6 v2:172.21.15.151:6824/2519745336 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0fbc1a3a70 con 0x7f0fa80716e0 2024-03-29T04:17:56.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.496+0000 7f0fcc45d640 1 -- 172.21.15.22:0/1804424869 >> [v2:172.21.15.151:6824/2519745336,v1:172.21.15.151:6825/2519745336] conn(0x7f0fa80716e0 msgr2=0x7f0fa8073b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.496+0000 7f0fcc45d640 1 --2- 172.21.15.22:0/1804424869 >> [v2:172.21.15.151:6824/2519745336,v1:172.21.15.151:6825/2519745336] conn(0x7f0fa80716e0 0x7f0fa8073b20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.496+0000 7f0fcc45d640 1 -- 172.21.15.22:0/1804424869 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0fa8067ff0 msgr2=0x7f0fa806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.496+0000 7f0fcc45d640 1 --2- 172.21.15.22:0/1804424869 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0fa8067ff0 0x7f0fa806a4b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc113280 tx=0x7f0fb4008040 comp rx=0 tx=0).stop 2024-03-29T04:17:56.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.496+0000 7f0fcc45d640 1 -- 172.21.15.22:0/1804424869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fbc054230 msgr2=0x7f0fbc1a32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.496+0000 7f0fcc45d640 1 --2- 172.21.15.22:0/1804424869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fbc054230 0x7f0fbc1a32b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000d800 tx=0x7f0fc000dcc0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.496+0000 7f0fca9fa640 1 -- 172.21.15.22:0/1804424869 reap_dead start 2024-03-29T04:17:56.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.500+0000 7f0fcc45d640 1 -- 172.21.15.22:0/1804424869 shutdown_connections 2024-03-29T04:17:56.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.500+0000 7f0fcc45d640 1 -- 172.21.15.22:0/1804424869 >> 172.21.15.22:0/1804424869 conn(0x7f0fbc009fc0 msgr2=0x7f0fbc0faeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.500+0000 7f0fcc45d640 1 -- 172.21.15.22:0/1804424869 shutdown_connections 2024-03-29T04:17:56.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.500+0000 7f0fcc45d640 1 -- 172.21.15.22:0/1804424869 wait complete. 2024-03-29T04:17:56.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.500+0000 7f7839e61640 1 -- 172.21.15.22:0/2306056055 --> [v2:172.21.15.151:6816/2643908371,v1:172.21.15.151:6817/2643908371] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f783416cc20 con 0x7f7820071810 2024-03-29T04:17:56.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.500+0000 7f781bfff640 1 -- 172.21.15.22:0/2306056055 <== osd.4 v2:172.21.15.151:6816/2643908371 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f783416cc20 con 0x7f7820071810 2024-03-29T04:17:56.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f4525a90640 1 -- 172.21.15.22:0/3054928051 --> [v2:172.21.15.151:6801/2961229122,v1:172.21.15.151:6803/2961229122] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4500002d60 con 0x7f4504071790 2024-03-29T04:17:56.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f450bfff640 1 -- 172.21.15.22:0/3054928051 <== osd.7 v2:172.21.15.151:6801/2961229122 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4500002d60 con 0x7f4504071790 2024-03-29T04:17:56.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f4525a90640 1 -- 172.21.15.22:0/3054928051 >> [v2:172.21.15.151:6801/2961229122,v1:172.21.15.151:6803/2961229122] conn(0x7f4504071790 msgr2=0x7f4504073bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f4525a90640 1 --2- 172.21.15.22:0/3054928051 >> [v2:172.21.15.151:6801/2961229122,v1:172.21.15.151:6803/2961229122] conn(0x7f4504071790 0x7f4504073bd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f4525a90640 1 -- 172.21.15.22:0/3054928051 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4504068010 msgr2=0x7f450406a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f4525a90640 1 --2- 172.21.15.22:0/3054928051 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4504068010 0x7f450406a4d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4518058250 tx=0x7f4520122070 comp rx=0 tx=0).stop 2024-03-29T04:17:56.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f4525a90640 1 -- 172.21.15.22:0/3054928051 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4518053c20 msgr2=0x7f45180e1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f4525a90640 1 --2- 172.21.15.22:0/3054928051 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4518053c20 0x7f45180e1740 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f451400b550 tx=0x7f451400ba10 comp rx=0 tx=0).stop 2024-03-29T04:17:56.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f7839e61640 1 -- 172.21.15.22:0/2306056055 >> [v2:172.21.15.151:6816/2643908371,v1:172.21.15.151:6817/2643908371] conn(0x7f7820071810 msgr2=0x7f7820073c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f7839e61640 1 --2- 172.21.15.22:0/2306056055 >> [v2:172.21.15.151:6816/2643908371,v1:172.21.15.151:6817/2643908371] conn(0x7f7820071810 0x7f7820073c50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f451ffff640 1 -- 172.21.15.22:0/3054928051 reap_dead start 2024-03-29T04:17:56.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f4525a90640 1 -- 172.21.15.22:0/3054928051 shutdown_connections 2024-03-29T04:17:56.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f4525a90640 1 -- 172.21.15.22:0/3054928051 >> 172.21.15.22:0/3054928051 conn(0x7f45180559f0 msgr2=0x7f4518039eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f7839e61640 1 -- 172.21.15.22:0/2306056055 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7820068090 msgr2=0x7f782006a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f7839e61640 1 --2- 172.21.15.22:0/2306056055 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7820068090 0x7f782006a550 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7834074a70 tx=0x7f7824008040 comp rx=0 tx=0).stop 2024-03-29T04:17:56.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f7839e61640 1 -- 172.21.15.22:0/2306056055 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7834157f40 msgr2=0x7f78340afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f7839e61640 1 --2- 172.21.15.22:0/2306056055 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7834157f40 0x7f78340afed0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f782c01c710 tx=0x7f782c009750 comp rx=0 tx=0).stop 2024-03-29T04:17:56.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f4525a90640 1 -- 172.21.15.22:0/3054928051 shutdown_connections 2024-03-29T04:17:56.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f4525a90640 1 -- 172.21.15.22:0/3054928051 wait complete. 2024-03-29T04:17:56.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f7833fff640 1 -- 172.21.15.22:0/2306056055 reap_dead start 2024-03-29T04:17:56.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f7839e61640 1 -- 172.21.15.22:0/2306056055 shutdown_connections 2024-03-29T04:17:56.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f7839e61640 1 -- 172.21.15.22:0/2306056055 >> 172.21.15.22:0/2306056055 conn(0x7f783406c570 msgr2=0x7f783416a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f7839e61640 1 -- 172.21.15.22:0/2306056055 shutdown_connections 2024-03-29T04:17:56.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.504+0000 7f7839e61640 1 -- 172.21.15.22:0/2306056055 wait complete. 2024-03-29T04:17:56.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.508+0000 7f867cbf7640 1 -- 172.21.15.22:0/436240616 --> [v2:172.21.15.22:6809/2216395437,v1:172.21.15.22:6811/2216395437] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8670008440 con 0x7f8658071770 2024-03-29T04:17:56.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.508+0000 7f148ffff640 1 -- 172.21.15.22:0/2102162361 --> [v2:172.21.15.151:6800/3763105575,v1:172.21.15.151:6802/3763105575] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1474002de0 con 0x7f1470071740 2024-03-29T04:17:56.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.508+0000 7f148cff9640 1 -- 172.21.15.22:0/2102162361 <== osd.5 v2:172.21.15.151:6800/3763105575 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1474002de0 con 0x7f1470071740 2024-03-29T04:17:56.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.508+0000 7f148ffff640 1 -- 172.21.15.22:0/2102162361 >> [v2:172.21.15.151:6800/3763105575,v1:172.21.15.151:6802/3763105575] conn(0x7f1470071740 msgr2=0x7f1470073b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.508+0000 7f148ffff640 1 --2- 172.21.15.22:0/2102162361 >> [v2:172.21.15.151:6800/3763105575,v1:172.21.15.151:6802/3763105575] conn(0x7f1470071740 0x7f1470073b80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.508+0000 7f148ffff640 1 -- 172.21.15.22:0/2102162361 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1470067fc0 msgr2=0x7f147006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.508+0000 7f148ffff640 1 --2- 172.21.15.22:0/2102162361 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1470067fc0 0x7f147006a480 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f14901f5d40 tx=0x7f1480009040 comp rx=0 tx=0).stop 2024-03-29T04:17:56.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.508+0000 7f148ffff640 1 -- 172.21.15.22:0/2102162361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14900cf5a0 msgr2=0x7f149015f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.508+0000 7f148ffff640 1 --2- 172.21.15.22:0/2102162361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14900cf5a0 0x7f149015f400 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f147800b550 tx=0x7f147800ba10 comp rx=0 tx=0).stop 2024-03-29T04:17:56.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.508+0000 7f148f7fe640 1 -- 172.21.15.22:0/2102162361 reap_dead start 2024-03-29T04:17:56.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.508+0000 7f148ffff640 1 -- 172.21.15.22:0/2102162361 shutdown_connections 2024-03-29T04:17:56.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.508+0000 7f148ffff640 1 -- 172.21.15.22:0/2102162361 >> 172.21.15.22:0/2102162361 conn(0x7f14900d40c0 msgr2=0x7f14900b5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.508+0000 7f148ffff640 1 -- 172.21.15.22:0/2102162361 shutdown_connections 2024-03-29T04:17:56.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.508+0000 7f148ffff640 1 -- 172.21.15.22:0/2102162361 wait complete. 2024-03-29T04:17:56.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.512+0000 7f45566fc640 1 -- 172.21.15.22:0/505834212 --> [v2:172.21.15.22:6813/583588642,v1:172.21.15.22:6816/583588642] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f454804f190 con 0x7f4534071780 2024-03-29T04:17:56.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.512+0000 7f4545ffb640 1 -- 172.21.15.22:0/505834212 <== osd.3 v2:172.21.15.22:6813/583588642 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f454804f190 con 0x7f4534071780 2024-03-29T04:17:56.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.516+0000 7f45566fc640 1 -- 172.21.15.22:0/505834212 >> [v2:172.21.15.22:6813/583588642,v1:172.21.15.22:6816/583588642] conn(0x7f4534071780 msgr2=0x7f4534073bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.516+0000 7f45566fc640 1 --2- 172.21.15.22:0/505834212 >> [v2:172.21.15.22:6813/583588642,v1:172.21.15.22:6816/583588642] conn(0x7f4534071780 0x7f4534073bc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.516+0000 7f45566fc640 1 -- 172.21.15.22:0/505834212 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4534068010 msgr2=0x7f453406a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.516+0000 7f45566fc640 1 --2- 172.21.15.22:0/505834212 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4534068010 0x7f453406a4d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f454c0b0000 tx=0x7f454c008040 comp rx=0 tx=0).stop 2024-03-29T04:17:56.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.516+0000 7f45566fc640 1 -- 172.21.15.22:0/505834212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45480df390 msgr2=0x7f45480cf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.516+0000 7f45566fc640 1 --2- 172.21.15.22:0/505834212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45480df390 0x7f45480cf9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f454000c4d0 tx=0x7f454000c990 comp rx=0 tx=0).stop 2024-03-29T04:17:56.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.516+0000 7f4555efb640 1 -- 172.21.15.22:0/505834212 reap_dead start 2024-03-29T04:17:56.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.516+0000 7f45566fc640 1 -- 172.21.15.22:0/505834212 shutdown_connections 2024-03-29T04:17:56.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.516+0000 7f45566fc640 1 -- 172.21.15.22:0/505834212 >> 172.21.15.22:0/505834212 conn(0x7f4548009fc0 msgr2=0x7f4548048600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.516+0000 7f45566fc640 1 -- 172.21.15.22:0/505834212 shutdown_connections 2024-03-29T04:17:56.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.516+0000 7f45566fc640 1 -- 172.21.15.22:0/505834212 wait complete. 2024-03-29T04:17:56.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.516+0000 7f507bac2640 1 -- 172.21.15.22:0/3863129983 --> [v2:172.21.15.22:6808/2469357214,v1:172.21.15.22:6810/2469357214] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5074068890 con 0x7f50580716c0 2024-03-29T04:17:56.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.520+0000 7f506a7fc640 1 -- 172.21.15.22:0/3863129983 <== osd.0 v2:172.21.15.22:6808/2469357214 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5074068890 con 0x7f50580716c0 2024-03-29T04:17:56.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.520+0000 7f507bac2640 1 -- 172.21.15.22:0/3863129983 >> [v2:172.21.15.22:6808/2469357214,v1:172.21.15.22:6810/2469357214] conn(0x7f50580716c0 msgr2=0x7f5058073b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.520+0000 7f507bac2640 1 --2- 172.21.15.22:0/3863129983 >> [v2:172.21.15.22:6808/2469357214,v1:172.21.15.22:6810/2469357214] conn(0x7f50580716c0 0x7f5058073b00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.520+0000 7f507bac2640 1 -- 172.21.15.22:0/3863129983 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5058067fc0 msgr2=0x7f505806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.520+0000 7f507bac2640 1 --2- 172.21.15.22:0/3863129983 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5058067fc0 0x7f505806a480 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f506c0b1000 tx=0x7f506c004a50 comp rx=0 tx=0).stop 2024-03-29T04:17:56.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.520+0000 7f507bac2640 1 -- 172.21.15.22:0/3863129983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50740b5060 msgr2=0x7f5074150c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.520+0000 7f507bac2640 1 --2- 172.21.15.22:0/3863129983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50740b5060 0x7f5074150c70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f507000c500 tx=0x7f507000c9c0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.520+0000 7f507a05f640 1 -- 172.21.15.22:0/3863129983 reap_dead start 2024-03-29T04:17:56.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.520+0000 7f507bac2640 1 -- 172.21.15.22:0/3863129983 shutdown_connections 2024-03-29T04:17:56.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.520+0000 7f507bac2640 1 -- 172.21.15.22:0/3863129983 >> 172.21.15.22:0/3863129983 conn(0x7f507406e3b0 msgr2=0x7f5074156620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.520+0000 7f8666ffd640 1 -- 172.21.15.22:0/436240616 <== osd.2 v2:172.21.15.22:6809/2216395437 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8670008440 con 0x7f8658071770 2024-03-29T04:17:56.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.520+0000 7f507bac2640 1 -- 172.21.15.22:0/3863129983 shutdown_connections 2024-03-29T04:17:56.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.520+0000 7f507bac2640 1 -- 172.21.15.22:0/3863129983 wait complete. 2024-03-29T04:17:56.560 INFO:teuthology.orchestra.run.smithi022.stdout:55834574856 2024-03-29T04:17:56.561 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-29T04:17:56.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.524+0000 7f867cbf7640 1 -- 172.21.15.22:0/436240616 >> [v2:172.21.15.22:6809/2216395437,v1:172.21.15.22:6811/2216395437] conn(0x7f8658071770 msgr2=0x7f8658073bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.524+0000 7f867cbf7640 1 --2- 172.21.15.22:0/436240616 >> [v2:172.21.15.22:6809/2216395437,v1:172.21.15.22:6811/2216395437] conn(0x7f8658071770 0x7f8658073bb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.524+0000 7f867cbf7640 1 -- 172.21.15.22:0/436240616 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8658067ff0 msgr2=0x7f865806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.524+0000 7f867cbf7640 1 --2- 172.21.15.22:0/436240616 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8658067ff0 0x7f865806a4b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8670039c20 tx=0x7f866c00a2a0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.524+0000 7f867cbf7640 1 -- 172.21.15.22:0/436240616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8670038500 msgr2=0x7f86700d5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.524+0000 7f867cbf7640 1 --2- 172.21.15.22:0/436240616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8670038500 0x7f86700d5860 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f86680ad040 tx=0x7f866801ada0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.524+0000 7f8676d9d640 1 -- 172.21.15.22:0/436240616 reap_dead start 2024-03-29T04:17:56.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.524+0000 7f867cbf7640 1 -- 172.21.15.22:0/436240616 shutdown_connections 2024-03-29T04:17:56.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.524+0000 7f867cbf7640 1 -- 172.21.15.22:0/436240616 >> 172.21.15.22:0/436240616 conn(0x7f8670055360 msgr2=0x7f8670038df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.524+0000 7f867cbf7640 1 -- 172.21.15.22:0/436240616 shutdown_connections 2024-03-29T04:17:56.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.524+0000 7f867cbf7640 1 -- 172.21.15.22:0/436240616 wait complete. 2024-03-29T04:17:56.563 INFO:teuthology.orchestra.run.smithi022.stdout:55834574857 2024-03-29T04:17:56.563 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-29T04:17:56.564 INFO:teuthology.orchestra.run.smithi022.stdout:55834574857 2024-03-29T04:17:56.564 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-29T04:17:56.565 INFO:teuthology.orchestra.run.smithi022.stdout:55834574857 2024-03-29T04:17:56.565 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-29T04:17:56.566 INFO:teuthology.orchestra.run.smithi022.stdout:55834574856 2024-03-29T04:17:56.566 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-29T04:17:56.567 INFO:teuthology.orchestra.run.smithi022.stdout:55834574856 2024-03-29T04:17:56.567 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-29T04:17:56.568 INFO:teuthology.orchestra.run.smithi022.stdout:55834574856 2024-03-29T04:17:56.568 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-29T04:17:56.569 INFO:teuthology.orchestra.run.smithi022.stdout:55834574857 2024-03-29T04:17:56.569 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-29T04:17:56.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.780+0000 7f5837b5c640 1 Processor -- start 2024-03-29T04:17:56.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.780+0000 7f5837b5c640 1 -- start start 2024-03-29T04:17:56.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.780+0000 7f5837b5c640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f583015a2c0 0x7f583006f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.780+0000 7f5837b5c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f583015a6a0 0x7f5830072810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.780+0000 7f5837b5c640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58300b32f0 con 0x7f583015aac0 2024-03-29T04:17:56.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.780+0000 7f5837b5c640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f583006f9a0 con 0x7f583015a6a0 2024-03-29T04:17:56.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.780+0000 7f5837b5c640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f58300b3e70 con 0x7f583015a2c0 2024-03-29T04:17:56.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.784+0000 7f58350f7640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f583015a6a0 0x7f5830072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.784+0000 7f58350f7640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f583015a6a0 0x7f5830072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51424/0 (socket says 172.21.15.22:51424) 2024-03-29T04:17:56.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.784+0000 7f58350f7640 1 -- 172.21.15.22:0/2706241379 learned_addr learned my addr 172.21.15.22:0/2706241379 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.784+0000 7f58358f8640 1 --2- 172.21.15.22:0/2706241379 >> v2:172.21.15.151:3300/0 conn(0x7f583015a2c0 0x7f583006f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.784+0000 7f58350f7640 1 -- 172.21.15.22:0/2706241379 >> v2:172.21.15.151:3300/0 conn(0x7f583015a2c0 msgr2=0x7f583006f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.784+0000 7f58350f7640 1 --2- 172.21.15.22:0/2706241379 >> v2:172.21.15.151:3300/0 conn(0x7f583015a2c0 0x7f583006f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.784+0000 7f58350f7640 1 -- 172.21.15.22:0/2706241379 >> v1:172.21.15.151:6789/0 conn(0x7f583015aac0 legacy=0x7f583006d9e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:17:56.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.784+0000 7f58350f7640 1 -- 172.21.15.22:0/2706241379 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58300b35d0 con 0x7f583015a6a0 2024-03-29T04:17:56.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.788+0000 7f58350f7640 1 --2- 172.21.15.22:0/2706241379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f583015a6a0 0x7f5830072810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f58300b6140 tx=0x7f582801ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad20b0f6c6040987 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.788+0000 7f58348f6640 1 -- 172.21.15.22:0/2706241379 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58280ad070 con 0x7f583015a6a0 2024-03-29T04:17:56.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.788+0000 7f58348f6640 1 -- 172.21.15.22:0/2706241379 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58280a7d80 con 0x7f583015a6a0 2024-03-29T04:17:56.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.788+0000 7f58348f6640 1 -- 172.21.15.22:0/2706241379 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582800be60 con 0x7f583015a6a0 2024-03-29T04:17:56.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.788+0000 7f5837b5c640 1 -- 172.21.15.22:0/2706241379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f583015a6a0 msgr2=0x7f5830072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.788+0000 7f5837b5c640 1 --2- 172.21.15.22:0/2706241379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f583015a6a0 0x7f5830072810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f58300b6140 tx=0x7f582801ecb0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.788+0000 7f5837b5c640 1 -- 172.21.15.22:0/2706241379 shutdown_connections 2024-03-29T04:17:56.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.788+0000 7f5837b5c640 1 --2- 172.21.15.22:0/2706241379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f583015a6a0 0x7f5830072810 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.788+0000 7f5837b5c640 1 --2- 172.21.15.22:0/2706241379 >> v2:172.21.15.151:3300/0 conn(0x7f583015a2c0 0x7f583006f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.788+0000 7f5837b5c640 1 -- 172.21.15.22:0/2706241379 >> 172.21.15.22:0/2706241379 conn(0x7f583006c570 msgr2=0x7f583006c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.788+0000 7f5837b5c640 1 -- 172.21.15.22:0/2706241379 shutdown_connections 2024-03-29T04:17:56.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.788+0000 7f5837b5c640 1 -- 172.21.15.22:0/2706241379 wait complete. 2024-03-29T04:17:56.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.792+0000 7f5837b5c640 1 Processor -- start 2024-03-29T04:17:56.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.792+0000 7f5837b5c640 1 -- start start 2024-03-29T04:17:56.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.792+0000 7f5837b5c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f583015a2c0 0x7f583015dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.792+0000 7f5837b5c640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f583015a6a0 0x7f583014e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.792+0000 7f5837b5c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f583015aac0 0x7f5830151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.792+0000 7f58358f8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f583015a2c0 0x7f583015dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.792+0000 7f58358f8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f583015a2c0 0x7f583015dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51426/0 (socket says 172.21.15.22:51426) 2024-03-29T04:17:56.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.792+0000 7f58350f7640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f583015a6a0 0x7f583014e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.792+0000 7f58350f7640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f583015a6a0 0x7f583014e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.22:52938/0 (socket says 172.21.15.22:52938) 2024-03-29T04:17:56.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.792+0000 7f58358f8640 1 -- 172.21.15.22:0/3876567666 learned_addr learned my addr 172.21.15.22:0/3876567666 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.792+0000 7f58360f9640 1 --2- 172.21.15.22:0/3876567666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f583015aac0 0x7f5830151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.792+0000 7f5837b5c640 1 -- 172.21.15.22:0/3876567666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f58300b2e80 con 0x7f583015a2c0 2024-03-29T04:17:56.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.792+0000 7f5837b5c640 1 -- 172.21.15.22:0/3876567666 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5830073160 con 0x7f583015aac0 2024-03-29T04:17:56.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.792+0000 7f5837b5c640 1 -- 172.21.15.22:0/3876567666 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f58300b3000 con 0x7f583015a6a0 2024-03-29T04:17:56.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.792+0000 7f58350f7640 1 -- 172.21.15.22:0/3876567666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f583015aac0 msgr2=0x7f5830151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.792+0000 7f58350f7640 1 --2- 172.21.15.22:0/3876567666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f583015aac0 0x7f5830151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.792+0000 7f58350f7640 1 -- 172.21.15.22:0/3876567666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f583015a2c0 msgr2=0x7f583015dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.792+0000 7f58350f7640 1 --2- 172.21.15.22:0/3876567666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f583015a2c0 0x7f583015dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.792+0000 7f58350f7640 1 -- 172.21.15.22:0/3876567666 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583015d140 con 0x7f583015a6a0 2024-03-29T04:17:56.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.796+0000 7f58358f8640 1 --2- 172.21.15.22:0/3876567666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f583015a2c0 0x7f583015dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:17:56.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.796+0000 7f58350f7640 1 --2- 172.21.15.22:0/3876567666 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f583015a6a0 0x7f583014e1f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f58280af000 tx=0x7f582801ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.796+0000 7f58267fc640 1 -- 172.21.15.22:0/3876567666 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58280c0070 con 0x7f583015a6a0 2024-03-29T04:17:56.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.796+0000 7f58267fc640 1 -- 172.21.15.22:0/3876567666 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5828008040 con 0x7f583015a6a0 2024-03-29T04:17:56.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.796+0000 7f58267fc640 1 -- 172.21.15.22:0/3876567666 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58280ad040 con 0x7f583015a6a0 2024-03-29T04:17:56.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.796+0000 7f5837b5c640 1 -- 172.21.15.22:0/3876567666 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5830151fd0 con 0x7f583015a6a0 2024-03-29T04:17:56.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.796+0000 7f5837b5c640 1 -- 172.21.15.22:0/3876567666 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f583014ee20 con 0x7f583015a6a0 2024-03-29T04:17:56.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.796+0000 7f5837b5c640 1 -- 172.21.15.22:0/3876567666 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58300743d0 con 0x7f583015a6a0 2024-03-29T04:17:56.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.800+0000 7f58267fc640 1 -- 172.21.15.22:0/3876567666 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f58280083f0 con 0x7f583015a6a0 2024-03-29T04:17:56.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.800+0000 7f58267fc640 1 --2- 172.21.15.22:0/3876567666 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f581c068120 0x7f581c06a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.800+0000 7f58267fc640 1 -- 172.21.15.22:0/3876567666 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5828121360 con 0x7f583015a6a0 2024-03-29T04:17:56.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.800+0000 7f58358f8640 1 --2- 172.21.15.22:0/3876567666 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f581c068120 0x7f581c06a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.800+0000 7f58358f8640 1 --2- 172.21.15.22:0/3876567666 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f581c068120 0x7f581c06a5e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f582c00aee0 tx=0x7f582c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.800+0000 7fd06a29d640 1 Processor -- start 2024-03-29T04:17:56.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.800+0000 7f1d4b59e640 1 Processor -- start 2024-03-29T04:17:56.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.800+0000 7f1d4b59e640 1 -- start start 2024-03-29T04:17:56.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.800+0000 7fd06a29d640 1 -- start start 2024-03-29T04:17:56.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fd06a29d640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd06406e780 0x7fd0640b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fd06a29d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd0640b5970 0x7fd0640b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fd06a29d640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0640b4f70 con 0x7fd06415afb0 2024-03-29T04:17:56.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fd06a29d640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fd0640b62e0 con 0x7fd0640b5970 2024-03-29T04:17:56.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fd06a29d640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fd06406edd0 con 0x7fd06406e780 2024-03-29T04:17:56.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7f1d4a59c640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f1d440e1320 0x7f1d44052710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.22:60080/0 (socket says 172.21.15.22:60080) 2024-03-29T04:17:56.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fd068a9a640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd06406e780 0x7fd0640b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fd068a9a640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd06406e780 0x7fd0640b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:46462/0 (socket says 172.21.15.22:46462) 2024-03-29T04:17:56.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fd068a9a640 1 -- 172.21.15.22:0/962707877 learned_addr learned my addr 172.21.15.22:0/962707877 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fd069a9c640 1 --2- 172.21.15.22:0/962707877 >> v2:172.21.15.22:3300/0 conn(0x7fd0640b5970 0x7fd0640b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7f1d49d9b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1d44054b60 0x7f1d44055360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44204/0 (socket says 172.21.15.22:44204) 2024-03-29T04:17:56.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7f1d4a59c640 1 -- 172.21.15.22:0/689763597 learned_addr learned my addr 172.21.15.22:0/689763597 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:56.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7f1d4b59e640 1 --2- 172.21.15.22:0/689763597 >> v2:172.21.15.22:3300/0 conn(0x7f1d44055a70 0x7f1d440366b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fd068a9a640 1 -- 172.21.15.22:0/962707877 >> v2:172.21.15.22:3300/0 conn(0x7fd0640b5970 msgr2=0x7fd0640b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fd068a9a640 1 --2- 172.21.15.22:0/962707877 >> v2:172.21.15.22:3300/0 conn(0x7fd0640b5970 0x7fd0640b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fd068a9a640 1 -- 172.21.15.22:0/962707877 >> v1:172.21.15.22:6789/0 conn(0x7fd06415afb0 legacy=0x7fd06406fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fd068a9a640 1 -- 172.21.15.22:0/962707877 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06406f0b0 con 0x7fd06406e780 2024-03-29T04:17:56.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fdb82f3a640 1 Processor -- start 2024-03-29T04:17:56.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fdb82f3a640 1 -- start start 2024-03-29T04:17:56.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7f1d4ad9d640 1 --2- 172.21.15.22:0/689763597 >> v2:172.21.15.22:3300/0 conn(0x7f1d44055a70 0x7f1d440366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7ff7036ab640 1 Processor -- start 2024-03-29T04:17:56.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7ff7036ab640 1 -- start start 2024-03-29T04:17:56.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fdb82f3a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fdb740df790 0x7fdb74055d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fdb82f3a640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fdb74036640 0x7fdb7403a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fdb82f3a640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb74036b20 con 0x7fdb7403a5f0 2024-03-29T04:17:56.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fdb81f38640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fdb740df790 0x7fdb74055d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fdb81f38640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fdb740df790 0x7fdb74055d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51452/0 (socket says 172.21.15.22:51452) 2024-03-29T04:17:56.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fdb81f38640 1 -- 172.21.15.22:0/2517810445 learned_addr learned my addr 172.21.15.22:0/2517810445 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.804+0000 7fdb82f3a640 1 -- 172.21.15.22:0/2517810445 --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fdb7403aa40 con 0x7fdb740df790 2024-03-29T04:17:56.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7fdb82f3a640 1 -- 172.21.15.22:0/2517810445 --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fdb74053900 con 0x7fdb74036640 2024-03-29T04:17:56.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7ff7036ab640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff6f40dfe40 0x7ff6f4054ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d4b59e640 1 -- 172.21.15.22:0/689763597 --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d440e5aa0 con 0x7f1d44054b60 2024-03-29T04:17:56.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d4b59e640 1 -- 172.21.15.22:0/689763597 --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d44051c30 con 0x7f1d440e1320 2024-03-29T04:17:56.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7fdb80f36640 1 -- 172.21.15.22:0/2517810445 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4192924746 0 0) 0x7fdb74036b20 con 0x7fdb7403a5f0 2024-03-29T04:17:56.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7fdb80f36640 1 -- 172.21.15.22:0/2517810445 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb68005740 con 0x7fdb7403a5f0 2024-03-29T04:17:56.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7fdb81737640 1 --2- 172.21.15.22:0/2517810445 >> v2:172.21.15.151:3300/0 conn(0x7fdb74036640 0x7fdb7403a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d4959a640 1 -- 172.21.15.22:0/689763597 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 564055328 0 0) 0x7f1d440e5aa0 con 0x7f1d44054b60 2024-03-29T04:17:56.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d4b59e640 1 -- 172.21.15.22:0/689763597 --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f1d44051f10 con 0x7f1d44055a70 2024-03-29T04:17:56.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7ff701447640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff6f40dfe40 0x7ff6f4054ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7ff701447640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff6f40dfe40 0x7ff6f4054ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:46478/0 (socket says 172.21.15.22:46478) 2024-03-29T04:17:56.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7fd068a9a640 1 --2- 172.21.15.22:0/962707877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd06406e780 0x7fd0640b4a30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd0640b0b90 tx=0x7fd06001c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d41cc8e80b25bf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7ff700c46640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff6f4038e90 0x7ff6f40e3ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44222/0 (socket says 172.21.15.22:44222) 2024-03-29T04:17:56.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d4959a640 1 -- 172.21.15.22:0/689763597 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d34003530 con 0x7f1d44054b60 2024-03-29T04:17:56.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7ff701447640 1 -- 172.21.15.22:0/940085191 learned_addr learned my addr 172.21.15.22:0/940085191 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7fdb81f38640 1 -- 172.21.15.22:0/2517810445 >> v2:172.21.15.151:3300/0 conn(0x7fdb74036640 msgr2=0x7fdb7403a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7fdb81f38640 1 --2- 172.21.15.22:0/2517810445 >> v2:172.21.15.151:3300/0 conn(0x7fdb74036640 0x7fdb7403a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7fdb81f38640 1 -- 172.21.15.22:0/2517810445 >> v1:172.21.15.22:6789/0 conn(0x7fdb7403a5f0 legacy=0x7fdb74006ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7fdb81f38640 1 -- 172.21.15.22:0/2517810445 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb74052fd0 con 0x7fdb740df790 2024-03-29T04:17:56.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7ff7036ab640 1 --2- 172.21.15.22:0/940085191 >> v2:172.21.15.22:3300/0 conn(0x7ff6f40e41e0 0x7ff6f40085a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7ff701c48640 1 --2- 172.21.15.22:0/940085191 >> v2:172.21.15.22:3300/0 conn(0x7ff6f40e41e0 0x7ff6f40085a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7ff7036ab640 1 -- 172.21.15.22:0/940085191 --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6f4055090 con 0x7ff6f4038e90 2024-03-29T04:17:56.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d4959a640 1 -- 172.21.15.22:0/689763597 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 601227382 0 0) 0x7f1d34003530 con 0x7f1d44054b60 2024-03-29T04:17:56.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d4959a640 1 -- 172.21.15.22:0/689763597 >> v2:172.21.15.22:3300/0 conn(0x7f1d44055a70 msgr2=0x7f1d440366b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d4959a640 1 --2- 172.21.15.22:0/689763597 >> v2:172.21.15.22:3300/0 conn(0x7f1d44055a70 0x7f1d440366b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d4959a640 1 -- 172.21.15.22:0/689763597 >> v1:172.21.15.151:6789/0 conn(0x7f1d440e1320 legacy=0x7f1d44052710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7ff7036ab640 1 -- 172.21.15.22:0/940085191 --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7ff6f4007b50 con 0x7ff6f40e41e0 2024-03-29T04:17:56.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d4959a640 1 -- 172.21.15.22:0/689763597 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d440521f0 con 0x7f1d44054b60 2024-03-29T04:17:56.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d4959a640 1 -- 172.21.15.22:0/689763597 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f1d40001680 con 0x7f1d44054b60 2024-03-29T04:17:56.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d4959a640 1 -- 172.21.15.22:0/689763597 >> v1:172.21.15.22:6789/0 conn(0x7f1d44054b60 legacy=0x7f1d44055360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d4959a640 1 --2- 172.21.15.22:0/689763597 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d340039c0 0x7f1d34003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d4959a640 1 --2- 172.21.15.22:0/689763597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d34004810 0x7f1d34004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d4959a640 1 --2- 172.21.15.22:0/689763597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d340051d0 0x7f1d340145d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d4959a640 1 -- 172.21.15.22:0/689763597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1d44051f10 con 0x7f1d34004810 2024-03-29T04:17:56.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d4959a640 1 -- 172.21.15.22:0/689763597 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d34009bf0 con 0x7f1d340051d0 2024-03-29T04:17:56.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7ff7036ab640 1 -- 172.21.15.22:0/940085191 --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7ff6f4055370 con 0x7ff6f40dfe40 2024-03-29T04:17:56.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d4959a640 1 -- 172.21.15.22:0/689763597 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1d34014b60 con 0x7f1d340039c0 2024-03-29T04:17:56.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f0bea228640 1 Processor -- start 2024-03-29T04:17:56.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f0bea228640 1 -- start start 2024-03-29T04:17:56.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d49d9b640 1 --2- 172.21.15.22:0/689763597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d34004810 0x7f1d34004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d4a59c640 1 --2- 172.21.15.22:0/689763597 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d340039c0 0x7f1d34003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7ff6f3fff640 1 -- 172.21.15.22:0/940085191 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 633569197 0 0) 0x7ff6f4055090 con 0x7ff6f4038e90 2024-03-29T04:17:56.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7ff6f3fff640 1 -- 172.21.15.22:0/940085191 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6e8003530 con 0x7ff6f4038e90 2024-03-29T04:17:56.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7fd05bfff640 1 -- 172.21.15.22:0/962707877 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0600b0070 con 0x7fd06406e780 2024-03-29T04:17:56.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7fd05bfff640 1 -- 172.21.15.22:0/962707877 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0600ab600 con 0x7fd06406e780 2024-03-29T04:17:56.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7fd05bfff640 1 -- 172.21.15.22:0/962707877 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0600ab9d0 con 0x7fd06406e780 2024-03-29T04:17:56.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7fd06a29d640 1 -- 172.21.15.22:0/962707877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd06406e780 msgr2=0x7fd0640b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7fd06a29d640 1 --2- 172.21.15.22:0/962707877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd06406e780 0x7fd0640b4a30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd0640b0b90 tx=0x7fd06001c490 comp rx=0 tx=0).stop 2024-03-29T04:17:56.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f0bea228640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0bdc0383b0 0x7f0bdc0f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f0bea228640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bdc0543d0 con 0x7f0bdc047d70 2024-03-29T04:17:56.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f0bea228640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bdc0546b0 con 0x7f0bdc0e7700 2024-03-29T04:17:56.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f0bea228640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f0bdc006df0 con 0x7f0bdc0383b0 2024-03-29T04:17:56.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7ff701447640 1 -- 172.21.15.22:0/940085191 >> v2:172.21.15.22:3300/0 conn(0x7ff6f40e41e0 msgr2=0x7ff6f40085a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7ff701447640 1 --2- 172.21.15.22:0/940085191 >> v2:172.21.15.22:3300/0 conn(0x7ff6f40e41e0 0x7ff6f40085a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7ff701447640 1 -- 172.21.15.22:0/940085191 >> v1:172.21.15.22:6789/0 conn(0x7ff6f4038e90 legacy=0x7ff6f40e3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7ff701447640 1 -- 172.21.15.22:0/940085191 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f4007da0 con 0x7ff6f40dfe40 2024-03-29T04:17:56.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f0be37fe640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f0bdc0e7700 0x7f0bdc007280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.22:60096/0 (socket says 172.21.15.22:60096) 2024-03-29T04:17:56.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f0be37fe640 1 -- 172.21.15.22:0/1956237362 learned_addr learned my addr 172.21.15.22:0/1956237362 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:56.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7fd06a29d640 1 -- 172.21.15.22:0/962707877 shutdown_connections 2024-03-29T04:17:56.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7fd06a29d640 1 --2- 172.21.15.22:0/962707877 >> v2:172.21.15.22:3300/0 conn(0x7fd0640b5970 0x7fd0640b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7fd06a29d640 1 --2- 172.21.15.22:0/962707877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd06406e780 0x7fd0640b4a30 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd0640b0b90 tx=0x7fd06001c490 comp rx=0 tx=0).stop 2024-03-29T04:17:56.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7fd06a29d640 1 -- 172.21.15.22:0/962707877 >> 172.21.15.22:0/962707877 conn(0x7fd064073440 msgr2=0x7fd06406c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d49d9b640 1 -- 172.21.15.22:0/689763597 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d340039c0 msgr2=0x7f1d34003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d49d9b640 1 --2- 172.21.15.22:0/689763597 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d340039c0 0x7f1d34003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d49d9b640 1 -- 172.21.15.22:0/689763597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d340051d0 msgr2=0x7f1d340145d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d49d9b640 1 --2- 172.21.15.22:0/689763597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d340051d0 0x7f1d340145d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d4ad9d640 1 -- 172.21.15.22:0/689763597 reap_dead start 2024-03-29T04:17:56.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.808+0000 7f1d49d9b640 1 -- 172.21.15.22:0/689763597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d440521f0 con 0x7f1d34004810 2024-03-29T04:17:56.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7f0be3fff640 1 --2- 172.21.15.22:0/1956237362 >> v2:172.21.15.22:3300/0 conn(0x7f0bdc0383b0 0x7f0bdc0f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7fd06a29d640 1 -- 172.21.15.22:0/962707877 shutdown_connections 2024-03-29T04:17:56.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7fdb81f38640 1 --2- 172.21.15.22:0/2517810445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb740df790 0x7fdb74055d20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb74056500 tx=0x7fdb7801c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c957c57e1d71ef70 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7f0be3fff640 1 -- 172.21.15.22:0/1956237362 >> v1:172.21.15.151:6789/0 conn(0x7f0bdc0e7700 legacy=0x7f0bdc007280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7f0be3fff640 1 -- 172.21.15.22:0/1956237362 >> v1:172.21.15.22:6789/0 conn(0x7f0bdc047d70 legacy=0x7f0bdc037ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7f0be3fff640 1 -- 172.21.15.22:0/1956237362 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bdc006a80 con 0x7f0bdc0383b0 2024-03-29T04:17:56.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7f1d49d9b640 1 --2- 172.21.15.22:0/689763597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d34004810 0x7f1d34004c90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000c1c0 tx=0x7f1d4001f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f26dc74e462ea server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7f1d4959a640 1 -- 172.21.15.22:0/689763597 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40007c60 con 0x7f1d34004810 2024-03-29T04:17:56.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7f1d4959a640 1 -- 172.21.15.22:0/689763597 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d40007270 con 0x7f1d34004810 2024-03-29T04:17:56.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7f1d4959a640 1 -- 172.21.15.22:0/689763597 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40007680 con 0x7f1d34004810 2024-03-29T04:17:56.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7fdb80f36640 1 -- 172.21.15.22:0/2517810445 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb780b2070 con 0x7fdb740df790 2024-03-29T04:17:56.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7fdb80f36640 1 -- 172.21.15.22:0/2517810445 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb780a5800 con 0x7fdb740df790 2024-03-29T04:17:56.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7fdb80f36640 1 -- 172.21.15.22:0/2517810445 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7800b040 con 0x7fdb740df790 2024-03-29T04:17:56.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7ff701447640 1 --2- 172.21.15.22:0/940085191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6f40dfe40 0x7ff6f4054ab0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40568a0 tx=0x7ff6f801e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77ab56e5c500937b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7ff6f3fff640 1 -- 172.21.15.22:0/940085191 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f80aa630 con 0x7ff6f40dfe40 2024-03-29T04:17:56.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7ff6f3fff640 1 -- 172.21.15.22:0/940085191 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f80aa7d0 con 0x7ff6f40dfe40 2024-03-29T04:17:56.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7ff6f3fff640 1 -- 172.21.15.22:0/940085191 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f80aab80 con 0x7ff6f40dfe40 2024-03-29T04:17:56.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7f1d4b59e640 1 -- 172.21.15.22:0/689763597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d34004810 msgr2=0x7f1d34004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7f1d4b59e640 1 --2- 172.21.15.22:0/689763597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d34004810 0x7f1d34004c90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000c1c0 tx=0x7f1d4001f330 comp rx=0 tx=0).stop 2024-03-29T04:17:56.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7fdb82f3a640 1 -- 172.21.15.22:0/2517810445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb740df790 msgr2=0x7fdb74055d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7fdb82f3a640 1 --2- 172.21.15.22:0/2517810445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb740df790 0x7fdb74055d20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb74056500 tx=0x7fdb7801c7e0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7fdb82f3a640 1 -- 172.21.15.22:0/2517810445 shutdown_connections 2024-03-29T04:17:56.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7fdb82f3a640 1 --2- 172.21.15.22:0/2517810445 >> v2:172.21.15.151:3300/0 conn(0x7fdb74036640 0x7fdb7403a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7fdb82f3a640 1 --2- 172.21.15.22:0/2517810445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb740df790 0x7fdb74055d20 secure :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb74056500 tx=0x7fdb7801c7e0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7fdb82f3a640 1 -- 172.21.15.22:0/2517810445 >> 172.21.15.22:0/2517810445 conn(0x7fdb7401e6d0 msgr2=0x7fdb74055400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7fd06a29d640 1 -- 172.21.15.22:0/962707877 wait complete. 2024-03-29T04:17:56.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7fd06a29d640 1 Processor -- start 2024-03-29T04:17:56.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7fdb82f3a640 1 -- 172.21.15.22:0/2517810445 shutdown_connections 2024-03-29T04:17:56.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7fdb82f3a640 1 -- 172.21.15.22:0/2517810445 wait complete. 2024-03-29T04:17:56.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7f58267fc640 1 -- 172.21.15.22:0/3876567666 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f58280ca020 con 0x7f583015a6a0 2024-03-29T04:17:56.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7f1d4b59e640 1 -- 172.21.15.22:0/689763597 shutdown_connections 2024-03-29T04:17:56.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7f1d4b59e640 1 --2- 172.21.15.22:0/689763597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d34004810 0x7f1d34004c90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7f1d4b59e640 1 -- 172.21.15.22:0/689763597 >> 172.21.15.22:0/689763597 conn(0x7f1d44020050 msgr2=0x7f1d44056d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7fd06a29d640 1 -- start start 2024-03-29T04:17:56.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7fdb82f3a640 1 Processor -- start 2024-03-29T04:17:56.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.812+0000 7fdb82f3a640 1 -- start start 2024-03-29T04:17:56.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7ff7036ab640 1 -- 172.21.15.22:0/940085191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6f40dfe40 msgr2=0x7ff6f4054ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7ff7036ab640 1 --2- 172.21.15.22:0/940085191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6f40dfe40 0x7ff6f4054ab0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40568a0 tx=0x7ff6f801e6c0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fdb82f3a640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb74036640 0x7fdb740d8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fdb82f3a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb7403a5f0 0x7fdb740cf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fdb82f3a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb740d1430 0x7fdb74034050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fdb82f3a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdb740dfcf0 con 0x7fdb7403a5f0 2024-03-29T04:17:56.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fdb82f3a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdb740dfb70 con 0x7fdb740d1430 2024-03-29T04:17:56.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fdb82f3a640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fdb74054da0 con 0x7fdb74036640 2024-03-29T04:17:56.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7ff7036ab640 1 -- 172.21.15.22:0/940085191 shutdown_connections 2024-03-29T04:17:56.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7ff7036ab640 1 --2- 172.21.15.22:0/940085191 >> v2:172.21.15.22:3300/0 conn(0x7ff6f40e41e0 0x7ff6f40085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7ff7036ab640 1 --2- 172.21.15.22:0/940085191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6f40dfe40 0x7ff6f4054ab0 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40568a0 tx=0x7ff6f801e6c0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7ff7036ab640 1 -- 172.21.15.22:0/940085191 >> 172.21.15.22:0/940085191 conn(0x7ff6f4055c60 msgr2=0x7ff6f4056070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fdb81f38640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb74036640 0x7fdb740d8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fdb81f38640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb74036640 0x7fdb740d8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.22:52954/0 (socket says 172.21.15.22:52954) 2024-03-29T04:17:56.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fdb81f38640 1 -- 172.21.15.22:0/2166884752 learned_addr learned my addr 172.21.15.22:0/2166884752 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7f0be3fff640 1 --2- 172.21.15.22:0/1956237362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bdc0383b0 0x7f0bdc0f22d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0bd80097c0 tx=0x7f0bd801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ecfa6172a2a5e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fd06a29d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0640b5970 0x7fd06412a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fd06a29d640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd06415afb0 0x7fd06414b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fd06a29d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd06414bbc0 0x7fd06415ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fd06929b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0640b5970 0x7fd06412a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fd06929b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0640b5970 0x7fd06412a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51488/0 (socket says 172.21.15.22:51488) 2024-03-29T04:17:56.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fd06929b640 1 -- 172.21.15.22:0/3880314 learned_addr learned my addr 172.21.15.22:0/3880314 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fd06a29d640 1 -- 172.21.15.22:0/3880314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd06415b510 con 0x7fd0640b5970 2024-03-29T04:17:56.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fd06a29d640 1 -- 172.21.15.22:0/3880314 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd06415b390 con 0x7fd06414bbc0 2024-03-29T04:17:56.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fd06a29d640 1 -- 172.21.15.22:0/3880314 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd0640ceb90 con 0x7fd06415afb0 2024-03-29T04:17:56.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fdb82739640 1 --2- 172.21.15.22:0/2166884752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb740d1430 0x7fdb74034050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7f1d4b59e640 1 -- 172.21.15.22:0/689763597 shutdown_connections 2024-03-29T04:17:56.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fdb81737640 1 --2- 172.21.15.22:0/2166884752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb7403a5f0 0x7fdb740cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fd06929b640 1 -- 172.21.15.22:0/3880314 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd06415afb0 msgr2=0x7fd06414b680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T04:17:56.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fd06929b640 1 --2- 172.21.15.22:0/3880314 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd06415afb0 0x7fd06414b680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fd06929b640 1 -- 172.21.15.22:0/3880314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd06414bbc0 msgr2=0x7fd06415ceb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T04:17:56.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fd06929b640 1 --2- 172.21.15.22:0/3880314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd06414bbc0 0x7fd06415ceb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fd06929b640 1 -- 172.21.15.22:0/3880314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06414db30 con 0x7fd0640b5970 2024-03-29T04:17:56.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fdb81f38640 1 -- 172.21.15.22:0/2166884752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb740d1430 msgr2=0x7fdb74034050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fdb81f38640 1 --2- 172.21.15.22:0/2166884752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb740d1430 0x7fdb74034050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fdb81f38640 1 -- 172.21.15.22:0/2166884752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb7403a5f0 msgr2=0x7fdb740cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fdb81f38640 1 --2- 172.21.15.22:0/2166884752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb7403a5f0 0x7fdb740cf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fdb81f38640 1 -- 172.21.15.22:0/2166884752 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb740d8b90 con 0x7fdb74036640 2024-03-29T04:17:56.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fd06929b640 1 --2- 172.21.15.22:0/3880314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0640b5970 0x7fd06412a040 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd05400c3f0 tx=0x7fd05400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fdb81f38640 1 --2- 172.21.15.22:0/2166884752 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb74036640 0x7fdb740d8650 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800a1c0 tx=0x7fdb78019580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7f0be27fc640 1 -- 172.21.15.22:0/1956237362 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd80b5070 con 0x7f0bdc0383b0 2024-03-29T04:17:56.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7f0be27fc640 1 -- 172.21.15.22:0/1956237362 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bd80a8990 con 0x7f0bdc0383b0 2024-03-29T04:17:56.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7f0be27fc640 1 -- 172.21.15.22:0/1956237362 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd80a8d60 con 0x7f0bdc0383b0 2024-03-29T04:17:56.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fd059ffb640 1 -- 172.21.15.22:0/3880314 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054017070 con 0x7fd0640b5970 2024-03-29T04:17:56.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fd059ffb640 1 -- 172.21.15.22:0/3880314 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0540052d0 con 0x7fd0640b5970 2024-03-29T04:17:56.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fd059ffb640 1 -- 172.21.15.22:0/3880314 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054003840 con 0x7fd0640b5970 2024-03-29T04:17:56.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fd06a29d640 1 -- 172.21.15.22:0/3880314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06414c050 con 0x7fd0640b5970 2024-03-29T04:17:56.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7fd06a29d640 1 -- 172.21.15.22:0/3880314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd06415c3d0 con 0x7fd0640b5970 2024-03-29T04:17:56.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7f1d4b59e640 1 -- 172.21.15.22:0/689763597 wait complete. 2024-03-29T04:17:56.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7f1d4b59e640 1 Processor -- start 2024-03-29T04:17:56.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7f0bea228640 1 -- 172.21.15.22:0/1956237362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bdc0383b0 msgr2=0x7f0bdc0f22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7f0bea228640 1 --2- 172.21.15.22:0/1956237362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bdc0383b0 0x7f0bdc0f22d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0bd80097c0 tx=0x7f0bd801ec90 comp rx=0 tx=0).stop 2024-03-29T04:17:56.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7f0bea228640 1 -- 172.21.15.22:0/1956237362 shutdown_connections 2024-03-29T04:17:56.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7f0bea228640 1 --2- 172.21.15.22:0/1956237362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bdc0383b0 0x7f0bdc0f22d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7f0bea228640 1 -- 172.21.15.22:0/1956237362 >> 172.21.15.22:0/1956237362 conn(0x7f0bdc054b00 msgr2=0x7f0bdc054f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7f0bea228640 1 -- 172.21.15.22:0/1956237362 shutdown_connections 2024-03-29T04:17:56.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7f0bea228640 1 -- 172.21.15.22:0/1956237362 wait complete. 2024-03-29T04:17:56.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7ff7036ab640 1 -- 172.21.15.22:0/940085191 shutdown_connections 2024-03-29T04:17:56.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.816+0000 7f1d4b59e640 1 -- start start 2024-03-29T04:17:56.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f1d4b59e640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d34004810 0x7f1d440e0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f1d4b59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d4404c1e0 0x7f1d4417a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f1d4b59e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d4404f740 0x7f1d4404fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f1d4b59e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1d440e5810 con 0x7f1d4404c1e0 2024-03-29T04:17:56.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f1d4b59e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d440e5690 con 0x7f1d4404f740 2024-03-29T04:17:56.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f1d4b59e640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1d44035da0 con 0x7f1d34004810 2024-03-29T04:17:56.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f1d49d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d4404c1e0 0x7f1d4417a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f1d4ad9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d4404f740 0x7f1d4404fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f1d49d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d4404c1e0 0x7f1d4417a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51494/0 (socket says 172.21.15.22:51494) 2024-03-29T04:17:56.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f1d49d9b640 1 -- 172.21.15.22:0/3613729851 learned_addr learned my addr 172.21.15.22:0/3613729851 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f1d4a59c640 1 --2- 172.21.15.22:0/3613729851 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d34004810 0x7f1d440e0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f0bea228640 1 Processor -- start 2024-03-29T04:17:56.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f1d4ad9d640 1 -- 172.21.15.22:0/3613729851 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d34004810 msgr2=0x7f1d440e0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f1d4ad9d640 1 --2- 172.21.15.22:0/3613729851 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d34004810 0x7f1d440e0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f1d4ad9d640 1 -- 172.21.15.22:0/3613729851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d4404c1e0 msgr2=0x7f1d4417a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f1d4ad9d640 1 --2- 172.21.15.22:0/3613729851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d4404c1e0 0x7f1d4417a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f1d4ad9d640 1 -- 172.21.15.22:0/3613729851 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d440527d0 con 0x7f1d4404f740 2024-03-29T04:17:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f1d4ad9d640 1 --2- 172.21.15.22:0/3613729851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d4404f740 0x7f1d4404fbc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c000b70 tx=0x7f1d4c069330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7fd059ffb640 1 -- 172.21.15.22:0/3880314 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fd05401a070 con 0x7fd0640b5970 2024-03-29T04:17:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7fd059ffb640 1 --2- 172.21.15.22:0/3880314 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd048068090 0x7fd04806a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7fd059ffb640 1 -- 172.21.15.22:0/3880314 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd054085b20 con 0x7fd0640b5970 2024-03-29T04:17:56.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7fdb727fc640 1 -- 172.21.15.22:0/2166884752 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb780b2050 con 0x7fdb74036640 2024-03-29T04:17:56.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7fdb727fc640 1 -- 172.21.15.22:0/2166884752 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb780be070 con 0x7fdb74036640 2024-03-29T04:17:56.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7fdb82f3a640 1 -- 172.21.15.22:0/2166884752 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb740d8e10 con 0x7fdb74036640 2024-03-29T04:17:56.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7fdb82f3a640 1 -- 172.21.15.22:0/2166884752 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb740d93d0 con 0x7fdb74036640 2024-03-29T04:17:56.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7fdb727fc640 1 -- 172.21.15.22:0/2166884752 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb780ad850 con 0x7fdb74036640 2024-03-29T04:17:56.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f0bea228640 1 -- start start 2024-03-29T04:17:56.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7ff7036ab640 1 -- 172.21.15.22:0/940085191 wait complete. 2024-03-29T04:17:56.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7ff7036ab640 1 Processor -- start 2024-03-29T04:17:56.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7fd068a9a640 1 --2- 172.21.15.22:0/3880314 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd048068090 0x7fd04806a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f0bea228640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bdc0383b0 0x7f0bdc04e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f0bea228640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bdc047d70 0x7f0bdc0dcb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f0bea228640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bdc0e7700 0x7f0bdc0dd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7fdb82f3a640 1 -- 172.21.15.22:0/2166884752 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb640052a0 con 0x7fdb74036640 2024-03-29T04:17:56.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f0bea228640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc0387e0 con 0x7f0bdc047d70 2024-03-29T04:17:56.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f0bea228640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc050160 con 0x7f0bdc0e7700 2024-03-29T04:17:56.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f0bea228640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0bdc038960 con 0x7f0bdc0383b0 2024-03-29T04:17:56.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f0be2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bdc047d70 0x7f0bdc0dcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f0be2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bdc047d70 0x7f0bdc0dcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51508/0 (socket says 172.21.15.22:51508) 2024-03-29T04:17:56.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f0be2ffd640 1 -- 172.21.15.22:0/1487693370 learned_addr learned my addr 172.21.15.22:0/1487693370 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f1d3affd640 1 -- 172.21.15.22:0/3613729851 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c074420 con 0x7f1d4404f740 2024-03-29T04:17:56.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f1d3affd640 1 -- 172.21.15.22:0/3613729851 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d4c0745c0 con 0x7f1d4404f740 2024-03-29T04:17:56.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f1d4b59e640 1 -- 172.21.15.22:0/3613729851 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4404c640 con 0x7f1d4404f740 2024-03-29T04:17:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f0be3fff640 1 --2- 172.21.15.22:0/1487693370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bdc0e7700 0x7f0bdc0dd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f0be37fe640 1 --2- 172.21.15.22:0/1487693370 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bdc0383b0 0x7f0bdc04e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f0be2ffd640 1 -- 172.21.15.22:0/1487693370 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bdc0383b0 msgr2=0x7f0bdc04e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f0be2ffd640 1 --2- 172.21.15.22:0/1487693370 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bdc0383b0 0x7f0bdc04e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f0be2ffd640 1 -- 172.21.15.22:0/1487693370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bdc0e7700 msgr2=0x7f0bdc0dd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f0be2ffd640 1 --2- 172.21.15.22:0/1487693370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bdc0e7700 0x7f0bdc0dd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f0be2ffd640 1 -- 172.21.15.22:0/1487693370 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bdc04cf70 con 0x7f0bdc047d70 2024-03-29T04:17:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f0be3fff640 1 --2- 172.21.15.22:0/1487693370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bdc0e7700 0x7f0bdc0dd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:17:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f0be37fe640 1 --2- 172.21.15.22:0/1487693370 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bdc0383b0 0x7f0bdc04e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:56.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7fd06a29d640 1 -- 172.21.15.22:0/3880314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd064068d80 con 0x7fd0640b5970 2024-03-29T04:17:56.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7fd068a9a640 1 --2- 172.21.15.22:0/3880314 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd048068090 0x7fd04806a550 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd0600ad000 tx=0x7fd0600a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7ff7036ab640 1 -- start start 2024-03-29T04:17:56.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7ff7036ab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6f4038e90 0x7ff6f40cd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7ff7036ab640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6f40e41e0 0x7ff6f40dcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7ff7036ab640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6f40dd0f0 0x7ff6f40d0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7ff7036ab640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f4006bc0 con 0x7ff6f4038e90 2024-03-29T04:17:56.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7ff7036ab640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f4006a40 con 0x7ff6f40dd0f0 2024-03-29T04:17:56.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7ff7036ab640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff6f4006d40 con 0x7ff6f40e41e0 2024-03-29T04:17:56.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7f1d3affd640 1 -- 172.21.15.22:0/3613729851 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c06c9e0 con 0x7f1d4404f740 2024-03-29T04:17:56.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7fdb727fc640 1 -- 172.21.15.22:0/2166884752 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fdb780ad9f0 con 0x7fdb74036640 2024-03-29T04:17:56.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7f0be2ffd640 1 --2- 172.21.15.22:0/1487693370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bdc047d70 0x7f0bdc0dcb80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000c500 tx=0x7f0bd000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7ff700c46640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6f40e41e0 0x7ff6f40dcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7ff700c46640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6f40e41e0 0x7ff6f40dcbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.22:52994/0 (socket says 172.21.15.22:52994) 2024-03-29T04:17:56.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.820+0000 7ff700c46640 1 -- 172.21.15.22:0/2212000592 learned_addr learned my addr 172.21.15.22:0/2212000592 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7f1d4b59e640 1 -- 172.21.15.22:0/3613729851 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d4404cbd0 con 0x7f1d4404f740 2024-03-29T04:17:56.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7fdb727fc640 1 --2- 172.21.15.22:0/2166884752 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdb600708d0 0x7fdb60072d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7fdb727fc640 1 -- 172.21.15.22:0/2166884752 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fdb78122110 con 0x7fdb74036640 2024-03-29T04:17:56.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7fdb81737640 1 --2- 172.21.15.22:0/2166884752 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdb600708d0 0x7fdb60072d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7ff701447640 1 --2- 172.21.15.22:0/2212000592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6f4038e90 0x7ff6f40cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7ff701447640 1 -- 172.21.15.22:0/2212000592 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6f40e41e0 msgr2=0x7ff6f40dcbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7ff701447640 1 --2- 172.21.15.22:0/2212000592 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6f40e41e0 0x7ff6f40dcbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7ff701447640 1 -- 172.21.15.22:0/2212000592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6f40dd0f0 msgr2=0x7ff6f40d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7ff701c48640 1 --2- 172.21.15.22:0/2212000592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6f40dd0f0 0x7ff6f40d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7ff701447640 1 --2- 172.21.15.22:0/2212000592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6f40dd0f0 0x7ff6f40d0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7ff701447640 1 -- 172.21.15.22:0/2212000592 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f40dd600 con 0x7ff6f4038e90 2024-03-29T04:17:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7ff701c48640 1 --2- 172.21.15.22:0/2212000592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6f40dd0f0 0x7ff6f40d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7f1d4b59e640 1 -- 172.21.15.22:0/3613729851 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d4403cc70 con 0x7f1d4404f740 2024-03-29T04:17:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7ff701447640 1 --2- 172.21.15.22:0/2212000592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6f4038e90 0x7ff6f40cd410 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80ae000 tx=0x7ff6f801b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7f1d3affd640 1 -- 172.21.15.22:0/3613729851 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1d4c093070 con 0x7f1d4404f740 2024-03-29T04:17:56.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7f1d3affd640 1 --2- 172.21.15.22:0/3613729851 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1d2c068040 0x7f1d2c06a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7f1d3affd640 1 -- 172.21.15.22:0/3613729851 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1d4c100080 con 0x7f1d4404f740 2024-03-29T04:17:56.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7f1d4a59c640 1 --2- 172.21.15.22:0/3613729851 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1d2c068040 0x7f1d2c06a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7fdb81737640 1 --2- 172.21.15.22:0/2166884752 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdb600708d0 0x7fdb60072d90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdb74023310 tx=0x7fdb6c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7f1d4a59c640 1 --2- 172.21.15.22:0/3613729851 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1d2c068040 0x7f1d2c06a500 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1d44003e90 tx=0x7f1d3c007d80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7f9208833640 1 Processor -- start 2024-03-29T04:17:56.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.824+0000 7f9208833640 1 -- start start 2024-03-29T04:17:56.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7f9208833640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f920415bc00 0x7f92040b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7f9208833640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f920415bfe0 0x7f9204070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7f9208833640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92040716b0 con 0x7f920415f490 2024-03-29T04:17:56.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7f9208833640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f9204071990 con 0x7f920415bc00 2024-03-29T04:17:56.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7f9208833640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f92040b5b00 con 0x7f920415bfe0 2024-03-29T04:17:56.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7f9202d9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f920415bc00 0x7f92040b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7f9202d9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f920415bc00 0x7f92040b65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51528/0 (socket says 172.21.15.22:51528) 2024-03-29T04:17:56.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7f9202d9d640 1 -- 172.21.15.22:0/3554439533 learned_addr learned my addr 172.21.15.22:0/3554439533 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7f9202d9d640 1 -- 172.21.15.22:0/3554439533 >> v2:172.21.15.151:3300/0 conn(0x7f920415bfe0 msgr2=0x7f9204070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7f920259c640 1 --2- 172.21.15.22:0/3554439533 >> v2:172.21.15.151:3300/0 conn(0x7f920415bfe0 0x7f9204070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7f9202d9d640 1 --2- 172.21.15.22:0/3554439533 >> v2:172.21.15.151:3300/0 conn(0x7f920415bfe0 0x7f9204070df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7f9202d9d640 1 -- 172.21.15.22:0/3554439533 >> v1:172.21.15.22:6789/0 conn(0x7f920415f490 legacy=0x7f92040d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7f9202d9d640 1 -- 172.21.15.22:0/3554439533 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92040b5de0 con 0x7f920415bc00 2024-03-29T04:17:56.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7fd059ffb640 1 -- 172.21.15.22:0/3880314 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fd06414db30 con 0x7fd0640b5970 2024-03-29T04:17:56.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7f0bcffff640 1 -- 172.21.15.22:0/1487693370 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0016020 con 0x7f0bdc047d70 2024-03-29T04:17:56.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7ff6f1ffb640 1 -- 172.21.15.22:0/2212000592 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f80aa980 con 0x7ff6f4038e90 2024-03-29T04:17:56.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7f0bcffff640 1 -- 172.21.15.22:0/1487693370 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bd0005220 con 0x7f0bdc047d70 2024-03-29T04:17:56.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7f0bea228640 1 -- 172.21.15.22:0/1487693370 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bdc04d250 con 0x7f0bdc047d70 2024-03-29T04:17:56.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7ff7036ab640 1 -- 172.21.15.22:0/2212000592 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f40dd8e0 con 0x7ff6f4038e90 2024-03-29T04:17:56.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7ff7036ab640 1 -- 172.21.15.22:0/2212000592 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f40ddea0 con 0x7ff6f4038e90 2024-03-29T04:17:56.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7f0bea228640 1 -- 172.21.15.22:0/1487693370 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bdc04d810 con 0x7f0bdc047d70 2024-03-29T04:17:56.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7ff6f1ffb640 1 -- 172.21.15.22:0/2212000592 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f80aa640 con 0x7ff6f4038e90 2024-03-29T04:17:56.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7ff6f1ffb640 1 -- 172.21.15.22:0/2212000592 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f80aad40 con 0x7ff6f4038e90 2024-03-29T04:17:56.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7f0bcffff640 1 -- 172.21.15.22:0/1487693370 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0007030 con 0x7f0bdc047d70 2024-03-29T04:17:56.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7f0bea228640 1 -- 172.21.15.22:0/1487693370 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bdc0053d0 con 0x7f0bdc047d70 2024-03-29T04:17:56.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.828+0000 7fdb727fc640 1 -- 172.21.15.22:0/2166884752 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fdb740d8b90 con 0x7fdb74036640 2024-03-29T04:17:56.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7f9202d9d640 1 --2- 172.21.15.22:0/3554439533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f920415bc00 0x7f92040b65e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009870 tx=0x7f91f401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40d33622d1fbc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7f9201d9b640 1 -- 172.21.15.22:0/3554439533 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f40b2070 con 0x7f920415bc00 2024-03-29T04:17:56.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7ff6f1ffb640 1 -- 172.21.15.22:0/2212000592 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff6f8003620 con 0x7ff6f4038e90 2024-03-29T04:17:56.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7ff6f1ffb640 1 --2- 172.21.15.22:0/2212000592 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff6e0067ff0 0x7ff6e006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7ff6f1ffb640 1 -- 172.21.15.22:0/2212000592 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff6f8120d60 con 0x7ff6f4038e90 2024-03-29T04:17:56.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7f0bcffff640 1 -- 172.21.15.22:0/1487693370 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f0bd00071f0 con 0x7f0bdc047d70 2024-03-29T04:17:56.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7f0bcffff640 1 --2- 172.21.15.22:0/1487693370 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0bc8067fc0 0x7f0bc806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7fba9e1f2640 1 Processor -- start 2024-03-29T04:17:56.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7fba9e1f2640 1 -- start start 2024-03-29T04:17:56.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7ff700c46640 1 --2- 172.21.15.22:0/2212000592 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff6e0067ff0 0x7ff6e006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7f0bcffff640 1 -- 172.21.15.22:0/1487693370 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0bd0011e60 con 0x7f0bdc047d70 2024-03-29T04:17:56.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7f0be37fe640 1 --2- 172.21.15.22:0/1487693370 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0bc8067fc0 0x7f0bc806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7fba9e1f2640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fba900e1320 0x7fba90054b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7f0be37fe640 1 --2- 172.21.15.22:0/1487693370 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0bc8067fc0 0x7f0bc806a480 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0be4051240 tx=0x7f0be406f040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7fba9e1f2640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fba90052710 0x7fba90055360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7fba9e1f2640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba900558a0 0x7fba900366b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7ff700c46640 1 --2- 172.21.15.22:0/2212000592 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff6e0067ff0 0x7ff6e006a4b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec000fc0 tx=0x7ff6ec007f50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7fba9d1f0640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fba900e1320 0x7fba90054b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7fba9d1f0640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fba900e1320 0x7fba90054b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:46554/0 (socket says 172.21.15.22:46554) 2024-03-29T04:17:56.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7fba9d1f0640 1 -- 172.21.15.22:0/4249573297 learned_addr learned my addr 172.21.15.22:0/4249573297 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7fba9c9ef640 1 --2- 172.21.15.22:0/4249573297 >> v2:172.21.15.22:3300/0 conn(0x7fba90052710 0x7fba90055360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7fba9e1f2640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fba900e5aa0 con 0x7fba90052710 2024-03-29T04:17:56.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7fba9e1f2640 1 -- 172.21.15.22:0/4249573297 --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fba90051c30 con 0x7fba900e1320 2024-03-29T04:17:56.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7fba9d9f1640 1 --2- 172.21.15.22:0/4249573297 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba900558a0 0x7fba900366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.832+0000 7fba9e1f2640 1 -- 172.21.15.22:0/4249573297 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fba90051f10 con 0x7fba900558a0 2024-03-29T04:17:56.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.836+0000 7fba9c9ef640 1 -- 172.21.15.22:0/4249573297 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba900558a0 msgr2=0x7fba900366b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.836+0000 7fba9c9ef640 1 --2- 172.21.15.22:0/4249573297 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba900558a0 0x7fba900366b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.836+0000 7fba9c9ef640 1 -- 172.21.15.22:0/4249573297 >> v2:172.21.15.151:3300/0 conn(0x7fba900e1320 msgr2=0x7fba90054b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:56.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.836+0000 7fba9c9ef640 1 --2- 172.21.15.22:0/4249573297 >> v2:172.21.15.151:3300/0 conn(0x7fba900e1320 0x7fba90054b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.836+0000 7fba9c9ef640 1 -- 172.21.15.22:0/4249573297 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba900521f0 con 0x7fba90052710 2024-03-29T04:17:56.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.836+0000 7fba9d9f1640 1 --2- 172.21.15.22:0/4249573297 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba900558a0 0x7fba900366b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:17:56.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.836+0000 7fba9c9ef640 1 --2- 172.21.15.22:0/4249573297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba90052710 0x7fba90055360 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fba900548d0 tx=0x7fba8801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905780e980451f99 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.836+0000 7f1d3affd640 1 -- 172.21.15.22:0/3613729851 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1d4c0cff80 con 0x7f1d4404f740 2024-03-29T04:17:56.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.836+0000 7f9201d9b640 1 -- 172.21.15.22:0/3554439533 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f40a9070 con 0x7f920415bc00 2024-03-29T04:17:56.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.836+0000 7f0bcffff640 1 -- 172.21.15.22:0/1487693370 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f0bd0020020 con 0x7f0bdc047d70 2024-03-29T04:17:56.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.836+0000 7fba8ffff640 1 -- 172.21.15.22:0/4249573297 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba880b0070 con 0x7fba90052710 2024-03-29T04:17:56.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.836+0000 7fba8ffff640 1 -- 172.21.15.22:0/4249573297 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba880ab600 con 0x7fba90052710 2024-03-29T04:17:56.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.836+0000 7fba8ffff640 1 -- 172.21.15.22:0/4249573297 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba880ab9d0 con 0x7fba90052710 2024-03-29T04:17:56.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.836+0000 7f9208833640 1 -- 172.21.15.22:0/3554439533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f920415bc00 msgr2=0x7f92040b65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.836+0000 7f9208833640 1 --2- 172.21.15.22:0/3554439533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f920415bc00 0x7f92040b65e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009870 tx=0x7f91f401c450 comp rx=0 tx=0).stop 2024-03-29T04:17:56.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.836+0000 7f9208833640 1 -- 172.21.15.22:0/3554439533 shutdown_connections 2024-03-29T04:17:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.836+0000 7f9208833640 1 --2- 172.21.15.22:0/3554439533 >> v2:172.21.15.151:3300/0 conn(0x7f920415bfe0 0x7f9204070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.836+0000 7f9208833640 1 --2- 172.21.15.22:0/3554439533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f920415bc00 0x7f92040b65e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.836+0000 7f9208833640 1 -- 172.21.15.22:0/3554439533 >> 172.21.15.22:0/3554439533 conn(0x7f92040ce960 msgr2=0x7f92040ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7fba9e1f2640 1 -- 172.21.15.22:0/4249573297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba90052710 msgr2=0x7fba90055360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7fba9e1f2640 1 --2- 172.21.15.22:0/4249573297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba90052710 0x7fba90055360 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fba900548d0 tx=0x7fba8801c450 comp rx=0 tx=0).stop 2024-03-29T04:17:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f9208833640 1 -- 172.21.15.22:0/3554439533 shutdown_connections 2024-03-29T04:17:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f9208833640 1 -- 172.21.15.22:0/3554439533 wait complete. 2024-03-29T04:17:56.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7fba9e1f2640 1 -- 172.21.15.22:0/4249573297 shutdown_connections 2024-03-29T04:17:56.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7fba9e1f2640 1 --2- 172.21.15.22:0/4249573297 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba900558a0 0x7fba900366b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7fba9e1f2640 1 --2- 172.21.15.22:0/4249573297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba90052710 0x7fba90055360 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7fba9e1f2640 1 --2- 172.21.15.22:0/4249573297 >> v2:172.21.15.151:3300/0 conn(0x7fba900e1320 0x7fba90054b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7fba9e1f2640 1 -- 172.21.15.22:0/4249573297 >> 172.21.15.22:0/4249573297 conn(0x7fba90020050 msgr2=0x7fba90056d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:56.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7fba9e1f2640 1 -- 172.21.15.22:0/4249573297 shutdown_connections 2024-03-29T04:17:56.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7fba9e1f2640 1 -- 172.21.15.22:0/4249573297 wait complete. 2024-03-29T04:17:56.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f9208833640 1 Processor -- start 2024-03-29T04:17:56.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7fba9e1f2640 1 Processor -- start 2024-03-29T04:17:56.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7fba9e1f2640 1 -- start start 2024-03-29T04:17:56.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f9208833640 1 -- start start 2024-03-29T04:17:56.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f9208833640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f920415bc00 0x7f92041fc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f9208833640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f920415bfe0 0x7f920414faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f9208833640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f920415f490 0x7f9204154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f9208833640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f92040d02a0 con 0x7f920415bfe0 2024-03-29T04:17:56.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f9208833640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f92040d1ae0 con 0x7f920415bc00 2024-03-29T04:17:56.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f9208833640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f92040d0420 con 0x7f920415f490 2024-03-29T04:17:56.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f920259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f920415bfe0 0x7f920414faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f920259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f920415bfe0 0x7f920414faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51560/0 (socket says 172.21.15.22:51560) 2024-03-29T04:17:56.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f920259c640 1 -- 172.21.15.22:0/3336120855 learned_addr learned my addr 172.21.15.22:0/3336120855 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f9202d9d640 1 --2- 172.21.15.22:0/3336120855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f920415bc00 0x7f92041fc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f920359e640 1 --2- 172.21.15.22:0/3336120855 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f920415f490 0x7f9204154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f920259c640 1 -- 172.21.15.22:0/3336120855 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f920415f490 msgr2=0x7f9204154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f920259c640 1 --2- 172.21.15.22:0/3336120855 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f920415f490 0x7f9204154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f920259c640 1 -- 172.21.15.22:0/3336120855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f920415bc00 msgr2=0x7f92041fc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f920259c640 1 --2- 172.21.15.22:0/3336120855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f920415bc00 0x7f92041fc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f920259c640 1 -- 172.21.15.22:0/3336120855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92040cbd60 con 0x7f920415bfe0 2024-03-29T04:17:56.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f9202d9d640 1 --2- 172.21.15.22:0/3336120855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f920415bc00 0x7f92041fc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:17:56.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f920259c640 1 --2- 172.21.15.22:0/3336120855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f920415bfe0 0x7f920414faa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f91f800c7f0 tx=0x7f91f800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f91eb7fe640 1 -- 172.21.15.22:0/3336120855 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8015070 con 0x7f920415bfe0 2024-03-29T04:17:56.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f91eb7fe640 1 -- 172.21.15.22:0/3336120855 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f8007820 con 0x7f920415bfe0 2024-03-29T04:17:56.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f91eb7fe640 1 -- 172.21.15.22:0/3336120855 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8007bd0 con 0x7f920415bfe0 2024-03-29T04:17:56.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7fba9e1f2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba90052710 0x7fba900dda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7fba9d1f0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba90052710 0x7fba900dda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7fba9d1f0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba90052710 0x7fba900dda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51564/0 (socket says 172.21.15.22:51564) 2024-03-29T04:17:56.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7fba9d1f0640 1 -- 172.21.15.22:0/4159707612 learned_addr learned my addr 172.21.15.22:0/4159707612 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:56.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7fba9e1f2640 1 --2- 172.21.15.22:0/4159707612 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba900558a0 0x7fba900df4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f9208833640 1 -- 172.21.15.22:0/3336120855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92041fccd0 con 0x7f920415bfe0 2024-03-29T04:17:56.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.840+0000 7f9208833640 1 -- 172.21.15.22:0/3336120855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92040cc300 con 0x7f920415bfe0 2024-03-29T04:17:56.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7fba9e1f2640 1 --2- 172.21.15.22:0/4159707612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba900e1320 0x7fba9004de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7fba9e1f2640 1 -- 172.21.15.22:0/4159707612 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fba90035f20 con 0x7fba90052710 2024-03-29T04:17:56.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7fba9c9ef640 1 --2- 172.21.15.22:0/4159707612 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba900558a0 0x7fba900df4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7fba9e1f2640 1 -- 172.21.15.22:0/4159707612 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba90035da0 con 0x7fba900e1320 2024-03-29T04:17:56.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7fba9e1f2640 1 -- 172.21.15.22:0/4159707612 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fba90053420 con 0x7fba900558a0 2024-03-29T04:17:56.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7fba9d9f1640 1 --2- 172.21.15.22:0/4159707612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba900e1320 0x7fba9004de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7f9208833640 1 -- 172.21.15.22:0/3336120855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9204068d80 con 0x7f920415bfe0 2024-03-29T04:17:56.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7f91eb7fe640 1 -- 172.21.15.22:0/3336120855 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f91f8018070 con 0x7f920415bfe0 2024-03-29T04:17:56.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7fba9c9ef640 1 -- 172.21.15.22:0/4159707612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba900e1320 msgr2=0x7fba9004de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7fba9c9ef640 1 --2- 172.21.15.22:0/4159707612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba900e1320 0x7fba9004de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7fba9c9ef640 1 -- 172.21.15.22:0/4159707612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba90052710 msgr2=0x7fba900dda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:56.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7fba9c9ef640 1 --2- 172.21.15.22:0/4159707612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba90052710 0x7fba900dda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:56.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7fba9c9ef640 1 -- 172.21.15.22:0/4159707612 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9004e390 con 0x7fba900558a0 2024-03-29T04:17:56.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7fba9c9ef640 1 --2- 172.21.15.22:0/4159707612 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba900558a0 0x7fba900df4c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fba9000c1a0 tx=0x7fba880191f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7fba8dffb640 1 -- 172.21.15.22:0/4159707612 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba880b0040 con 0x7fba900558a0 2024-03-29T04:17:56.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7fba8dffb640 1 -- 172.21.15.22:0/4159707612 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba880b9070 con 0x7fba900558a0 2024-03-29T04:17:56.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7fba9e1f2640 1 -- 172.21.15.22:0/4159707612 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba900de700 con 0x7fba900558a0 2024-03-29T04:17:56.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7fba8dffb640 1 -- 172.21.15.22:0/4159707612 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba880b4650 con 0x7fba900558a0 2024-03-29T04:17:56.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7fba9e1f2640 1 -- 172.21.15.22:0/4159707612 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fba900decc0 con 0x7fba900558a0 2024-03-29T04:17:56.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7f91eb7fe640 1 --2- 172.21.15.22:0/3336120855 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f91f0067fc0 0x7f91f006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7f91eb7fe640 1 -- 172.21.15.22:0/3336120855 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f91f8086500 con 0x7f920415bfe0 2024-03-29T04:17:56.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7fba8dffb640 1 -- 172.21.15.22:0/4159707612 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fba880b47f0 con 0x7fba900558a0 2024-03-29T04:17:56.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7fba8dffb640 1 --2- 172.21.15.22:0/4159707612 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fba7c067fc0 0x7fba7c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:56.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7fba8dffb640 1 -- 172.21.15.22:0/4159707612 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fba8811fcd0 con 0x7fba900558a0 2024-03-29T04:17:56.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.844+0000 7fba9d1f0640 1 --2- 172.21.15.22:0/4159707612 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fba7c067fc0 0x7fba7c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.848+0000 7f9202d9d640 1 --2- 172.21.15.22:0/3336120855 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f91f0067fc0 0x7f91f006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:56.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.848+0000 7fba9d1f0640 1 --2- 172.21.15.22:0/4159707612 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fba7c067fc0 0x7fba7c06a480 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fba94006f50 tx=0x7fba94009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.848+0000 7f9202d9d640 1 --2- 172.21.15.22:0/3336120855 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f91f0067fc0 0x7f91f006a480 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f91f40af000 tx=0x7f91f40a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:56.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.848+0000 7f91eb7fe640 1 -- 172.21.15.22:0/3336120855 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f91f8056310 con 0x7f920415bfe0 2024-03-29T04:17:56.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.852+0000 7fba9e1f2640 1 -- 172.21.15.22:0/4159707612 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba900057f0 con 0x7fba900558a0 2024-03-29T04:17:56.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.860+0000 7ff7036ab640 1 -- 172.21.15.22:0/2212000592 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6dc0052a0 con 0x7ff6f4038e90 2024-03-29T04:17:56.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.864+0000 7fba8dffb640 1 -- 172.21.15.22:0/4159707612 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fba88016070 con 0x7fba900558a0 2024-03-29T04:17:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:56.872+0000 7ff6f1ffb640 1 -- 172.21.15.22:0/2212000592 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff6f80ae030 con 0x7ff6f4038e90 2024-03-29T04:17:57.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.116+0000 7fd06a29d640 1 -- 172.21.15.22:0/3880314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd0640cf4e0 con 0x7fd0640b5970 2024-03-29T04:17:57.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.116+0000 7fd059ffb640 1 -- 172.21.15.22:0/3880314 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd054005480 con 0x7fd0640b5970 2024-03-29T04:17:57.101 INFO:teuthology.orchestra.run.smithi022.stdout:55834574855 2024-03-29T04:17:57.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.124+0000 7fd0477fe640 1 -- 172.21.15.22:0/3880314 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd048068090 msgr2=0x7fd04806a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:57.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.124+0000 7fd0477fe640 1 --2- 172.21.15.22:0/3880314 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd048068090 0x7fd04806a550 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd0600ad000 tx=0x7fd0600a6000 comp rx=0 tx=0).stop 2024-03-29T04:17:57.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.124+0000 7fd0477fe640 1 -- 172.21.15.22:0/3880314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0640b5970 msgr2=0x7fd06412a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:57.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.124+0000 7fd0477fe640 1 --2- 172.21.15.22:0/3880314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0640b5970 0x7fd06412a040 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd05400c3f0 tx=0x7fd05400c8b0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.124+0000 7fd0477fe640 1 -- 172.21.15.22:0/3880314 shutdown_connections 2024-03-29T04:17:57.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.124+0000 7fd0477fe640 1 --2- 172.21.15.22:0/3880314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd06414bbc0 0x7fd06415ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.124+0000 7fd0477fe640 1 --2- 172.21.15.22:0/3880314 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd06415afb0 0x7fd06414b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.124+0000 7fd0477fe640 1 --2- 172.21.15.22:0/3880314 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd048068090 0x7fd04806a550 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.124+0000 7fd0477fe640 1 --2- 172.21.15.22:0/3880314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0640b5970 0x7fd06412a040 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.124+0000 7fd0477fe640 1 -- 172.21.15.22:0/3880314 >> 172.21.15.22:0/3880314 conn(0x7fd064073440 msgr2=0x7fd0640b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:57.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.124+0000 7fd0477fe640 1 -- 172.21.15.22:0/3880314 shutdown_connections 2024-03-29T04:17:57.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.124+0000 7fd0477fe640 1 -- 172.21.15.22:0/3880314 wait complete. 2024-03-29T04:17:57.127 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2024-03-29T04:17:57.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.156+0000 7fdb82f3a640 1 -- 172.21.15.22:0/2166884752 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fdb64005860 con 0x7fdb74036640 2024-03-29T04:17:57.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.156+0000 7fdb727fc640 1 -- 172.21.15.22:0/2166884752 <== mon.2 v2:172.21.15.151:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdb78016400 con 0x7fdb74036640 2024-03-29T04:17:57.139 INFO:teuthology.orchestra.run.smithi022.stdout:55834574856 2024-03-29T04:17:57.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.160+0000 7fdb82f3a640 1 -- 172.21.15.22:0/2166884752 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdb600708d0 msgr2=0x7fdb60072d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:57.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.160+0000 7fdb82f3a640 1 --2- 172.21.15.22:0/2166884752 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdb600708d0 0x7fdb60072d90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdb74023310 tx=0x7fdb6c008040 comp rx=0 tx=0).stop 2024-03-29T04:17:57.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.160+0000 7fdb82f3a640 1 -- 172.21.15.22:0/2166884752 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb74036640 msgr2=0x7fdb740d8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:57.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.160+0000 7fdb82f3a640 1 --2- 172.21.15.22:0/2166884752 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb74036640 0x7fdb740d8650 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800a1c0 tx=0x7fdb78019580 comp rx=0 tx=0).stop 2024-03-29T04:17:57.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.160+0000 7fdb82f3a640 1 -- 172.21.15.22:0/2166884752 shutdown_connections 2024-03-29T04:17:57.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.160+0000 7fdb82f3a640 1 --2- 172.21.15.22:0/2166884752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb740d1430 0x7fdb74034050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.160+0000 7fdb82f3a640 1 --2- 172.21.15.22:0/2166884752 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdb600708d0 0x7fdb60072d90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.160+0000 7fdb82f3a640 1 --2- 172.21.15.22:0/2166884752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb7403a5f0 0x7fdb740cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.160+0000 7fdb82f3a640 1 --2- 172.21.15.22:0/2166884752 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdb74036640 0x7fdb740d8650 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.160+0000 7fdb82f3a640 1 -- 172.21.15.22:0/2166884752 >> 172.21.15.22:0/2166884752 conn(0x7fdb7401e6d0 msgr2=0x7fdb74055e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.160+0000 7fdb82f3a640 1 -- 172.21.15.22:0/2166884752 shutdown_connections 2024-03-29T04:17:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.160+0000 7fdb82f3a640 1 -- 172.21.15.22:0/2166884752 wait complete. 2024-03-29T04:17:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.164+0000 7f5837b5c640 1 -- 172.21.15.22:0/3876567666 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f58300686e0 con 0x7f583015a6a0 2024-03-29T04:17:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.164+0000 7f58267fc640 1 -- 172.21.15.22:0/3876567666 <== mon.2 v2:172.21.15.151:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5828018960 con 0x7f583015a6a0 2024-03-29T04:17:57.145 INFO:teuthology.orchestra.run.smithi022.stdout:55834574855 2024-03-29T04:17:57.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.168+0000 7f5837b5c640 1 -- 172.21.15.22:0/3876567666 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f581c068120 msgr2=0x7f581c06a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:57.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.168+0000 7f5837b5c640 1 --2- 172.21.15.22:0/3876567666 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f581c068120 0x7f581c06a5e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f582c00aee0 tx=0x7f582c008040 comp rx=0 tx=0).stop 2024-03-29T04:17:57.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.168+0000 7f5837b5c640 1 -- 172.21.15.22:0/3876567666 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f583015a6a0 msgr2=0x7f583014e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:57.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.168+0000 7f5837b5c640 1 --2- 172.21.15.22:0/3876567666 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f583015a6a0 0x7f583014e1f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f58280af000 tx=0x7f582801ba50 comp rx=0 tx=0).stop 2024-03-29T04:17:57.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.168+0000 7f5837b5c640 1 -- 172.21.15.22:0/3876567666 shutdown_connections 2024-03-29T04:17:57.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.168+0000 7f5837b5c640 1 --2- 172.21.15.22:0/3876567666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f583015aac0 0x7f5830151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.168+0000 7f5837b5c640 1 --2- 172.21.15.22:0/3876567666 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f583015a6a0 0x7f583014e1f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.168+0000 7f5837b5c640 1 --2- 172.21.15.22:0/3876567666 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f581c068120 0x7f581c06a5e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.168+0000 7f5837b5c640 1 --2- 172.21.15.22:0/3876567666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f583015a2c0 0x7f583015dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.168+0000 7f5837b5c640 1 -- 172.21.15.22:0/3876567666 >> 172.21.15.22:0/3876567666 conn(0x7f583006c570 msgr2=0x7f58300b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:57.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.168+0000 7f5837b5c640 1 -- 172.21.15.22:0/3876567666 shutdown_connections 2024-03-29T04:17:57.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.168+0000 7f5837b5c640 1 -- 172.21.15.22:0/3876567666 wait complete. 2024-03-29T04:17:57.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.168+0000 7f1d4b59e640 1 -- 172.21.15.22:0/3613729851 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f1d440057f0 con 0x7f1d4404f740 2024-03-29T04:17:57.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.172+0000 7f1d3affd640 1 -- 172.21.15.22:0/3613729851 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1d4c0d4280 con 0x7f1d4404f740 2024-03-29T04:17:57.151 INFO:teuthology.orchestra.run.smithi022.stdout:55834574856 2024-03-29T04:17:57.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.172+0000 7f1d4b59e640 1 -- 172.21.15.22:0/3613729851 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1d2c068040 msgr2=0x7f1d2c06a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:57.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.172+0000 7f1d4b59e640 1 --2- 172.21.15.22:0/3613729851 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1d2c068040 0x7f1d2c06a500 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1d44003e90 tx=0x7f1d3c007d80 comp rx=0 tx=0).stop 2024-03-29T04:17:57.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.172+0000 7f1d4b59e640 1 -- 172.21.15.22:0/3613729851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d4404f740 msgr2=0x7f1d4404fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:57.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.172+0000 7f1d4b59e640 1 --2- 172.21.15.22:0/3613729851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d4404f740 0x7f1d4404fbc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c000b70 tx=0x7f1d4c069330 comp rx=0 tx=0).stop 2024-03-29T04:17:57.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.172+0000 7f1d4b59e640 1 -- 172.21.15.22:0/3613729851 shutdown_connections 2024-03-29T04:17:57.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.172+0000 7f1d4b59e640 1 --2- 172.21.15.22:0/3613729851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d4404f740 0x7f1d4404fbc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.172+0000 7f1d4b59e640 1 --2- 172.21.15.22:0/3613729851 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1d2c068040 0x7f1d2c06a500 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.172+0000 7f1d4b59e640 1 --2- 172.21.15.22:0/3613729851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d4404c1e0 0x7f1d4417a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.172+0000 7f1d4b59e640 1 --2- 172.21.15.22:0/3613729851 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d34004810 0x7f1d440e0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.172+0000 7f1d4b59e640 1 -- 172.21.15.22:0/3613729851 >> 172.21.15.22:0/3613729851 conn(0x7f1d44020050 msgr2=0x7f1d4400d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:57.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.172+0000 7f1d4b59e640 1 -- 172.21.15.22:0/3613729851 shutdown_connections 2024-03-29T04:17:57.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.172+0000 7f1d4b59e640 1 -- 172.21.15.22:0/3613729851 wait complete. 2024-03-29T04:17:57.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.176+0000 7f0bea228640 1 -- 172.21.15.22:0/1487693370 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f0bdc0556b0 con 0x7f0bdc047d70 2024-03-29T04:17:57.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.176+0000 7f0bcffff640 1 -- 172.21.15.22:0/1487693370 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0bd0057540 con 0x7f0bdc047d70 2024-03-29T04:17:57.159 INFO:teuthology.orchestra.run.smithi022.stdout:55834574856 2024-03-29T04:17:57.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.180+0000 7f9208833640 1 -- 172.21.15.22:0/3336120855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f9204071b40 con 0x7f920415bfe0 2024-03-29T04:17:57.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.180+0000 7f91eb7fe640 1 -- 172.21.15.22:0/3336120855 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f91f805a610 con 0x7f920415bfe0 2024-03-29T04:17:57.162 INFO:teuthology.orchestra.run.smithi022.stdout:55834574855 2024-03-29T04:17:57.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.180+0000 7f0bea228640 1 -- 172.21.15.22:0/1487693370 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0bc8067fc0 msgr2=0x7f0bc806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:57.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.180+0000 7f0bea228640 1 --2- 172.21.15.22:0/1487693370 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0bc8067fc0 0x7f0bc806a480 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0be4051240 tx=0x7f0be406f040 comp rx=0 tx=0).stop 2024-03-29T04:17:57.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.180+0000 7f0bea228640 1 -- 172.21.15.22:0/1487693370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bdc047d70 msgr2=0x7f0bdc0dcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:57.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.180+0000 7f0bea228640 1 --2- 172.21.15.22:0/1487693370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bdc047d70 0x7f0bdc0dcb80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000c500 tx=0x7f0bd000c9c0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.180+0000 7f0bea228640 1 -- 172.21.15.22:0/1487693370 shutdown_connections 2024-03-29T04:17:57.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.180+0000 7f0bea228640 1 --2- 172.21.15.22:0/1487693370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bdc0e7700 0x7f0bdc0dd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.180+0000 7f0bea228640 1 --2- 172.21.15.22:0/1487693370 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0bc8067fc0 0x7f0bc806a480 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.180+0000 7f0bea228640 1 --2- 172.21.15.22:0/1487693370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bdc047d70 0x7f0bdc0dcb80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.180+0000 7f0bea228640 1 --2- 172.21.15.22:0/1487693370 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bdc0383b0 0x7f0bdc04e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.180+0000 7f0bea228640 1 -- 172.21.15.22:0/1487693370 >> 172.21.15.22:0/1487693370 conn(0x7f0bdc054b00 msgr2=0x7f0bdc0e65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:57.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.180+0000 7f0bea228640 1 -- 172.21.15.22:0/1487693370 shutdown_connections 2024-03-29T04:17:57.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.180+0000 7f0bea228640 1 -- 172.21.15.22:0/1487693370 wait complete. 2024-03-29T04:17:57.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.184+0000 7f9208833640 1 -- 172.21.15.22:0/3336120855 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f91f0067fc0 msgr2=0x7f91f006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:57.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.184+0000 7f9208833640 1 --2- 172.21.15.22:0/3336120855 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f91f0067fc0 0x7f91f006a480 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f91f40af000 tx=0x7f91f40a7000 comp rx=0 tx=0).stop 2024-03-29T04:17:57.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.184+0000 7f9208833640 1 -- 172.21.15.22:0/3336120855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f920415bfe0 msgr2=0x7f920414faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:57.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.184+0000 7f9208833640 1 --2- 172.21.15.22:0/3336120855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f920415bfe0 0x7f920414faa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f91f800c7f0 tx=0x7f91f800ccb0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.184+0000 7f9208833640 1 -- 172.21.15.22:0/3336120855 shutdown_connections 2024-03-29T04:17:57.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.184+0000 7f9208833640 1 --2- 172.21.15.22:0/3336120855 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f920415f490 0x7f9204154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.184+0000 7f9208833640 1 --2- 172.21.15.22:0/3336120855 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f91f0067fc0 0x7f91f006a480 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.184+0000 7f9208833640 1 --2- 172.21.15.22:0/3336120855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f920415bfe0 0x7f920414faa0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.184+0000 7f9208833640 1 --2- 172.21.15.22:0/3336120855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f920415bc00 0x7f92041fc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.184+0000 7f9208833640 1 -- 172.21.15.22:0/3336120855 >> 172.21.15.22:0/3336120855 conn(0x7f92040ce960 msgr2=0x7f92040b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:57.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.184+0000 7f9208833640 1 -- 172.21.15.22:0/3336120855 shutdown_connections 2024-03-29T04:17:57.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.184+0000 7f9208833640 1 -- 172.21.15.22:0/3336120855 wait complete. 2024-03-29T04:17:57.174 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.7 2024-03-29T04:17:57.176 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2024-03-29T04:17:57.178 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.6 2024-03-29T04:17:57.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.196+0000 7ff7036ab640 1 -- 172.21.15.22:0/2212000592 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ff6dc005590 con 0x7ff6f4038e90 2024-03-29T04:17:57.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.196+0000 7ff6f1ffb640 1 -- 172.21.15.22:0/2212000592 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff6f80182e0 con 0x7ff6f4038e90 2024-03-29T04:17:57.179 INFO:teuthology.orchestra.run.smithi022.stdout:55834574855 2024-03-29T04:17:57.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.196+0000 7fba9e1f2640 1 -- 172.21.15.22:0/4159707612 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fba90055140 con 0x7fba900558a0 2024-03-29T04:17:57.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.196+0000 7fba8dffb640 1 -- 172.21.15.22:0/4159707612 <== mon.2 v2:172.21.15.151:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fba880efb00 con 0x7fba900558a0 2024-03-29T04:17:57.179 INFO:teuthology.orchestra.run.smithi022.stdout:55834574856 2024-03-29T04:17:57.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7ff7036ab640 1 -- 172.21.15.22:0/2212000592 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff6e0067ff0 msgr2=0x7ff6e006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:57.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7ff7036ab640 1 --2- 172.21.15.22:0/2212000592 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff6e0067ff0 0x7ff6e006a4b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec000fc0 tx=0x7ff6ec007f50 comp rx=0 tx=0).stop 2024-03-29T04:17:57.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7ff7036ab640 1 -- 172.21.15.22:0/2212000592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6f4038e90 msgr2=0x7ff6f40cd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:57.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7ff7036ab640 1 --2- 172.21.15.22:0/2212000592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6f4038e90 0x7ff6f40cd410 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80ae000 tx=0x7ff6f801b460 comp rx=0 tx=0).stop 2024-03-29T04:17:57.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7ff7036ab640 1 -- 172.21.15.22:0/2212000592 shutdown_connections 2024-03-29T04:17:57.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7ff7036ab640 1 --2- 172.21.15.22:0/2212000592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6f40dd0f0 0x7ff6f40d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7ff7036ab640 1 --2- 172.21.15.22:0/2212000592 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6f40e41e0 0x7ff6f40dcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7ff7036ab640 1 --2- 172.21.15.22:0/2212000592 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff6e0067ff0 0x7ff6e006a4b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7ff7036ab640 1 --2- 172.21.15.22:0/2212000592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6f4038e90 0x7ff6f40cd410 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7ff7036ab640 1 -- 172.21.15.22:0/2212000592 >> 172.21.15.22:0/2212000592 conn(0x7ff6f4055c60 msgr2=0x7ff6f40e2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:57.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7ff7036ab640 1 -- 172.21.15.22:0/2212000592 shutdown_connections 2024-03-29T04:17:57.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7ff7036ab640 1 -- 172.21.15.22:0/2212000592 wait complete. 2024-03-29T04:17:57.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7fba9e1f2640 1 -- 172.21.15.22:0/4159707612 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fba7c067fc0 msgr2=0x7fba7c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:57.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7fba9e1f2640 1 --2- 172.21.15.22:0/4159707612 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fba7c067fc0 0x7fba7c06a480 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fba94006f50 tx=0x7fba94009240 comp rx=0 tx=0).stop 2024-03-29T04:17:57.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7fba9e1f2640 1 -- 172.21.15.22:0/4159707612 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba900558a0 msgr2=0x7fba900df4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:57.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7fba9e1f2640 1 --2- 172.21.15.22:0/4159707612 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba900558a0 0x7fba900df4c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fba9000c1a0 tx=0x7fba880191f0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7fba9e1f2640 1 -- 172.21.15.22:0/4159707612 shutdown_connections 2024-03-29T04:17:57.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7fba9e1f2640 1 --2- 172.21.15.22:0/4159707612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba900e1320 0x7fba9004de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7fba9e1f2640 1 --2- 172.21.15.22:0/4159707612 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fba900558a0 0x7fba900df4c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7fba9e1f2640 1 --2- 172.21.15.22:0/4159707612 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fba7c067fc0 0x7fba7c06a480 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7fba9e1f2640 1 --2- 172.21.15.22:0/4159707612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba90052710 0x7fba900dda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:57.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7fba9e1f2640 1 -- 172.21.15.22:0/4159707612 >> 172.21.15.22:0/4159707612 conn(0x7fba90020050 msgr2=0x7fba900da5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:57.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7fba9e1f2640 1 -- 172.21.15.22:0/4159707612 shutdown_connections 2024-03-29T04:17:57.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:57.200+0000 7fba9e1f2640 1 -- 172.21.15.22:0/4159707612 wait complete. 2024-03-29T04:17:57.193 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.3 2024-03-29T04:17:57.194 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2024-03-29T04:17:57.203 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.2 2024-03-29T04:17:57.205 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2024-03-29T04:17:58.128 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-29T04:17:58.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-29T04:17:58.177 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-29T04:17:58.179 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-29T04:17:58.194 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-29T04:17:58.195 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-29T04:17:58.204 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-29T04:17:58.205 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-29T04:17:58.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.304+0000 7f475bd95640 1 Processor -- start 2024-03-29T04:17:58.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.304+0000 7f475bd95640 1 -- start start 2024-03-29T04:17:58.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.308+0000 7f475bd95640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f475415a2c0 0x7f475406f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.308+0000 7f475bd95640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f475415a6a0 0x7f4754072810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.308+0000 7f475bd95640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47540b32f0 con 0x7f475415aac0 2024-03-29T04:17:58.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.308+0000 7f475bd95640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f475406f9a0 con 0x7f475415a2c0 2024-03-29T04:17:58.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.308+0000 7f475bd95640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f47540b3e70 con 0x7f475415a6a0 2024-03-29T04:17:58.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.308+0000 7f4759330640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f475415a6a0 0x7f4754072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.308+0000 7f4759b31640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f475415a2c0 0x7f475406f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.308+0000 7f4759330640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f475415a6a0 0x7f4754072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.22:53052/0 (socket says 172.21.15.22:53052) 2024-03-29T04:17:58.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.308+0000 7f4759330640 1 -- 172.21.15.22:0/3377731857 learned_addr learned my addr 172.21.15.22:0/3377731857 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:58.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.308+0000 7f4759b31640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f475415a2c0 0x7f475406f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:46586/0 (socket says 172.21.15.22:46586) 2024-03-29T04:17:58.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.308+0000 7f475a332640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f475415aac0 0x7f475406d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.22:60106/0 (socket says 172.21.15.22:60106) 2024-03-29T04:17:58.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.308+0000 7f4759330640 1 -- 172.21.15.22:0/3377731857 >> v2:172.21.15.151:3300/0 conn(0x7f475415a2c0 msgr2=0x7f475406f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:58.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.308+0000 7f4759330640 1 --2- 172.21.15.22:0/3377731857 >> v2:172.21.15.151:3300/0 conn(0x7f475415a2c0 0x7f475406f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.308+0000 7f4759330640 1 -- 172.21.15.22:0/3377731857 >> v1:172.21.15.151:6789/0 conn(0x7f475415aac0 legacy=0x7f475406d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.308+0000 7f4759330640 1 -- 172.21.15.22:0/3377731857 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47540b35d0 con 0x7f475415a6a0 2024-03-29T04:17:58.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.312+0000 7f4759330640 1 --2- 172.21.15.22:0/3377731857 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f475415a6a0 0x7f4754072810 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f474c00cf90 tx=0x7f474c01eb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e55ba1433005a3c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.312+0000 7f4758b2f640 1 -- 172.21.15.22:0/3377731857 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c00c2d0 con 0x7f475415a6a0 2024-03-29T04:17:58.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.312+0000 7f4758b2f640 1 -- 172.21.15.22:0/3377731857 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f474c0a7a60 con 0x7f475415a6a0 2024-03-29T04:17:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.312+0000 7f475bd95640 1 -- 172.21.15.22:0/3377731857 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f475415a6a0 msgr2=0x7f4754072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.312+0000 7f475bd95640 1 --2- 172.21.15.22:0/3377731857 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f475415a6a0 0x7f4754072810 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f474c00cf90 tx=0x7f474c01eb90 comp rx=0 tx=0).stop 2024-03-29T04:17:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.312+0000 7f475bd95640 1 -- 172.21.15.22:0/3377731857 shutdown_connections 2024-03-29T04:17:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.312+0000 7f475bd95640 1 --2- 172.21.15.22:0/3377731857 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f475415a6a0 0x7f4754072810 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.312+0000 7f475bd95640 1 --2- 172.21.15.22:0/3377731857 >> v2:172.21.15.151:3300/0 conn(0x7f475415a2c0 0x7f475406f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.312+0000 7f475bd95640 1 -- 172.21.15.22:0/3377731857 >> 172.21.15.22:0/3377731857 conn(0x7f475406c570 msgr2=0x7f475406c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.312+0000 7f475bd95640 1 -- 172.21.15.22:0/3377731857 shutdown_connections 2024-03-29T04:17:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.312+0000 7f475bd95640 1 -- 172.21.15.22:0/3377731857 wait complete. 2024-03-29T04:17:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.316+0000 7f475bd95640 1 Processor -- start 2024-03-29T04:17:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.316+0000 7f475bd95640 1 -- start start 2024-03-29T04:17:58.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.316+0000 7f475bd95640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f475415a2c0 0x7f4754164350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.316+0000 7f475bd95640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f475415a6a0 0x7f4754164890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.316+0000 7f475bd95640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f475415aac0 0x7f475414c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.316+0000 7f475bd95640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f47540b2e80 con 0x7f475415a6a0 2024-03-29T04:17:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.316+0000 7f475bd95640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4754073160 con 0x7f475415aac0 2024-03-29T04:17:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.316+0000 7f475bd95640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f47540b3000 con 0x7f475415a2c0 2024-03-29T04:17:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.316+0000 7f4759330640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f475415a6a0 0x7f4754164890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.316+0000 7f4759330640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f475415a6a0 0x7f4754164890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51578/0 (socket says 172.21.15.22:51578) 2024-03-29T04:17:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.316+0000 7f4759330640 1 -- 172.21.15.22:0/3833804544 learned_addr learned my addr 172.21.15.22:0/3833804544 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.316+0000 7f475a332640 1 --2- 172.21.15.22:0/3833804544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f475415aac0 0x7f475414c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.316+0000 7f4759b31640 1 --2- 172.21.15.22:0/3833804544 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f475415a2c0 0x7f4754164350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.316+0000 7f4759330640 1 -- 172.21.15.22:0/3833804544 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f475415a2c0 msgr2=0x7f4754164350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.316+0000 7f4759330640 1 --2- 172.21.15.22:0/3833804544 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f475415a2c0 0x7f4754164350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.316+0000 7f4759330640 1 -- 172.21.15.22:0/3833804544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f475415aac0 msgr2=0x7f475414c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.316+0000 7f4759330640 1 --2- 172.21.15.22:0/3833804544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f475415aac0 0x7f475414c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.316+0000 7f4759330640 1 -- 172.21.15.22:0/3833804544 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47540cec60 con 0x7f475415a6a0 2024-03-29T04:17:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.316+0000 7f4759330640 1 --2- 172.21.15.22:0/3833804544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f475415a6a0 0x7f4754164890 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f474c01d240 tx=0x7f474c01b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.320+0000 7f474a7fc640 1 -- 172.21.15.22:0/3833804544 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c0afdf0 con 0x7f475415a6a0 2024-03-29T04:17:58.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.324+0000 7f475bd95640 1 -- 172.21.15.22:0/3833804544 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4754150560 con 0x7f475415a6a0 2024-03-29T04:17:58.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.324+0000 7f475bd95640 1 -- 172.21.15.22:0/3833804544 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47540cf1c0 con 0x7f475415a6a0 2024-03-29T04:17:58.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.324+0000 7f474a7fc640 1 -- 172.21.15.22:0/3833804544 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f474c0be070 con 0x7f475415a6a0 2024-03-29T04:17:58.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.324+0000 7f474a7fc640 1 -- 172.21.15.22:0/3833804544 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c0b9640 con 0x7f475415a6a0 2024-03-29T04:17:58.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.324+0000 7f474a7fc640 1 -- 172.21.15.22:0/3833804544 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f474c00c2e0 con 0x7f475415a6a0 2024-03-29T04:17:58.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.328+0000 7f474a7fc640 1 --2- 172.21.15.22:0/3833804544 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4740068230 0x7f474006a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.328+0000 7f474a7fc640 1 -- 172.21.15.22:0/3833804544 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f474c122290 con 0x7f475415a6a0 2024-03-29T04:17:58.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.328+0000 7f4759b31640 1 --2- 172.21.15.22:0/3833804544 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4740068230 0x7f474006a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.328+0000 7f475bd95640 1 -- 172.21.15.22:0/3833804544 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f473c0052a0 con 0x7f475415a6a0 2024-03-29T04:17:58.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.336+0000 7f474a7fc640 1 -- 172.21.15.22:0/3833804544 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f47540cec60 con 0x7f475415a6a0 2024-03-29T04:17:58.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.340+0000 7f4759b31640 1 --2- 172.21.15.22:0/3833804544 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4740068230 0x7f474006a6f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f47541642e0 tx=0x7f475000f040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.356+0000 7f755ca83640 1 Processor -- start 2024-03-29T04:17:58.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.356+0000 7f755ca83640 1 -- start start 2024-03-29T04:17:58.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.356+0000 7f755ca83640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f755815a2c0 0x7f755806f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.356+0000 7f755ca83640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7558072f20 0x7f755806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.356+0000 7f755ca83640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75580b32f0 con 0x7f755815a6a0 2024-03-29T04:17:58.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.356+0000 7f755ca83640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f755806f9a0 con 0x7f7558072f20 2024-03-29T04:17:58.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.356+0000 7f755ca83640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f75580b3e70 con 0x7f755815a2c0 2024-03-29T04:17:58.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.356+0000 7f7555d9b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f755815a6a0 0x7f7558072810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44252/0 (socket says 172.21.15.22:44252) 2024-03-29T04:17:58.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.356+0000 7f7555d9b640 1 -- 172.21.15.22:0/1640433311 learned_addr learned my addr 172.21.15.22:0/1640433311 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:58.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.356+0000 7f755659c640 1 --2- 172.21.15.22:0/1640433311 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f755815a2c0 0x7f755806f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.356+0000 7f7556d9d640 1 --2- 172.21.15.22:0/1640433311 >> v2:172.21.15.22:3300/0 conn(0x7f7558072f20 0x7f755806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.356+0000 7f755559a640 1 -- 172.21.15.22:0/1640433311 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 977657408 0 0) 0x7f75580b32f0 con 0x7f755815a6a0 2024-03-29T04:17:58.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.356+0000 7f755559a640 1 -- 172.21.15.22:0/1640433311 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7540003530 con 0x7f755815a6a0 2024-03-29T04:17:58.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.356+0000 7f7556d9d640 1 -- 172.21.15.22:0/1640433311 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f755815a2c0 msgr2=0x7f755806f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:58.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.356+0000 7f7556d9d640 1 --2- 172.21.15.22:0/1640433311 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f755815a2c0 0x7f755806f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.360+0000 7f7556d9d640 1 -- 172.21.15.22:0/1640433311 >> v1:172.21.15.22:6789/0 conn(0x7f755815a6a0 legacy=0x7f7558072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.360+0000 7f7556d9d640 1 -- 172.21.15.22:0/1640433311 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75580b35d0 con 0x7f7558072f20 2024-03-29T04:17:58.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.360+0000 7f755659c640 1 --2- 172.21.15.22:0/1640433311 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f755815a2c0 0x7f755806f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:17:58.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.360+0000 7f7556d9d640 1 --2- 172.21.15.22:0/1640433311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7558072f20 0x7f755806d9e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f754c0097c0 tx=0x7f754c01d510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2607255b262f079c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.360+0000 7f755559a640 1 -- 172.21.15.22:0/1640433311 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754c0ab660 con 0x7f7558072f20 2024-03-29T04:17:58.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.360+0000 7f755559a640 1 -- 172.21.15.22:0/1640433311 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f754c0b0070 con 0x7f7558072f20 2024-03-29T04:17:58.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.364+0000 7f755ca83640 1 -- 172.21.15.22:0/1640433311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7558072f20 msgr2=0x7f755806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.364+0000 7f755ca83640 1 --2- 172.21.15.22:0/1640433311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7558072f20 0x7f755806d9e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f754c0097c0 tx=0x7f754c01d510 comp rx=0 tx=0).stop 2024-03-29T04:17:58.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.364+0000 7f755ca83640 1 -- 172.21.15.22:0/1640433311 shutdown_connections 2024-03-29T04:17:58.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.364+0000 7f755ca83640 1 --2- 172.21.15.22:0/1640433311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7558072f20 0x7f755806d9e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.364+0000 7f755ca83640 1 --2- 172.21.15.22:0/1640433311 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f755815a2c0 0x7f755806f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.364+0000 7f755ca83640 1 -- 172.21.15.22:0/1640433311 >> 172.21.15.22:0/1640433311 conn(0x7f755806c570 msgr2=0x7f755806c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:58.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.364+0000 7f755ca83640 1 -- 172.21.15.22:0/1640433311 shutdown_connections 2024-03-29T04:17:58.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.364+0000 7faef72c2640 1 Processor -- start 2024-03-29T04:17:58.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.364+0000 7f755ca83640 1 -- 172.21.15.22:0/1640433311 wait complete. 2024-03-29T04:17:58.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.364+0000 7f755ca83640 1 Processor -- start 2024-03-29T04:17:58.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.364+0000 7faef72c2640 1 -- start start 2024-03-29T04:17:58.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.364+0000 7f755ca83640 1 -- start start 2024-03-29T04:17:58.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.364+0000 7faef72c2640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7faef006f460 0x7faef015a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.364+0000 7faef72c2640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7faef006e0f0 0x7faef00723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.364+0000 7faef72c2640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faef00b32f0 con 0x7faef015a800 2024-03-29T04:17:58.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.364+0000 7faef72c2640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7faef00b3e70 con 0x7faef006f460 2024-03-29T04:17:58.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.364+0000 7faef72c2640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7faef00b35d0 con 0x7faef006e0f0 2024-03-29T04:17:58.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7faef505e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7faef006f460 0x7faef015a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7faef505e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7faef006f460 0x7faef015a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51594/0 (socket says 172.21.15.22:51594) 2024-03-29T04:17:58.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7faef505e640 1 -- 172.21.15.22:0/2780030513 learned_addr learned my addr 172.21.15.22:0/2780030513 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:58.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7f755ca83640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7558072f20 0x7f7558154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7f755ca83640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f755815a2c0 0x7f75580ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7f755ca83640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f755815a6a0 0x7f7558207f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7f755ca83640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f75580b2e80 con 0x7f755815a2c0 2024-03-29T04:17:58.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7f755ca83640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f75580b3000 con 0x7f7558072f20 2024-03-29T04:17:58.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7f755ca83640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7558167250 con 0x7f755815a6a0 2024-03-29T04:17:58.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7f755659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7558072f20 0x7f7558154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7f7555d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f755815a2c0 0x7f75580ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7f755659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7558072f20 0x7f7558154810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:46600/0 (socket says 172.21.15.22:46600) 2024-03-29T04:17:58.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7f755659c640 1 -- 172.21.15.22:0/1545315361 learned_addr learned my addr 172.21.15.22:0/1545315361 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:58.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7f7556d9d640 1 --2- 172.21.15.22:0/1545315361 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f755815a6a0 0x7f7558207f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7f7555d9b640 1 -- 172.21.15.22:0/1545315361 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f755815a6a0 msgr2=0x7f7558207f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7f7555d9b640 1 --2- 172.21.15.22:0/1545315361 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f755815a6a0 0x7f7558207f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7f7555d9b640 1 -- 172.21.15.22:0/1545315361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7558072f20 msgr2=0x7f7558154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7f7555d9b640 1 --2- 172.21.15.22:0/1545315361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7558072f20 0x7f7558154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7f7555d9b640 1 -- 172.21.15.22:0/1545315361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75580aa2c0 con 0x7f755815a2c0 2024-03-29T04:17:58.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7f7555d9b640 1 --2- 172.21.15.22:0/1545315361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f755815a2c0 0x7f75580ae0f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f754800c400 tx=0x7f754800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7faef505e640 1 -- 172.21.15.22:0/2780030513 >> v2:172.21.15.151:3300/0 conn(0x7faef006e0f0 msgr2=0x7faef00723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:58.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7faef505e640 1 --2- 172.21.15.22:0/2780030513 >> v2:172.21.15.151:3300/0 conn(0x7faef006e0f0 0x7faef00723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7faef505e640 1 -- 172.21.15.22:0/2780030513 >> v1:172.21.15.151:6789/0 conn(0x7faef015a800 legacy=0x7faef006d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:58.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7faef505e640 1 -- 172.21.15.22:0/2780030513 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef00b38b0 con 0x7faef006f460 2024-03-29T04:17:58.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7f7546ffd640 1 -- 172.21.15.22:0/1545315361 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548017070 con 0x7f755815a2c0 2024-03-29T04:17:58.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7faef505e640 1 --2- 172.21.15.22:0/2780030513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faef006f460 0x7faef015a2c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faef0073ab0 tx=0x7faee801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e68e9d6da19649 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7f7546ffd640 1 -- 172.21.15.22:0/1545315361 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7548005cf0 con 0x7f755815a2c0 2024-03-29T04:17:58.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.368+0000 7f7546ffd640 1 -- 172.21.15.22:0/1545315361 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75480072d0 con 0x7f755815a2c0 2024-03-29T04:17:58.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.372+0000 7f755ca83640 1 -- 172.21.15.22:0/1545315361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75580aa5a0 con 0x7f755815a2c0 2024-03-29T04:17:58.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.372+0000 7f755ca83640 1 -- 172.21.15.22:0/1545315361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75580aab60 con 0x7f755815a2c0 2024-03-29T04:17:58.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.372+0000 7faee7fff640 1 -- 172.21.15.22:0/2780030513 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee80a5750 con 0x7faef006f460 2024-03-29T04:17:58.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.372+0000 7faee7fff640 1 -- 172.21.15.22:0/2780030513 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee80a58f0 con 0x7faef006f460 2024-03-29T04:17:58.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.372+0000 7f755ca83640 1 -- 172.21.15.22:0/1545315361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75580743d0 con 0x7f755815a2c0 2024-03-29T04:17:58.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.372+0000 7f7546ffd640 1 -- 172.21.15.22:0/1545315361 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f754801a070 con 0x7f755815a2c0 2024-03-29T04:17:58.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.372+0000 7faef72c2640 1 -- 172.21.15.22:0/2780030513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faef006f460 msgr2=0x7faef015a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.372+0000 7faef72c2640 1 --2- 172.21.15.22:0/2780030513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faef006f460 0x7faef015a2c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faef0073ab0 tx=0x7faee801c980 comp rx=0 tx=0).stop 2024-03-29T04:17:58.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.372+0000 7faef72c2640 1 -- 172.21.15.22:0/2780030513 shutdown_connections 2024-03-29T04:17:58.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.372+0000 7faef72c2640 1 --2- 172.21.15.22:0/2780030513 >> v2:172.21.15.151:3300/0 conn(0x7faef006e0f0 0x7faef00723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.372+0000 7faef72c2640 1 --2- 172.21.15.22:0/2780030513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faef006f460 0x7faef015a2c0 secure :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faef0073ab0 tx=0x7faee801c980 comp rx=0 tx=0).stop 2024-03-29T04:17:58.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.372+0000 7faef72c2640 1 -- 172.21.15.22:0/2780030513 >> 172.21.15.22:0/2780030513 conn(0x7faef006c570 msgr2=0x7faef006c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:58.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.372+0000 7f7546ffd640 1 --2- 172.21.15.22:0/1545315361 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7538068010 0x7f753806a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.372+0000 7faef72c2640 1 -- 172.21.15.22:0/2780030513 shutdown_connections 2024-03-29T04:17:58.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.372+0000 7f7546ffd640 1 -- 172.21.15.22:0/1545315361 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7548085bb0 con 0x7f755815a2c0 2024-03-29T04:17:58.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.372+0000 7f755659c640 1 --2- 172.21.15.22:0/1545315361 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7538068010 0x7f753806a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7f755659c640 1 --2- 172.21.15.22:0/1545315361 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7538068010 0x7f753806a4d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7550009c50 tx=0x7f755000d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.372+0000 7faef72c2640 1 -- 172.21.15.22:0/2780030513 wait complete. 2024-03-29T04:17:58.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7feb66f00640 1 Processor -- start 2024-03-29T04:17:58.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7faef72c2640 1 Processor -- start 2024-03-29T04:17:58.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7feb66f00640 1 -- start start 2024-03-29T04:17:58.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7feb66f00640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7feb58047c20 0x7feb580eb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7feb66f00640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7feb58005ea0 0x7feb580062b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7feb66f00640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb58034220 con 0x7feb58033e40 2024-03-29T04:17:58.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7feb66f00640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7feb58050210 con 0x7feb58005ea0 2024-03-29T04:17:58.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7feb66f00640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7feb58037bc0 con 0x7feb58047c20 2024-03-29T04:17:58.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7faef72c2640 1 -- start start 2024-03-29T04:17:58.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7feb666ff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7feb58005ea0 0x7feb580062b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7feb666ff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7feb58005ea0 0x7feb580062b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51610/0 (socket says 172.21.15.22:51610) 2024-03-29T04:17:58.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7faef72c2640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faef006e0f0 0x7faef0158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7feb666ff640 1 -- 172.21.15.22:0/2645477433 learned_addr learned my addr 172.21.15.22:0/2645477433 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:58.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7faef72c2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faef015a800 0x7faef0156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7faef72c2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faef015d2d0 0x7faef015c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7faef72c2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faef015ada0 con 0x7faef015d2d0 2024-03-29T04:17:58.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7faef505e640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faef006e0f0 0x7faef0158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7faef72c2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faef015ac20 con 0x7faef015a800 2024-03-29T04:17:58.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7faef72c2640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7faef006e500 con 0x7faef006e0f0 2024-03-29T04:17:58.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7faef505e640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faef006e0f0 0x7faef0158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.22:53092/0 (socket says 172.21.15.22:53092) 2024-03-29T04:17:58.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7faef505e640 1 -- 172.21.15.22:0/3724260380 learned_addr learned my addr 172.21.15.22:0/3724260380 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:58.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7faef485d640 1 --2- 172.21.15.22:0/3724260380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faef015a800 0x7faef0156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7faef585f640 1 --2- 172.21.15.22:0/3724260380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faef015d2d0 0x7faef015c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7faef585f640 1 -- 172.21.15.22:0/3724260380 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faef006e0f0 msgr2=0x7faef0158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7faef585f640 1 --2- 172.21.15.22:0/3724260380 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faef006e0f0 0x7faef0158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7faef585f640 1 -- 172.21.15.22:0/3724260380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faef015a800 msgr2=0x7faef0156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7faef585f640 1 --2- 172.21.15.22:0/3724260380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faef015a800 0x7faef0156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7faef585f640 1 -- 172.21.15.22:0/3724260380 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef01570d0 con 0x7faef015d2d0 2024-03-29T04:17:58.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7feb666ff640 1 -- 172.21.15.22:0/2645477433 >> v2:172.21.15.151:3300/0 conn(0x7feb58047c20 msgr2=0x7feb580eb890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T04:17:58.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7feb666ff640 1 --2- 172.21.15.22:0/2645477433 >> v2:172.21.15.151:3300/0 conn(0x7feb58047c20 0x7feb580eb890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7feb666ff640 1 -- 172.21.15.22:0/2645477433 >> v1:172.21.15.151:6789/0 conn(0x7feb58033e40 legacy=0x7feb580eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:58.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7feb666ff640 1 -- 172.21.15.22:0/2645477433 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb5804fa30 con 0x7feb58005ea0 2024-03-29T04:17:58.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.376+0000 7faef585f640 1 --2- 172.21.15.22:0/3724260380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faef015d2d0 0x7faef015c770 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faee000b3c0 tx=0x7faee000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.380+0000 7feb666ff640 1 --2- 172.21.15.22:0/2645477433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb58005ea0 0x7feb580062b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7feb6007ae00 tx=0x7feb6009a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9259a87c30dacb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.380+0000 7f7546ffd640 1 -- 172.21.15.22:0/1545315361 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f75480559b0 con 0x7f755815a2c0 2024-03-29T04:17:58.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.380+0000 7faee5ffb640 1 -- 172.21.15.22:0/3724260380 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0011830 con 0x7faef015d2d0 2024-03-29T04:17:58.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.380+0000 7faef72c2640 1 -- 172.21.15.22:0/3724260380 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef01573b0 con 0x7faef015d2d0 2024-03-29T04:17:58.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.380+0000 7faef72c2640 1 -- 172.21.15.22:0/3724260380 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef0157970 con 0x7faef015d2d0 2024-03-29T04:17:58.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.380+0000 7feb64efc640 1 -- 172.21.15.22:0/2645477433 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6012f070 con 0x7feb58005ea0 2024-03-29T04:17:58.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.380+0000 7feb64efc640 1 -- 172.21.15.22:0/2645477433 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb6012a600 con 0x7feb58005ea0 2024-03-29T04:17:58.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.380+0000 7feb64efc640 1 -- 172.21.15.22:0/2645477433 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6012aa70 con 0x7feb58005ea0 2024-03-29T04:17:58.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.380+0000 7faee5ffb640 1 -- 172.21.15.22:0/3724260380 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee0016070 con 0x7faef015d2d0 2024-03-29T04:17:58.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.380+0000 7faee5ffb640 1 -- 172.21.15.22:0/3724260380 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0011c10 con 0x7faef015d2d0 2024-03-29T04:17:58.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.384+0000 7feb66f00640 1 -- 172.21.15.22:0/2645477433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb58005ea0 msgr2=0x7feb580062b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.384+0000 7feb66f00640 1 --2- 172.21.15.22:0/2645477433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb58005ea0 0x7feb580062b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7feb6007ae00 tx=0x7feb6009a410 comp rx=0 tx=0).stop 2024-03-29T04:17:58.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.384+0000 7feb66f00640 1 -- 172.21.15.22:0/2645477433 shutdown_connections 2024-03-29T04:17:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.384+0000 7feb66f00640 1 --2- 172.21.15.22:0/2645477433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb58005ea0 0x7feb580062b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.384+0000 7feb66f00640 1 --2- 172.21.15.22:0/2645477433 >> v2:172.21.15.151:3300/0 conn(0x7feb58047c20 0x7feb580eb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.384+0000 7feb66f00640 1 -- 172.21.15.22:0/2645477433 >> 172.21.15.22:0/2645477433 conn(0x7feb58009fc0 msgr2=0x7feb5800a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.384+0000 7feb66f00640 1 -- 172.21.15.22:0/2645477433 shutdown_connections 2024-03-29T04:17:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.384+0000 7feb66f00640 1 -- 172.21.15.22:0/2645477433 wait complete. 2024-03-29T04:17:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.384+0000 7feb66f00640 1 Processor -- start 2024-03-29T04:17:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.384+0000 7faee5ffb640 1 -- 172.21.15.22:0/3724260380 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7faee001b460 con 0x7faef015d2d0 2024-03-29T04:17:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.384+0000 7faee5ffb640 1 --2- 172.21.15.22:0/3724260380 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faed4067fc0 0x7faed406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.384+0000 7faee5ffb640 1 -- 172.21.15.22:0/3724260380 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7faee0086430 con 0x7faef015d2d0 2024-03-29T04:17:58.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.384+0000 7faef505e640 1 --2- 172.21.15.22:0/3724260380 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faed4067fc0 0x7faed406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.384+0000 7faef72c2640 1 -- 172.21.15.22:0/3724260380 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faed00052a0 con 0x7faef015d2d0 2024-03-29T04:17:58.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.384+0000 7feb66f00640 1 -- start start 2024-03-29T04:17:58.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7feb66f00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb58005ea0 0x7feb580cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7faef505e640 1 --2- 172.21.15.22:0/3724260380 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faed4067fc0 0x7faed406a480 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7faee801c9c0 tx=0x7faee80a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7feb65efe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb58005ea0 0x7feb580cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7feb65efe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb58005ea0 0x7feb580cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51626/0 (socket says 172.21.15.22:51626) 2024-03-29T04:17:58.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7feb66f00640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb58033e40 0x7feb580d4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7feb65efe640 1 -- 172.21.15.22:0/1739452924 learned_addr learned my addr 172.21.15.22:0/1739452924 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:58.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7feb66f00640 1 --2- 172.21.15.22:0/1739452924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb58047c20 0x7feb58006dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7feb66f00640 1 -- 172.21.15.22:0/1739452924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7feb5804fd80 con 0x7feb58005ea0 2024-03-29T04:17:58.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7feb66f00640 1 -- 172.21.15.22:0/1739452924 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feb5804f090 con 0x7feb58047c20 2024-03-29T04:17:58.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7feb66f00640 1 -- 172.21.15.22:0/1739452924 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7feb5804f5c0 con 0x7feb58033e40 2024-03-29T04:17:58.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7feb656fd640 1 --2- 172.21.15.22:0/1739452924 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb58033e40 0x7feb580d4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7feb656fd640 1 -- 172.21.15.22:0/1739452924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb58047c20 msgr2=0x7feb58006dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T04:17:58.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7feb656fd640 1 --2- 172.21.15.22:0/1739452924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb58047c20 0x7feb58006dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7feb656fd640 1 -- 172.21.15.22:0/1739452924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb58005ea0 msgr2=0x7feb580cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7feb656fd640 1 --2- 172.21.15.22:0/1739452924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb58005ea0 0x7feb580cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7feb656fd640 1 -- 172.21.15.22:0/1739452924 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb580d5910 con 0x7feb58033e40 2024-03-29T04:17:58.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7feb656fd640 1 --2- 172.21.15.22:0/1739452924 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb58033e40 0x7feb580d4a20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7feb5000b410 tx=0x7feb5000b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7feb567fc640 1 -- 172.21.15.22:0/1739452924 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50015020 con 0x7feb58033e40 2024-03-29T04:17:58.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7feb66f00640 1 -- 172.21.15.22:0/1739452924 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb580d5bf0 con 0x7feb58033e40 2024-03-29T04:17:58.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7feb66f00640 1 -- 172.21.15.22:0/1739452924 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feb580487d0 con 0x7feb58033e40 2024-03-29T04:17:58.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7feb567fc640 1 -- 172.21.15.22:0/1739452924 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb50011850 con 0x7feb58033e40 2024-03-29T04:17:58.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.388+0000 7feb567fc640 1 -- 172.21.15.22:0/1739452924 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50011c40 con 0x7feb58033e40 2024-03-29T04:17:58.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.392+0000 7feb66f00640 1 -- 172.21.15.22:0/1739452924 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb580489d0 con 0x7feb58033e40 2024-03-29T04:17:58.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.392+0000 7feb567fc640 1 -- 172.21.15.22:0/1739452924 <== mon.2 v2:172.21.15.151:3301/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7feb50018020 con 0x7feb58033e40 2024-03-29T04:17:58.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.392+0000 7feb567fc640 1 --2- 172.21.15.22:0/1739452924 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7feb44067ff0 0x7feb4406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.392+0000 7feb567fc640 1 -- 172.21.15.22:0/1739452924 <== mon.2 v2:172.21.15.151:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7feb50086b70 con 0x7feb58033e40 2024-03-29T04:17:58.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.392+0000 7feb65efe640 1 --2- 172.21.15.22:0/1739452924 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7feb44067ff0 0x7feb4406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.396+0000 7feb65efe640 1 --2- 172.21.15.22:0/1739452924 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7feb44067ff0 0x7feb4406a4b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7feb580d5f80 tx=0x7feb5c003700 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.396+0000 7faee5ffb640 1 -- 172.21.15.22:0/3724260380 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7faee00561b0 con 0x7faef015d2d0 2024-03-29T04:17:58.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.408+0000 7feb567fc640 1 -- 172.21.15.22:0/1739452924 <== mon.2 v2:172.21.15.151:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7feb580489d0 con 0x7feb58033e40 2024-03-29T04:17:58.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.420+0000 7f413c440640 1 Processor -- start 2024-03-29T04:17:58.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.420+0000 7f413c440640 1 -- start start 2024-03-29T04:17:58.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.420+0000 7f413c440640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f413415a2c0 0x7f413406f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.420+0000 7f413c440640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f4134072f20 0x7f413406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.420+0000 7f413c440640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41340b32f0 con 0x7f413415a6a0 2024-03-29T04:17:58.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.420+0000 7f413c440640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f413406f9a0 con 0x7f4134072f20 2024-03-29T04:17:58.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.420+0000 7f413c440640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f41340b3e70 con 0x7f413415a2c0 2024-03-29T04:17:58.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.420+0000 7f41399db640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f413415a6a0 0x7f4134072810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44262/0 (socket says 172.21.15.22:44262) 2024-03-29T04:17:58.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.420+0000 7f41399db640 1 -- 172.21.15.22:0/4030483347 learned_addr learned my addr 172.21.15.22:0/4030483347 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:58.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.420+0000 7f413a9dd640 1 --2- 172.21.15.22:0/4030483347 >> v2:172.21.15.151:3300/0 conn(0x7f4134072f20 0x7f413406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.420+0000 7f413a1dc640 1 --2- 172.21.15.22:0/4030483347 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f413415a2c0 0x7f413406f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.420+0000 7f41391da640 1 -- 172.21.15.22:0/4030483347 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1402753450 0 0) 0x7f41340b32f0 con 0x7f413415a6a0 2024-03-29T04:17:58.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.420+0000 7f41391da640 1 -- 172.21.15.22:0/4030483347 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4120003530 con 0x7f413415a6a0 2024-03-29T04:17:58.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.420+0000 7f413a9dd640 1 -- 172.21.15.22:0/4030483347 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f413415a2c0 msgr2=0x7f413406f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:58.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.420+0000 7f413a9dd640 1 --2- 172.21.15.22:0/4030483347 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f413415a2c0 0x7f413406f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.420+0000 7f413a9dd640 1 -- 172.21.15.22:0/4030483347 >> v1:172.21.15.22:6789/0 conn(0x7f413415a6a0 legacy=0x7f4134072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.420+0000 7f413a9dd640 1 -- 172.21.15.22:0/4030483347 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41340b35d0 con 0x7f4134072f20 2024-03-29T04:17:58.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f90865e0640 1 Processor -- start 2024-03-29T04:17:58.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a6e404640 1 Processor -- start 2024-03-29T04:17:58.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f90865e0640 1 -- start start 2024-03-29T04:17:58.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7ff7a4ca3640 1 Processor -- start 2024-03-29T04:17:58.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a6e404640 1 -- start start 2024-03-29T04:17:58.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a6e404640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a680b4450 0x7f8a680b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a6e404640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a6815b120 con 0x7f8a68157e50 2024-03-29T04:17:58.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a6e404640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a6815b400 con 0x7f8a68073300 2024-03-29T04:17:58.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a6e404640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8a6815e160 con 0x7f8a680b4450 2024-03-29T04:17:58.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f90865e0640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9078055260 0x7f90780355b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f90865e0640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9078036fa0 0x7f9078052590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a6d402640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f8a68157e50 0x7f8a6815a8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44274/0 (socket says 172.21.15.22:44274) 2024-03-29T04:17:58.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a6d402640 1 -- 172.21.15.22:0/3655317386 learned_addr learned my addr 172.21.15.22:0/3655317386 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:58.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f907ffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9078055260 0x7f90780355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f907ffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9078055260 0x7f90780355b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51630/0 (socket says 172.21.15.22:51630) 2024-03-29T04:17:58.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a6dc03640 1 --2- 172.21.15.22:0/3655317386 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a680b4450 0x7f8a680b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a5ffff640 1 -- 172.21.15.22:0/3655317386 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3384072010 0 0) 0x7f8a6815b120 con 0x7f8a68157e50 2024-03-29T04:17:58.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a5ffff640 1 -- 172.21.15.22:0/3655317386 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a54004640 con 0x7f8a68157e50 2024-03-29T04:17:58.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f90865e0640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9078055640 con 0x7f9078052ad0 2024-03-29T04:17:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f90865e0640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f907803b670 con 0x7f9078055260 2024-03-29T04:17:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f90865e0640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f907803aa60 con 0x7f9078036fa0 2024-03-29T04:17:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f9084b7d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f9078052ad0 0x7f90780df8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44290/0 (socket says 172.21.15.22:44290) 2024-03-29T04:17:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f9084b7d640 1 -- 172.21.15.22:0/1771841305 learned_addr learned my addr 172.21.15.22:0/1771841305 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f907f7fe640 1 --2- 172.21.15.22:0/1771841305 >> v2:172.21.15.151:3300/0 conn(0x7f9078036fa0 0x7f9078052590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f907effd640 1 -- 172.21.15.22:0/1771841305 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3229280435 0 0) 0x7f9078055640 con 0x7f9078052ad0 2024-03-29T04:17:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f907effd640 1 -- 172.21.15.22:0/1771841305 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9070003530 con 0x7f9078052ad0 2024-03-29T04:17:58.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7ff7a4ca3640 1 -- start start 2024-03-29T04:17:58.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a5ffff640 1 -- 172.21.15.22:0/3655317386 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 441634358 0 0) 0x7f8a54004640 con 0x7f8a68157e50 2024-03-29T04:17:58.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a5ffff640 1 -- 172.21.15.22:0/3655317386 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a680b4450 msgr2=0x7f8a680b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:58.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a5ffff640 1 --2- 172.21.15.22:0/3655317386 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a680b4450 0x7f8a680b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a5ffff640 1 -- 172.21.15.22:0/3655317386 >> v1:172.21.15.151:6789/0 conn(0x7f8a68073300 legacy=0x7f8a6815e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:58.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7ff7a4ca3640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff7980501b0 0x7ff798006540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7ff7a4ca3640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff798033d00 0x7ff798006a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7ff7a4ca3640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff798054120 con 0x7ff7980df2b0 2024-03-29T04:17:58.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7ff7a4ca3640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7ff7980df690 con 0x7ff7980501b0 2024-03-29T04:17:58.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7ff7a4ca3640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7ff798054400 con 0x7ff798033d00 2024-03-29T04:17:58.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f413a9dd640 1 --2- 172.21.15.22:0/4030483347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4134072f20 0x7f413406d9e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4124009fd0 tx=0x7f412401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c0542e1cac166e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a5ffff640 1 -- 172.21.15.22:0/3655317386 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6815b6e0 con 0x7f8a68157e50 2024-03-29T04:17:58.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a5ffff640 1 -- 172.21.15.22:0/3655317386 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f8a60005060 con 0x7f8a68157e50 2024-03-29T04:17:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a5ffff640 1 -- 172.21.15.22:0/3655317386 >> v1:172.21.15.22:6789/0 conn(0x7f8a68157e50 legacy=0x7f8a6815a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7ff79ffff640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff798033d00 0x7ff798006a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a5ffff640 1 --2- 172.21.15.22:0/3655317386 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a680b4450 0x7f8a54004b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a5ffff640 1 --2- 172.21.15.22:0/3655317386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a54005040 0x7f8a54005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a5ffff640 1 --2- 172.21.15.22:0/3655317386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a54005e60 0x7f8a540062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a5ffff640 1 -- 172.21.15.22:0/3655317386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6815e160 con 0x7f8a54005040 2024-03-29T04:17:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a5ffff640 1 -- 172.21.15.22:0/3655317386 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a54003560 con 0x7f8a54005e60 2024-03-29T04:17:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f8a5ffff640 1 -- 172.21.15.22:0/3655317386 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8a54003760 con 0x7f8a680b4450 2024-03-29T04:17:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7ff79ffff640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff798033d00 0x7ff798006a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:46646/0 (socket says 172.21.15.22:46646) 2024-03-29T04:17:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7ff79f7fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff7980df2b0 0x7ff79804faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44306/0 (socket says 172.21.15.22:44306) 2024-03-29T04:17:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7ff79ffff640 1 -- 172.21.15.22:0/1002608433 learned_addr learned my addr 172.21.15.22:0/1002608433 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f907ffff640 1 -- 172.21.15.22:0/1771841305 >> v2:172.21.15.151:3300/0 conn(0x7f9078036fa0 msgr2=0x7f9078052590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f907ffff640 1 --2- 172.21.15.22:0/1771841305 >> v2:172.21.15.151:3300/0 conn(0x7f9078036fa0 0x7f9078052590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f907ffff640 1 -- 172.21.15.22:0/1771841305 >> v1:172.21.15.22:6789/0 conn(0x7f9078052ad0 legacy=0x7f90780df8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f907ffff640 1 -- 172.21.15.22:0/1771841305 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907803ad40 con 0x7f9078055260 2024-03-29T04:17:58.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.424+0000 7f907f7fe640 1 --2- 172.21.15.22:0/1771841305 >> v2:172.21.15.151:3300/0 conn(0x7f9078036fa0 0x7f9078052590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:17:58.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7f41391da640 1 -- 172.21.15.22:0/4030483347 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41240b1070 con 0x7f4134072f20 2024-03-29T04:17:58.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7f41391da640 1 -- 172.21.15.22:0/4030483347 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41240a8050 con 0x7f4134072f20 2024-03-29T04:17:58.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7f41391da640 1 -- 172.21.15.22:0/4030483347 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41240ac840 con 0x7f4134072f20 2024-03-29T04:17:58.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7f8a6dc03640 1 --2- 172.21.15.22:0/3655317386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a54005e60 0x7f8a540062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7f8a6dc03640 1 -- 172.21.15.22:0/3655317386 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a680b4450 msgr2=0x7f8a54004b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:58.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7f8a6dc03640 1 --2- 172.21.15.22:0/3655317386 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a680b4450 0x7f8a54004b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7f8a6dc03640 1 -- 172.21.15.22:0/3655317386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a54005040 msgr2=0x7f8a54005920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:17:58.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7f8a6dc03640 1 --2- 172.21.15.22:0/3655317386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a54005040 0x7f8a54005920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7f8a6dc03640 1 -- 172.21.15.22:0/3655317386 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6815b6e0 con 0x7f8a54005e60 2024-03-29T04:17:58.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7f413c440640 1 -- 172.21.15.22:0/4030483347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4134072f20 msgr2=0x7f413406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7f413c440640 1 --2- 172.21.15.22:0/4030483347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4134072f20 0x7f413406d9e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4124009fd0 tx=0x7f412401c3e0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7ff79ffff640 1 -- 172.21.15.22:0/1002608433 >> v2:172.21.15.22:3300/0 conn(0x7ff7980501b0 msgr2=0x7ff798006540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:58.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7ff79e7fc640 1 -- 172.21.15.22:0/1002608433 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 717231376 0 0) 0x7ff798054120 con 0x7ff7980df2b0 2024-03-29T04:17:58.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7ff79effd640 1 --2- 172.21.15.22:0/1002608433 >> v2:172.21.15.22:3300/0 conn(0x7ff7980501b0 0x7ff798006540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7f413c440640 1 -- 172.21.15.22:0/4030483347 shutdown_connections 2024-03-29T04:17:58.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7f413c440640 1 --2- 172.21.15.22:0/4030483347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4134072f20 0x7f413406d9e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7f413c440640 1 --2- 172.21.15.22:0/4030483347 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f413415a2c0 0x7f413406f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7f413c440640 1 -- 172.21.15.22:0/4030483347 >> 172.21.15.22:0/4030483347 conn(0x7f413406c570 msgr2=0x7f413406c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:58.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7f413c440640 1 -- 172.21.15.22:0/4030483347 shutdown_connections 2024-03-29T04:17:58.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7f413c440640 1 -- 172.21.15.22:0/4030483347 wait complete. 2024-03-29T04:17:58.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f413c440640 1 Processor -- start 2024-03-29T04:17:58.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f413c440640 1 -- start start 2024-03-29T04:17:58.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7ff79ffff640 1 --2- 172.21.15.22:0/1002608433 >> v2:172.21.15.22:3300/0 conn(0x7ff7980501b0 0x7ff798006540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7ff79ffff640 1 -- 172.21.15.22:0/1002608433 >> v1:172.21.15.22:6789/0 conn(0x7ff7980df2b0 legacy=0x7ff79804faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.428+0000 7ff79ffff640 1 -- 172.21.15.22:0/1002608433 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7980546e0 con 0x7ff798033d00 2024-03-29T04:17:58.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7ff79effd640 1 --2- 172.21.15.22:0/1002608433 >> v2:172.21.15.22:3300/0 conn(0x7ff7980501b0 0x7ff798006540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:58.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f8a6dc03640 1 --2- 172.21.15.22:0/3655317386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a54005e60 0x7f8a540062e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400a990 tx=0x7f8a6401c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=905a1778d4101397 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f907ffff640 1 --2- 172.21.15.22:0/1771841305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9078055260 0x7f90780355b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f907803bae0 tx=0x7f907401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac17cbc5ec33325 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f8a5ffff640 1 -- 172.21.15.22:0/3655317386 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6400a590 con 0x7f8a54005e60 2024-03-29T04:17:58.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7ff79ffff640 1 --2- 172.21.15.22:0/1002608433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff798033d00 0x7ff798006a80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0069df0 tx=0x7ff7a009a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=510b370a2a69609e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f8a5ffff640 1 -- 172.21.15.22:0/3655317386 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a640b0070 con 0x7f8a54005e60 2024-03-29T04:17:58.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f8a5ffff640 1 -- 172.21.15.22:0/3655317386 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a640ab850 con 0x7f8a54005e60 2024-03-29T04:17:58.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7ff79e7fc640 1 -- 172.21.15.22:0/1002608433 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0128070 con 0x7ff798033d00 2024-03-29T04:17:58.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7ff79e7fc640 1 -- 172.21.15.22:0/1002608433 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a012c400 con 0x7ff798033d00 2024-03-29T04:17:58.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7ff79e7fc640 1 -- 172.21.15.22:0/1002608433 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a012c830 con 0x7ff798033d00 2024-03-29T04:17:58.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f413c440640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4134072f20 0x7f413414a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f413c440640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f413415a2c0 0x7f41341566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f413c440640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f413415a6a0 0x7f4134156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f41399db640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f413415a2c0 0x7f41341566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f413a1dc640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4134072f20 0x7f413414a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f41399db640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f413415a2c0 0x7f41341566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:46664/0 (socket says 172.21.15.22:46664) 2024-03-29T04:17:58.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f41399db640 1 -- 172.21.15.22:0/2028055744 learned_addr learned my addr 172.21.15.22:0/2028055744 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:58.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f8a6e404640 1 -- 172.21.15.22:0/3655317386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a54005e60 msgr2=0x7f8a540062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f8a6e404640 1 --2- 172.21.15.22:0/3655317386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a54005e60 0x7f8a540062e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400a990 tx=0x7f8a6401c5d0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f907effd640 1 -- 172.21.15.22:0/1771841305 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90740b0070 con 0x7f9078055260 2024-03-29T04:17:58.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f907effd640 1 -- 172.21.15.22:0/1771841305 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90740ab600 con 0x7f9078055260 2024-03-29T04:17:58.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f907effd640 1 -- 172.21.15.22:0/1771841305 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90740ab9d0 con 0x7f9078055260 2024-03-29T04:17:58.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f413a9dd640 1 --2- 172.21.15.22:0/2028055744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f413415a6a0 0x7f4134156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f413c440640 1 -- 172.21.15.22:0/2028055744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f41340b2e80 con 0x7f413415a6a0 2024-03-29T04:17:58.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f413c440640 1 -- 172.21.15.22:0/2028055744 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f41340b3000 con 0x7f413415a2c0 2024-03-29T04:17:58.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f413c440640 1 -- 172.21.15.22:0/2028055744 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4134167250 con 0x7f4134072f20 2024-03-29T04:17:58.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f90865e0640 1 -- 172.21.15.22:0/1771841305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9078055260 msgr2=0x7f90780355b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f90865e0640 1 --2- 172.21.15.22:0/1771841305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9078055260 0x7f90780355b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f907803bae0 tx=0x7f907401c480 comp rx=0 tx=0).stop 2024-03-29T04:17:58.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f8a6dc03640 1 -- 172.21.15.22:0/3655317386 reap_dead start 2024-03-29T04:17:58.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f8a6e404640 1 -- 172.21.15.22:0/3655317386 shutdown_connections 2024-03-29T04:17:58.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f8a6e404640 1 -- 172.21.15.22:0/3655317386 >> 172.21.15.22:0/3655317386 conn(0x7f8a6806c5a0 msgr2=0x7f8a6806c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:58.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f90865e0640 1 -- 172.21.15.22:0/1771841305 shutdown_connections 2024-03-29T04:17:58.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f90865e0640 1 --2- 172.21.15.22:0/1771841305 >> v2:172.21.15.151:3300/0 conn(0x7f9078036fa0 0x7f9078052590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f90865e0640 1 --2- 172.21.15.22:0/1771841305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9078055260 0x7f90780355b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f90865e0640 1 -- 172.21.15.22:0/1771841305 >> 172.21.15.22:0/1771841305 conn(0x7f907801ef50 msgr2=0x7f9078055c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:58.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7ff7a4ca3640 1 -- 172.21.15.22:0/1002608433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff798033d00 msgr2=0x7ff798006a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7ff7a4ca3640 1 --2- 172.21.15.22:0/1002608433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff798033d00 0x7ff798006a80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0069df0 tx=0x7ff7a009a080 comp rx=0 tx=0).stop 2024-03-29T04:17:58.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7ff7a4ca3640 1 -- 172.21.15.22:0/1002608433 shutdown_connections 2024-03-29T04:17:58.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7ff7a4ca3640 1 --2- 172.21.15.22:0/1002608433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff798033d00 0x7ff798006a80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7ff7a4ca3640 1 --2- 172.21.15.22:0/1002608433 >> v2:172.21.15.22:3300/0 conn(0x7ff7980501b0 0x7ff798006540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7ff7a4ca3640 1 -- 172.21.15.22:0/1002608433 >> 172.21.15.22:0/1002608433 conn(0x7ff798009fc0 msgr2=0x7ff79800a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:58.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f90865e0640 1 -- 172.21.15.22:0/1771841305 shutdown_connections 2024-03-29T04:17:58.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f90865e0640 1 -- 172.21.15.22:0/1771841305 wait complete. 2024-03-29T04:17:58.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f8a6e404640 1 -- 172.21.15.22:0/3655317386 shutdown_connections 2024-03-29T04:17:58.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7ff7a4ca3640 1 -- 172.21.15.22:0/1002608433 shutdown_connections 2024-03-29T04:17:58.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f413a9dd640 1 -- 172.21.15.22:0/2028055744 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4134072f20 msgr2=0x7f413414a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f413a9dd640 1 --2- 172.21.15.22:0/2028055744 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4134072f20 0x7f413414a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f413a9dd640 1 -- 172.21.15.22:0/2028055744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f413415a2c0 msgr2=0x7f41341566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f413a9dd640 1 --2- 172.21.15.22:0/2028055744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f413415a2c0 0x7f41341566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f413a9dd640 1 -- 172.21.15.22:0/2028055744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4134135380 con 0x7f413415a6a0 2024-03-29T04:17:58.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f413a9dd640 1 --2- 172.21.15.22:0/2028055744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f413415a6a0 0x7f4134156bf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4134070f50 tx=0x7f4124019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f412affd640 1 -- 172.21.15.22:0/2028055744 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41240bb070 con 0x7f413415a6a0 2024-03-29T04:17:58.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f412affd640 1 -- 172.21.15.22:0/2028055744 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41240038e0 con 0x7f413415a6a0 2024-03-29T04:17:58.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f412affd640 1 -- 172.21.15.22:0/2028055744 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41240b1040 con 0x7f413415a6a0 2024-03-29T04:17:58.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f413c440640 1 -- 172.21.15.22:0/2028055744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4134135660 con 0x7f413415a6a0 2024-03-29T04:17:58.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.432+0000 7f8a6e404640 1 -- 172.21.15.22:0/3655317386 wait complete. 2024-03-29T04:17:58.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f8a6e404640 1 Processor -- start 2024-03-29T04:17:58.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f8a6e404640 1 -- start start 2024-03-29T04:17:58.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f8a6e404640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a6813e690 0x7f8a681500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f8a6e404640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a68154a40 0x7f8a68154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f8a6e404640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a68159260 0x7f8a681596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f8a6e404640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6815ed00 con 0x7f8a6813e690 2024-03-29T04:17:58.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f8a6e404640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a680b4000 con 0x7f8a68154a40 2024-03-29T04:17:58.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f8a6e404640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8a68145290 con 0x7f8a68159260 2024-03-29T04:17:58.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f90865e0640 1 Processor -- start 2024-03-29T04:17:58.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f90865e0640 1 -- start start 2024-03-29T04:17:58.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f90865e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9078036fa0 0x7f90780d8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f90865e0640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9078052ad0 0x7f90780de4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f90865e0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9078055260 0x7f90780de9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f90865e0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f90780e03f0 con 0x7f9078036fa0 2024-03-29T04:17:58.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f90865e0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9078008ee0 con 0x7f9078055260 2024-03-29T04:17:58.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f90865e0640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f90780ec680 con 0x7f9078052ad0 2024-03-29T04:17:58.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f413c440640 1 -- 172.21.15.22:0/2028055744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f413414abd0 con 0x7f413415a6a0 2024-03-29T04:17:58.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f412affd640 1 -- 172.21.15.22:0/2028055744 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f4124003c90 con 0x7f413415a6a0 2024-03-29T04:17:58.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f412affd640 1 --2- 172.21.15.22:0/2028055744 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4118067ff0 0x7f411806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f8a6d402640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a6813e690 0x7f8a681500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f8a6d402640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a6813e690 0x7f8a681500d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51662/0 (socket says 172.21.15.22:51662) 2024-03-29T04:17:58.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f8a6d402640 1 -- 172.21.15.22:0/1802934524 learned_addr learned my addr 172.21.15.22:0/1802934524 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:58.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f8a6d402640 1 -- 172.21.15.22:0/1802934524 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a68159260 msgr2=0x7f8a681596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f8a6d402640 1 --2- 172.21.15.22:0/1802934524 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a68159260 0x7f8a681596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f8a6d402640 1 -- 172.21.15.22:0/1802934524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a68154a40 msgr2=0x7f8a68154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f8a6d402640 1 --2- 172.21.15.22:0/1802934524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a68154a40 0x7f8a68154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f8a6d402640 1 -- 172.21.15.22:0/1802934524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a68159c30 con 0x7f8a6813e690 2024-03-29T04:17:58.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f8a6d402640 1 --2- 172.21.15.22:0/1802934524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a6813e690 0x7f8a681500d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000f780 tx=0x7f8a6000fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f907ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9078036fa0 0x7f90780d8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f907ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9078036fa0 0x7f90780d8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51654/0 (socket says 172.21.15.22:51654) 2024-03-29T04:17:58.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f907ffff640 1 -- 172.21.15.22:0/1308593224 learned_addr learned my addr 172.21.15.22:0/1308593224 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:58.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f907ffff640 1 -- 172.21.15.22:0/1308593224 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9078052ad0 msgr2=0x7f90780de4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f907ffff640 1 --2- 172.21.15.22:0/1308593224 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9078052ad0 0x7f90780de4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f907ffff640 1 -- 172.21.15.22:0/1308593224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9078055260 msgr2=0x7f90780de9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f907ffff640 1 --2- 172.21.15.22:0/1308593224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9078055260 0x7f90780de9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f907ffff640 1 -- 172.21.15.22:0/1308593224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90780d8600 con 0x7f9078036fa0 2024-03-29T04:17:58.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f8a5dffb640 1 -- 172.21.15.22:0/1802934524 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6001a070 con 0x7f8a6813e690 2024-03-29T04:17:58.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f907ffff640 1 --2- 172.21.15.22:0/1308593224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9078036fa0 0x7f90780d8020 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f90740ae040 tx=0x7f907401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f412affd640 1 -- 172.21.15.22:0/2028055744 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f4124124060 con 0x7f413415a6a0 2024-03-29T04:17:58.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7ff7a4ca3640 1 -- 172.21.15.22:0/1002608433 wait complete. 2024-03-29T04:17:58.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7ff7a4ca3640 1 Processor -- start 2024-03-29T04:17:58.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f8a6e404640 1 -- 172.21.15.22:0/1802934524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a68159eb0 con 0x7f8a6813e690 2024-03-29T04:17:58.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.436+0000 7f8a6e404640 1 -- 172.21.15.22:0/1802934524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a6815a470 con 0x7f8a6813e690 2024-03-29T04:17:58.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7f8a5dffb640 1 -- 172.21.15.22:0/1802934524 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a6000a270 con 0x7f8a6813e690 2024-03-29T04:17:58.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7f8a5dffb640 1 -- 172.21.15.22:0/1802934524 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6000d840 con 0x7f8a6813e690 2024-03-29T04:17:58.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7f413a1dc640 1 --2- 172.21.15.22:0/2028055744 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4118067ff0 0x7f411806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7f907cff9640 1 -- 172.21.15.22:0/1308593224 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90740b0050 con 0x7f9078036fa0 2024-03-29T04:17:58.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7f90865e0640 1 -- 172.21.15.22:0/1308593224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90780d88e0 con 0x7f9078036fa0 2024-03-29T04:17:58.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7f90865e0640 1 -- 172.21.15.22:0/1308593224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90780d8ea0 con 0x7f9078036fa0 2024-03-29T04:17:58.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7ff7a4ca3640 1 -- start start 2024-03-29T04:17:58.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7ff7a4ca3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff798033d00 0x7ff7980cf860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7ff7a4ca3640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff7980501b0 0x7ff79804bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7ff7a4ca3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7980df2b0 0x7ff7980d9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7ff7a4ca3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff79804f7e0 con 0x7ff7980df2b0 2024-03-29T04:17:58.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7ff7a4ca3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff79804f660 con 0x7ff798033d00 2024-03-29T04:17:58.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7ff7a4ca3640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff7980ec050 con 0x7ff7980501b0 2024-03-29T04:17:58.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7ff79f7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff798033d00 0x7ff7980cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7ff79f7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff798033d00 0x7ff7980cf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:46700/0 (socket says 172.21.15.22:46700) 2024-03-29T04:17:58.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7ff79f7fe640 1 -- 172.21.15.22:0/589260450 learned_addr learned my addr 172.21.15.22:0/589260450 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:58.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7ff79effd640 1 --2- 172.21.15.22:0/589260450 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff7980501b0 0x7ff79804bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7ff79ffff640 1 --2- 172.21.15.22:0/589260450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7980df2b0 0x7ff7980d9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7f413a1dc640 1 --2- 172.21.15.22:0/2028055744 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4118067ff0 0x7f411806a4b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4130004150 tx=0x7f413000b390 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7ff79ffff640 1 -- 172.21.15.22:0/589260450 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff7980501b0 msgr2=0x7ff79804bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7ff79ffff640 1 --2- 172.21.15.22:0/589260450 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff7980501b0 0x7ff79804bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7ff79ffff640 1 -- 172.21.15.22:0/589260450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff798033d00 msgr2=0x7ff7980cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7ff79ffff640 1 --2- 172.21.15.22:0/589260450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff798033d00 0x7ff7980cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7ff79ffff640 1 -- 172.21.15.22:0/589260450 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7980376e0 con 0x7ff7980df2b0 2024-03-29T04:17:58.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7ff79ffff640 1 --2- 172.21.15.22:0/589260450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7980df2b0 0x7ff7980d9860 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff7a007a980 tx=0x7ff7a0096e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7f8a5dffb640 1 -- 172.21.15.22:0/1802934524 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f8a6001d070 con 0x7f8a6813e690 2024-03-29T04:17:58.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7f907cff9640 1 -- 172.21.15.22:0/1308593224 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90740bc070 con 0x7f9078036fa0 2024-03-29T04:17:58.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7f907cff9640 1 -- 172.21.15.22:0/1308593224 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90740b5810 con 0x7f9078036fa0 2024-03-29T04:17:58.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7f8a5dffb640 1 --2- 172.21.15.22:0/1802934524 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8a4c0680c0 0x7f8a4c06a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7f8a5dffb640 1 -- 172.21.15.22:0/1802934524 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8a60088ac0 con 0x7f8a6813e690 2024-03-29T04:17:58.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7f8a6cc01640 1 --2- 172.21.15.22:0/1802934524 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8a4c0680c0 0x7f8a4c06a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7f413c440640 1 -- 172.21.15.22:0/2028055744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41340686e0 con 0x7f413415a6a0 2024-03-29T04:17:58.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.440+0000 7f907cff9640 1 -- 172.21.15.22:0/1308593224 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f90740b59b0 con 0x7f9078036fa0 2024-03-29T04:17:58.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.444+0000 7f8a6cc01640 1 --2- 172.21.15.22:0/1802934524 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8a4c0680c0 0x7f8a4c06a580 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8a58007fd0 tx=0x7f8a580079c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.444+0000 7f8a6e404640 1 -- 172.21.15.22:0/1802934524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a480052a0 con 0x7f8a6813e690 2024-03-29T04:17:58.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.444+0000 7f907cff9640 1 --2- 172.21.15.22:0/1308593224 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9064067fc0 0x7f906406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.444+0000 7ff78bfff640 1 -- 172.21.15.22:0/589260450 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0128040 con 0x7ff7980df2b0 2024-03-29T04:17:58.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.444+0000 7ff78bfff640 1 -- 172.21.15.22:0/589260450 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a012ce10 con 0x7ff7980df2b0 2024-03-29T04:17:58.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.444+0000 7ff7a4ca3640 1 -- 172.21.15.22:0/589260450 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7980d7880 con 0x7ff7980df2b0 2024-03-29T04:17:58.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.444+0000 7f907cff9640 1 -- 172.21.15.22:0/1308593224 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9074120e80 con 0x7f9078036fa0 2024-03-29T04:17:58.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.444+0000 7f907f7fe640 1 --2- 172.21.15.22:0/1308593224 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9064067fc0 0x7f906406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.444+0000 7ff7a4ca3640 1 -- 172.21.15.22:0/589260450 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7980d7b20 con 0x7ff7980df2b0 2024-03-29T04:17:58.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.444+0000 7ff78bfff640 1 -- 172.21.15.22:0/589260450 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a006fdf0 con 0x7ff7980df2b0 2024-03-29T04:17:58.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.444+0000 7ff7a4ca3640 1 -- 172.21.15.22:0/589260450 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7980053a0 con 0x7ff7980df2b0 2024-03-29T04:17:58.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.444+0000 7f907f7fe640 1 --2- 172.21.15.22:0/1308593224 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9064067fc0 0x7f906406a480 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f906c006f10 tx=0x7f906c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.448+0000 7f90865e0640 1 -- 172.21.15.22:0/1308593224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90600052a0 con 0x7f9078036fa0 2024-03-29T04:17:58.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.448+0000 7f412affd640 1 -- 172.21.15.22:0/2028055744 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f4124016090 con 0x7f413415a6a0 2024-03-29T04:17:58.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.452+0000 7f8a5dffb640 1 -- 172.21.15.22:0/1802934524 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f8a68159c30 con 0x7f8a6813e690 2024-03-29T04:17:58.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.456+0000 7ff78bfff640 1 -- 172.21.15.22:0/589260450 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff7a013d060 con 0x7ff7980df2b0 2024-03-29T04:17:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.456+0000 7f907cff9640 1 -- 172.21.15.22:0/1308593224 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9074016070 con 0x7f9078036fa0 2024-03-29T04:17:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.460+0000 7ff78bfff640 1 --2- 172.21.15.22:0/589260450 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff784068040 0x7ff78406a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.460+0000 7ff78bfff640 1 -- 172.21.15.22:0/589260450 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff7a019b6d0 con 0x7ff7980df2b0 2024-03-29T04:17:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.460+0000 7ff79f7fe640 1 --2- 172.21.15.22:0/589260450 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff784068040 0x7ff78406a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.460+0000 7ff79f7fe640 1 --2- 172.21.15.22:0/589260450 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff784068040 0x7ff78406a500 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff794000f70 tx=0x7ff794007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.468+0000 7ff78bfff640 1 -- 172.21.15.22:0/589260450 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff7a0093ea0 con 0x7ff7980df2b0 2024-03-29T04:17:58.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.676+0000 7f475bd95640 1 -- 172.21.15.22:0/3833804544 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f473c005590 con 0x7f475415a6a0 2024-03-29T04:17:58.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.676+0000 7f474a7fc640 1 -- 172.21.15.22:0/3833804544 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f474c018840 con 0x7f475415a6a0 2024-03-29T04:17:58.662 INFO:teuthology.orchestra.run.smithi022.stdout:55834574856 2024-03-29T04:17:58.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.680+0000 7f4737fff640 1 -- 172.21.15.22:0/3833804544 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4740068230 msgr2=0x7f474006a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.680+0000 7f4737fff640 1 --2- 172.21.15.22:0/3833804544 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4740068230 0x7f474006a6f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f47541642e0 tx=0x7f475000f040 comp rx=0 tx=0).stop 2024-03-29T04:17:58.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.680+0000 7f4737fff640 1 -- 172.21.15.22:0/3833804544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f475415a6a0 msgr2=0x7f4754164890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.680+0000 7f4737fff640 1 --2- 172.21.15.22:0/3833804544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f475415a6a0 0x7f4754164890 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f474c01d240 tx=0x7f474c01b930 comp rx=0 tx=0).stop 2024-03-29T04:17:58.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.680+0000 7f4737fff640 1 -- 172.21.15.22:0/3833804544 shutdown_connections 2024-03-29T04:17:58.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.680+0000 7f4737fff640 1 --2- 172.21.15.22:0/3833804544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f475415aac0 0x7f475414c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.680+0000 7f4737fff640 1 --2- 172.21.15.22:0/3833804544 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4740068230 0x7f474006a6f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.680+0000 7f4737fff640 1 --2- 172.21.15.22:0/3833804544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f475415a6a0 0x7f4754164890 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.680+0000 7f4737fff640 1 --2- 172.21.15.22:0/3833804544 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f475415a2c0 0x7f4754164350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.680+0000 7f4737fff640 1 -- 172.21.15.22:0/3833804544 >> 172.21.15.22:0/3833804544 conn(0x7f475406c570 msgr2=0x7f475415c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:58.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.684+0000 7f4737fff640 1 -- 172.21.15.22:0/3833804544 shutdown_connections 2024-03-29T04:17:58.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.684+0000 7f4737fff640 1 -- 172.21.15.22:0/3833804544 wait complete. 2024-03-29T04:17:58.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.688+0000 7faef72c2640 1 -- 172.21.15.22:0/3724260380 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7faed0005590 con 0x7faef015d2d0 2024-03-29T04:17:58.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.688+0000 7faee5ffb640 1 -- 172.21.15.22:0/3724260380 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faee005a4b0 con 0x7faef015d2d0 2024-03-29T04:17:58.671 INFO:teuthology.orchestra.run.smithi022.stdout:55834574856 2024-03-29T04:17:58.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.692+0000 7faedb7fe640 1 -- 172.21.15.22:0/3724260380 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faed4067fc0 msgr2=0x7faed406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.692+0000 7faedb7fe640 1 --2- 172.21.15.22:0/3724260380 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faed4067fc0 0x7faed406a480 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7faee801c9c0 tx=0x7faee80a7070 comp rx=0 tx=0).stop 2024-03-29T04:17:58.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.692+0000 7faedb7fe640 1 -- 172.21.15.22:0/3724260380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faef015d2d0 msgr2=0x7faef015c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.692+0000 7faedb7fe640 1 --2- 172.21.15.22:0/3724260380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faef015d2d0 0x7faef015c770 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faee000b3c0 tx=0x7faee000b880 comp rx=0 tx=0).stop 2024-03-29T04:17:58.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.692+0000 7faedb7fe640 1 -- 172.21.15.22:0/3724260380 shutdown_connections 2024-03-29T04:17:58.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.692+0000 7faedb7fe640 1 --2- 172.21.15.22:0/3724260380 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faed4067fc0 0x7faed406a480 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.692+0000 7faedb7fe640 1 --2- 172.21.15.22:0/3724260380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faef015d2d0 0x7faef015c770 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.692+0000 7faedb7fe640 1 --2- 172.21.15.22:0/3724260380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faef015a800 0x7faef0156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.692+0000 7faedb7fe640 1 --2- 172.21.15.22:0/3724260380 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faef006e0f0 0x7faef0158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.692+0000 7faedb7fe640 1 -- 172.21.15.22:0/3724260380 >> 172.21.15.22:0/3724260380 conn(0x7faef006c570 msgr2=0x7faef00cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:58.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.692+0000 7faedb7fe640 1 -- 172.21.15.22:0/3724260380 shutdown_connections 2024-03-29T04:17:58.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.692+0000 7faedb7fe640 1 -- 172.21.15.22:0/3724260380 wait complete. 2024-03-29T04:17:58.688 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2024-03-29T04:17:58.688 DEBUG:teuthology.parallel:result is None 2024-03-29T04:17:58.699 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2024-03-29T04:17:58.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.716+0000 7f755ca83640 1 -- 172.21.15.22:0/1545315361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f75580686e0 con 0x7f755815a2c0 2024-03-29T04:17:58.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.716+0000 7f7546ffd640 1 -- 172.21.15.22:0/1545315361 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7548059cb0 con 0x7f755815a2c0 2024-03-29T04:17:58.699 DEBUG:teuthology.parallel:result is None 2024-03-29T04:17:58.700 INFO:teuthology.orchestra.run.smithi022.stdout:55834574857 2024-03-29T04:17:58.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.720+0000 7f755ca83640 1 -- 172.21.15.22:0/1545315361 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7538068010 msgr2=0x7f753806a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.720+0000 7f755ca83640 1 --2- 172.21.15.22:0/1545315361 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7538068010 0x7f753806a4d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7550009c50 tx=0x7f755000d040 comp rx=0 tx=0).stop 2024-03-29T04:17:58.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.720+0000 7f755ca83640 1 -- 172.21.15.22:0/1545315361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f755815a2c0 msgr2=0x7f75580ae0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.720+0000 7f755ca83640 1 --2- 172.21.15.22:0/1545315361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f755815a2c0 0x7f75580ae0f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f754800c400 tx=0x7f754800c8c0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.720+0000 7f755ca83640 1 -- 172.21.15.22:0/1545315361 shutdown_connections 2024-03-29T04:17:58.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.720+0000 7f755ca83640 1 --2- 172.21.15.22:0/1545315361 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f755815a6a0 0x7f7558207f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.720+0000 7f755ca83640 1 --2- 172.21.15.22:0/1545315361 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7538068010 0x7f753806a4d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.720+0000 7f755ca83640 1 --2- 172.21.15.22:0/1545315361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f755815a2c0 0x7f75580ae0f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.720+0000 7f755ca83640 1 --2- 172.21.15.22:0/1545315361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7558072f20 0x7f7558154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.720+0000 7f755ca83640 1 -- 172.21.15.22:0/1545315361 >> 172.21.15.22:0/1545315361 conn(0x7f755806c570 msgr2=0x7f755815af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:58.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.720+0000 7f755ca83640 1 -- 172.21.15.22:0/1545315361 shutdown_connections 2024-03-29T04:17:58.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.720+0000 7f755ca83640 1 -- 172.21.15.22:0/1545315361 wait complete. 2024-03-29T04:17:58.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.732+0000 7feb66f00640 1 -- 172.21.15.22:0/1739452924 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7feb580053a0 con 0x7feb58033e40 2024-03-29T04:17:58.715 INFO:teuthology.orchestra.run.smithi022.stdout:55834574857 2024-03-29T04:17:58.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.736+0000 7feb567fc640 1 -- 172.21.15.22:0/1739452924 <== mon.2 v2:172.21.15.151:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feb5001f020 con 0x7feb58033e40 2024-03-29T04:17:58.717 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.7 2024-03-29T04:17:58.717 DEBUG:teuthology.parallel:result is None 2024-03-29T04:17:58.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.736+0000 7feb66f00640 1 -- 172.21.15.22:0/1739452924 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7feb44067ff0 msgr2=0x7feb4406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.736+0000 7feb66f00640 1 --2- 172.21.15.22:0/1739452924 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7feb44067ff0 0x7feb4406a4b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7feb580d5f80 tx=0x7feb5c003700 comp rx=0 tx=0).stop 2024-03-29T04:17:58.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.736+0000 7feb66f00640 1 -- 172.21.15.22:0/1739452924 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb58033e40 msgr2=0x7feb580d4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.736+0000 7feb66f00640 1 --2- 172.21.15.22:0/1739452924 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb58033e40 0x7feb580d4a20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7feb5000b410 tx=0x7feb5000b8d0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.736+0000 7feb66f00640 1 -- 172.21.15.22:0/1739452924 shutdown_connections 2024-03-29T04:17:58.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.736+0000 7feb66f00640 1 --2- 172.21.15.22:0/1739452924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb58047c20 0x7feb58006dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.736+0000 7feb66f00640 1 --2- 172.21.15.22:0/1739452924 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feb58033e40 0x7feb580d4a20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.736+0000 7feb66f00640 1 --2- 172.21.15.22:0/1739452924 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7feb44067ff0 0x7feb4406a4b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.736+0000 7feb66f00640 1 --2- 172.21.15.22:0/1739452924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb58005ea0 0x7feb580cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.736+0000 7feb66f00640 1 -- 172.21.15.22:0/1739452924 >> 172.21.15.22:0/1739452924 conn(0x7feb58009fc0 msgr2=0x7feb580d9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:58.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.736+0000 7feb66f00640 1 -- 172.21.15.22:0/1739452924 shutdown_connections 2024-03-29T04:17:58.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.736+0000 7feb66f00640 1 -- 172.21.15.22:0/1739452924 wait complete. 2024-03-29T04:17:58.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.752+0000 7f8a6e404640 1 -- 172.21.15.22:0/1802934524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f8a48005590 con 0x7f8a6813e690 2024-03-29T04:17:58.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.752+0000 7f8a5dffb640 1 -- 172.21.15.22:0/1802934524 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8a600588c0 con 0x7f8a6813e690 2024-03-29T04:17:58.734 INFO:teuthology.orchestra.run.smithi022.stdout:55834574857 2024-03-29T04:17:58.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.756+0000 7f8a6e404640 1 -- 172.21.15.22:0/1802934524 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8a4c0680c0 msgr2=0x7f8a4c06a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.756+0000 7f8a6e404640 1 --2- 172.21.15.22:0/1802934524 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8a4c0680c0 0x7f8a4c06a580 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8a58007fd0 tx=0x7f8a580079c0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.756+0000 7f8a6e404640 1 -- 172.21.15.22:0/1802934524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a6813e690 msgr2=0x7f8a681500d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.756+0000 7f8a6e404640 1 --2- 172.21.15.22:0/1802934524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a6813e690 0x7f8a681500d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000f780 tx=0x7f8a6000fc40 comp rx=0 tx=0).stop 2024-03-29T04:17:58.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.756+0000 7f8a6e404640 1 -- 172.21.15.22:0/1802934524 shutdown_connections 2024-03-29T04:17:58.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.756+0000 7f8a6e404640 1 --2- 172.21.15.22:0/1802934524 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8a68159260 0x7f8a681596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.756+0000 7f8a6e404640 1 --2- 172.21.15.22:0/1802934524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a68154a40 0x7f8a68154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.756+0000 7f8a6e404640 1 --2- 172.21.15.22:0/1802934524 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8a4c0680c0 0x7f8a4c06a580 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.756+0000 7f8a6e404640 1 --2- 172.21.15.22:0/1802934524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a6813e690 0x7f8a681500d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.756+0000 7f8a6e404640 1 -- 172.21.15.22:0/1802934524 >> 172.21.15.22:0/1802934524 conn(0x7f8a6806c5a0 msgr2=0x7f8a680b12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:58.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.756+0000 7f8a6e404640 1 -- 172.21.15.22:0/1802934524 shutdown_connections 2024-03-29T04:17:58.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.756+0000 7f8a6e404640 1 -- 172.21.15.22:0/1802934524 wait complete. 2024-03-29T04:17:58.738 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2024-03-29T04:17:58.738 DEBUG:teuthology.parallel:result is None 2024-03-29T04:17:58.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.768+0000 7ff7a4ca3640 1 -- 172.21.15.22:0/589260450 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff798047f90 con 0x7ff7980df2b0 2024-03-29T04:17:58.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.768+0000 7ff78bfff640 1 -- 172.21.15.22:0/589260450 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff7a016b560 con 0x7ff7980df2b0 2024-03-29T04:17:58.748 INFO:teuthology.orchestra.run.smithi022.stdout:55834574857 2024-03-29T04:17:58.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.768+0000 7ff7a4ca3640 1 -- 172.21.15.22:0/589260450 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff784068040 msgr2=0x7ff78406a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.768+0000 7ff7a4ca3640 1 --2- 172.21.15.22:0/589260450 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff784068040 0x7ff78406a500 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff794000f70 tx=0x7ff794007eb0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.768+0000 7ff7a4ca3640 1 -- 172.21.15.22:0/589260450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7980df2b0 msgr2=0x7ff7980d9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.768+0000 7ff7a4ca3640 1 --2- 172.21.15.22:0/589260450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7980df2b0 0x7ff7980d9860 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff7a007a980 tx=0x7ff7a0096e70 comp rx=0 tx=0).stop 2024-03-29T04:17:58.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.768+0000 7ff7a4ca3640 1 -- 172.21.15.22:0/589260450 shutdown_connections 2024-03-29T04:17:58.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.768+0000 7ff7a4ca3640 1 --2- 172.21.15.22:0/589260450 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff784068040 0x7ff78406a500 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.768+0000 7ff7a4ca3640 1 --2- 172.21.15.22:0/589260450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7980df2b0 0x7ff7980d9860 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.768+0000 7ff7a4ca3640 1 --2- 172.21.15.22:0/589260450 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff7980501b0 0x7ff79804bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.768+0000 7ff7a4ca3640 1 --2- 172.21.15.22:0/589260450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff798033d00 0x7ff7980cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.768+0000 7ff7a4ca3640 1 -- 172.21.15.22:0/589260450 >> 172.21.15.22:0/589260450 conn(0x7ff798009fc0 msgr2=0x7ff798006440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:58.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.768+0000 7ff7a4ca3640 1 -- 172.21.15.22:0/589260450 shutdown_connections 2024-03-29T04:17:58.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.768+0000 7ff7a4ca3640 1 -- 172.21.15.22:0/589260450 wait complete. 2024-03-29T04:17:58.752 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.2 2024-03-29T04:17:58.752 DEBUG:teuthology.parallel:result is None 2024-03-29T04:17:58.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.784+0000 7f413c440640 1 -- 172.21.15.22:0/2028055744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f41340743d0 con 0x7f413415a6a0 2024-03-29T04:17:58.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.784+0000 7f412affd640 1 -- 172.21.15.22:0/2028055744 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f41240ef800 con 0x7f413415a6a0 2024-03-29T04:17:58.765 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2024-03-29T04:17:58.766 INFO:teuthology.orchestra.run.smithi022.stdout:55834574856 2024-03-29T04:17:58.766 DEBUG:teuthology.parallel:result is None 2024-03-29T04:17:58.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f90865e0640 1 -- 172.21.15.22:0/1308593224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9060005860 con 0x7f9078036fa0 2024-03-29T04:17:58.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f907cff9640 1 -- 172.21.15.22:0/1308593224 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f90740f14a0 con 0x7f9078036fa0 2024-03-29T04:17:58.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f413c440640 1 -- 172.21.15.22:0/2028055744 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4118067ff0 msgr2=0x7f411806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f413c440640 1 --2- 172.21.15.22:0/2028055744 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4118067ff0 0x7f411806a4b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4130004150 tx=0x7f413000b390 comp rx=0 tx=0).stop 2024-03-29T04:17:58.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f413c440640 1 -- 172.21.15.22:0/2028055744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f413415a6a0 msgr2=0x7f4134156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f413c440640 1 --2- 172.21.15.22:0/2028055744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f413415a6a0 0x7f4134156bf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4134070f50 tx=0x7f4124019180 comp rx=0 tx=0).stop 2024-03-29T04:17:58.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f413c440640 1 -- 172.21.15.22:0/2028055744 shutdown_connections 2024-03-29T04:17:58.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f413c440640 1 --2- 172.21.15.22:0/2028055744 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4118067ff0 0x7f411806a4b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f413c440640 1 --2- 172.21.15.22:0/2028055744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f413415a6a0 0x7f4134156bf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f413c440640 1 --2- 172.21.15.22:0/2028055744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f413415a2c0 0x7f41341566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f413c440640 1 --2- 172.21.15.22:0/2028055744 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4134072f20 0x7f413414a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f413c440640 1 -- 172.21.15.22:0/2028055744 >> 172.21.15.22:0/2028055744 conn(0x7f413406c570 msgr2=0x7f41340ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:58.769 INFO:teuthology.orchestra.run.smithi022.stdout:55834574856 2024-03-29T04:17:58.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f413c440640 1 -- 172.21.15.22:0/2028055744 shutdown_connections 2024-03-29T04:17:58.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f413c440640 1 -- 172.21.15.22:0/2028055744 wait complete. 2024-03-29T04:17:58.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f90865e0640 1 -- 172.21.15.22:0/1308593224 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9064067fc0 msgr2=0x7f906406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f90865e0640 1 --2- 172.21.15.22:0/1308593224 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9064067fc0 0x7f906406a480 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f906c006f10 tx=0x7f906c008040 comp rx=0 tx=0).stop 2024-03-29T04:17:58.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f90865e0640 1 -- 172.21.15.22:0/1308593224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9078036fa0 msgr2=0x7f90780d8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f90865e0640 1 --2- 172.21.15.22:0/1308593224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9078036fa0 0x7f90780d8020 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f90740ae040 tx=0x7f907401acc0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f90865e0640 1 -- 172.21.15.22:0/1308593224 shutdown_connections 2024-03-29T04:17:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f90865e0640 1 --2- 172.21.15.22:0/1308593224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9078055260 0x7f90780de9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f90865e0640 1 --2- 172.21.15.22:0/1308593224 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9078052ad0 0x7f90780de4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f90865e0640 1 --2- 172.21.15.22:0/1308593224 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9064067fc0 0x7f906406a480 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f90865e0640 1 --2- 172.21.15.22:0/1308593224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9078036fa0 0x7f90780d8020 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f90865e0640 1 -- 172.21.15.22:0/1308593224 >> 172.21.15.22:0/1308593224 conn(0x7f907801ef50 msgr2=0x7f90780d9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f90865e0640 1 -- 172.21.15.22:0/1308593224 shutdown_connections 2024-03-29T04:17:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.788+0000 7f90865e0640 1 -- 172.21.15.22:0/1308593224 wait complete. 2024-03-29T04:17:58.786 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2024-03-29T04:17:58.786 DEBUG:teuthology.parallel:result is None 2024-03-29T04:17:58.787 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-03-29T04:17:58.787 DEBUG:teuthology.parallel:result is None 2024-03-29T04:17:58.787 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-29T04:17:58.787 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:17:58.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.964+0000 7f2741240640 1 Processor -- start 2024-03-29T04:17:58.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.964+0000 7f2741240640 1 -- start start 2024-03-29T04:17:58.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.964+0000 7f2741240640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f273c15c1c0 0x7f273c0cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.964+0000 7f2741240640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f273c0d0900 0x7f273c0d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.964+0000 7f2741240640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f273c0b5060 0x7f273c0ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.964+0000 7f2741240640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f273c0ce3a0 con 0x7f273c0d0900 2024-03-29T04:17:58.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.964+0000 7f2741240640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f273c0cce80 con 0x7f273c15c1c0 2024-03-29T04:17:58.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.964+0000 7f2741240640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f273c0b2cb0 con 0x7f273c0b5060 2024-03-29T04:17:58.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.964+0000 7f273a59c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f273c0d0900 0x7f273c0d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.964+0000 7f273a59c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f273c0d0900 0x7f273c0d02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51682/0 (socket says 172.21.15.22:51682) 2024-03-29T04:17:58.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.964+0000 7f273a59c640 1 -- 172.21.15.22:0/235533539 learned_addr learned my addr 172.21.15.22:0/235533539 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:58.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.964+0000 7f273b59e640 1 --2- 172.21.15.22:0/235533539 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f273c0b5060 0x7f273c0ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.964+0000 7f273ad9d640 1 --2- 172.21.15.22:0/235533539 >> v2:172.21.15.151:3300/0 conn(0x7f273c15c1c0 0x7f273c0cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.964+0000 7f273a59c640 1 -- 172.21.15.22:0/235533539 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f273c0b5060 msgr2=0x7f273c0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:58.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.964+0000 7f273a59c640 1 --2- 172.21.15.22:0/235533539 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f273c0b5060 0x7f273c0ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.964+0000 7f273a59c640 1 -- 172.21.15.22:0/235533539 >> v2:172.21.15.151:3300/0 conn(0x7f273c15c1c0 msgr2=0x7f273c0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:58.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.964+0000 7f273a59c640 1 --2- 172.21.15.22:0/235533539 >> v2:172.21.15.151:3300/0 conn(0x7f273c15c1c0 0x7f273c0cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.964+0000 7f273a59c640 1 -- 172.21.15.22:0/235533539 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273c0ccba0 con 0x7f273c0d0900 2024-03-29T04:17:58.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.964+0000 7f273ad9d640 1 --2- 172.21.15.22:0/235533539 >> v2:172.21.15.151:3300/0 conn(0x7f273c15c1c0 0x7f273c0cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:58.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f273a59c640 1 --2- 172.21.15.22:0/235533539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f273c0d0900 0x7f273c0d02a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f273c0d3230 tx=0x7f273001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3210e95b208833d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f2739d9b640 1 -- 172.21.15.22:0/235533539 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27300aa610 con 0x7f273c0d0900 2024-03-29T04:17:58.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f2739d9b640 1 -- 172.21.15.22:0/235533539 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27300af070 con 0x7f273c0d0900 2024-03-29T04:17:58.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f2739d9b640 1 -- 172.21.15.22:0/235533539 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27300aa9d0 con 0x7f273c0d0900 2024-03-29T04:17:58.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f2741240640 1 -- 172.21.15.22:0/235533539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f273c0d0900 msgr2=0x7f273c0d02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f2741240640 1 --2- 172.21.15.22:0/235533539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f273c0d0900 0x7f273c0d02a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f273c0d3230 tx=0x7f273001c4f0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f2741240640 1 -- 172.21.15.22:0/235533539 shutdown_connections 2024-03-29T04:17:58.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f2741240640 1 --2- 172.21.15.22:0/235533539 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f273c0b5060 0x7f273c0ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f2741240640 1 --2- 172.21.15.22:0/235533539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f273c0d0900 0x7f273c0d02a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f2741240640 1 --2- 172.21.15.22:0/235533539 >> v2:172.21.15.151:3300/0 conn(0x7f273c15c1c0 0x7f273c0cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f2741240640 1 -- 172.21.15.22:0/235533539 >> 172.21.15.22:0/235533539 conn(0x7f273c0b3d60 msgr2=0x7f273c0b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:58.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f2741240640 1 -- 172.21.15.22:0/235533539 shutdown_connections 2024-03-29T04:17:58.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f2741240640 1 -- 172.21.15.22:0/235533539 wait complete. 2024-03-29T04:17:58.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f2741240640 1 Processor -- start 2024-03-29T04:17:58.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f2741240640 1 -- start start 2024-03-29T04:17:58.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f2741240640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f273c0b5060 0x7f273c1653e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f2741240640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f273c0d0900 0x7f273c1557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f2741240640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f273c15c1c0 0x7f273c1533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f2741240640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f273c15c720 con 0x7f273c15c1c0 2024-03-29T04:17:58.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f2741240640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f273c15c5a0 con 0x7f273c0d0900 2024-03-29T04:17:58.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f2741240640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f273c15fea0 con 0x7f273c0b5060 2024-03-29T04:17:58.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f273b59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f273c15c1c0 0x7f273c1533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f273b59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f273c15c1c0 0x7f273c1533c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51690/0 (socket says 172.21.15.22:51690) 2024-03-29T04:17:58.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f273b59e640 1 -- 172.21.15.22:0/241851297 learned_addr learned my addr 172.21.15.22:0/241851297 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:58.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f273ad9d640 1 --2- 172.21.15.22:0/241851297 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f273c0b5060 0x7f273c1653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f273a59c640 1 --2- 172.21.15.22:0/241851297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f273c0d0900 0x7f273c1557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f273b59e640 1 -- 172.21.15.22:0/241851297 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f273c0b5060 msgr2=0x7f273c1653e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f273b59e640 1 --2- 172.21.15.22:0/241851297 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f273c0b5060 0x7f273c1653e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f273b59e640 1 -- 172.21.15.22:0/241851297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f273c0d0900 msgr2=0x7f273c1557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:58.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f273b59e640 1 --2- 172.21.15.22:0/241851297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f273c0d0900 0x7f273c1557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:58.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f273b59e640 1 -- 172.21.15.22:0/241851297 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273c155d30 con 0x7f273c15c1c0 2024-03-29T04:17:58.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f273a59c640 1 --2- 172.21.15.22:0/241851297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f273c0d0900 0x7f273c1557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:58.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f273b59e640 1 --2- 172.21.15.22:0/241851297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f273c15c1c0 0x7f273c1533c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2734007e60 tx=0x7f273400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f27237fe640 1 -- 172.21.15.22:0/241851297 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734014420 con 0x7f273c15c1c0 2024-03-29T04:17:58.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f27237fe640 1 -- 172.21.15.22:0/241851297 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2734019070 con 0x7f273c15c1c0 2024-03-29T04:17:58.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f2741240640 1 -- 172.21.15.22:0/241851297 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f273c1659c0 con 0x7f273c15c1c0 2024-03-29T04:17:58.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.968+0000 7f27237fe640 1 -- 172.21.15.22:0/241851297 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734014800 con 0x7f273c15c1c0 2024-03-29T04:17:58.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.972+0000 7f2741240640 1 -- 172.21.15.22:0/241851297 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f273c165f80 con 0x7f273c15c1c0 2024-03-29T04:17:58.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.972+0000 7f2741240640 1 -- 172.21.15.22:0/241851297 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f273c068890 con 0x7f273c15c1c0 2024-03-29T04:17:58.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.972+0000 7f27237fe640 1 -- 172.21.15.22:0/241851297 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f27340149a0 con 0x7f273c15c1c0 2024-03-29T04:17:58.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.972+0000 7f27237fe640 1 --2- 172.21.15.22:0/241851297 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2728067fc0 0x7f272806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:58.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.972+0000 7f27237fe640 1 -- 172.21.15.22:0/241851297 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2734087b50 con 0x7f273c15c1c0 2024-03-29T04:17:58.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.972+0000 7f273ad9d640 1 --2- 172.21.15.22:0/241851297 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2728067fc0 0x7f272806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:58.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.972+0000 7f273ad9d640 1 --2- 172.21.15.22:0/241851297 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2728067fc0 0x7f272806a480 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f272c006fd0 tx=0x7f272c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:58.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:58.976+0000 7f27237fe640 1 -- 172.21.15.22:0/241851297 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f273c155d30 con 0x7f273c15c1c0 2024-03-29T04:17:59.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.128+0000 7f2741240640 1 -- 172.21.15.22:0/241851297 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f273c0b4d70 con 0x7f2728067fc0 2024-03-29T04:17:59.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.156+0000 7f27237fe640 1 -- 172.21.15.22:0/241851297 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f273c0b4d70 con 0x7f2728067fc0 2024-03-29T04:17:59.140 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:59.143 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-29T04:17:59.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.160+0000 7f2741240640 1 -- 172.21.15.22:0/241851297 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2728067fc0 msgr2=0x7f272806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:59.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.160+0000 7f2741240640 1 --2- 172.21.15.22:0/241851297 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2728067fc0 0x7f272806a480 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f272c006fd0 tx=0x7f272c008040 comp rx=0 tx=0).stop 2024-03-29T04:17:59.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.160+0000 7f2741240640 1 -- 172.21.15.22:0/241851297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f273c15c1c0 msgr2=0x7f273c1533c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:59.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.160+0000 7f2741240640 1 --2- 172.21.15.22:0/241851297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f273c15c1c0 0x7f273c1533c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2734007e60 tx=0x7f273400e4b0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.160+0000 7f2741240640 1 -- 172.21.15.22:0/241851297 shutdown_connections 2024-03-29T04:17:59.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.160+0000 7f2741240640 1 --2- 172.21.15.22:0/241851297 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2728067fc0 0x7f272806a480 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.160+0000 7f2741240640 1 --2- 172.21.15.22:0/241851297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f273c15c1c0 0x7f273c1533c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.160+0000 7f2741240640 1 --2- 172.21.15.22:0/241851297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f273c0d0900 0x7f273c1557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.160+0000 7f2741240640 1 --2- 172.21.15.22:0/241851297 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f273c0b5060 0x7f273c1653e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.160+0000 7f2741240640 1 -- 172.21.15.22:0/241851297 >> 172.21.15.22:0/241851297 conn(0x7f273c0b3d60 msgr2=0x7f273c157460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:59.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.160+0000 7f2741240640 1 -- 172.21.15.22:0/241851297 shutdown_connections 2024-03-29T04:17:59.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.160+0000 7f2741240640 1 -- 172.21.15.22:0/241851297 wait complete. 2024-03-29T04:17:59.160 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-03-29T04:17:57.699001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236208,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738512,"statfs":{"total":773094113280,"available":772852236288,"internally_reserved":0,"allocated":3481600,"data_stored":2086528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004600"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653655+0000","last_change":"2024-03-29T04:17:35.587951+0000","last_active":"2024-03-29T04:17:42.653655+0000","last_peered":"2024-03-29T04:17:42.653655+0000","last_clean":"2024-03-29T04:17:42.653655+0000","last_became_active":"2024-03-29T04:17:35.587644+0000","last_became_peered":"2024-03-29T04:17:35.587644+0000","last_unstale":"2024-03-29T04:17:42.653655+0000","last_undegraded":"2024-03-29T04:17:42.653655+0000","last_fullsized":"2024-03-29T04:17:42.653655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:07.631047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650468+0000","last_change":"2024-03-29T04:17:36.604695+0000","last_active":"2024-03-29T04:17:42.650468+0000","last_peered":"2024-03-29T04:17:42.650468+0000","last_clean":"2024-03-29T04:17:42.650468+0000","last_became_active":"2024-03-29T04:17:36.602172+0000","last_became_peered":"2024-03-29T04:17:36.602172+0000","last_unstale":"2024-03-29T04:17:42.650468+0000","last_undegraded":"2024-03-29T04:17:42.650468+0000","last_fullsized":"2024-03-29T04:17:42.650468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:30.729030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649719+0000","last_change":"2024-03-29T04:17:41.168150+0000","last_active":"2024-03-29T04:17:42.649719+0000","last_peered":"2024-03-29T04:17:42.649719+0000","last_clean":"2024-03-29T04:17:42.649719+0000","last_became_active":"2024-03-29T04:17:41.166854+0000","last_became_peered":"2024-03-29T04:17:41.166854+0000","last_unstale":"2024-03-29T04:17:42.649719+0000","last_undegraded":"2024-03-29T04:17:42.649719+0000","last_fullsized":"2024-03-29T04:17:42.649719+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:01:07.401841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650673+0000","last_change":"2024-03-29T04:17:35.575675+0000","last_active":"2024-03-29T04:17:42.650673+0000","last_peered":"2024-03-29T04:17:42.650673+0000","last_clean":"2024-03-29T04:17:42.650673+0000","last_became_active":"2024-03-29T04:17:35.575189+0000","last_became_peered":"2024-03-29T04:17:35.575189+0000","last_unstale":"2024-03-29T04:17:42.650673+0000","last_undegraded":"2024-03-29T04:17:42.650673+0000","last_fullsized":"2024-03-29T04:17:42.650673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:55:50.253366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653534+0000","last_change":"2024-03-29T04:17:36.598621+0000","last_active":"2024-03-29T04:17:42.653534+0000","last_peered":"2024-03-29T04:17:42.653534+0000","last_clean":"2024-03-29T04:17:42.653534+0000","last_became_active":"2024-03-29T04:17:36.596823+0000","last_became_peered":"2024-03-29T04:17:36.596823+0000","last_unstale":"2024-03-29T04:17:42.653534+0000","last_undegraded":"2024-03-29T04:17:42.653534+0000","last_fullsized":"2024-03-29T04:17:42.653534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:11.037419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656537+0000","last_change":"2024-03-29T04:17:40.663898+0000","last_active":"2024-03-29T04:17:42.656537+0000","last_peered":"2024-03-29T04:17:42.656537+0000","last_clean":"2024-03-29T04:17:42.656537+0000","last_became_active":"2024-03-29T04:17:40.663387+0000","last_became_peered":"2024-03-29T04:17:40.663387+0000","last_unstale":"2024-03-29T04:17:42.656537+0000","last_undegraded":"2024-03-29T04:17:42.656537+0000","last_fullsized":"2024-03-29T04:17:42.656537+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:46.993432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.635633+0000","last_change":"2024-03-29T04:17:35.604714+0000","last_active":"2024-03-29T04:17:44.635633+0000","last_peered":"2024-03-29T04:17:44.635633+0000","last_clean":"2024-03-29T04:17:44.635633+0000","last_became_active":"2024-03-29T04:17:35.604080+0000","last_became_peered":"2024-03-29T04:17:35.604080+0000","last_unstale":"2024-03-29T04:17:44.635633+0000","last_undegraded":"2024-03-29T04:17:44.635633+0000","last_fullsized":"2024-03-29T04:17:44.635633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:12:36.590426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553394+0000","last_change":"2024-03-29T04:17:36.615673+0000","last_active":"2024-03-29T04:17:43.553394+0000","last_peered":"2024-03-29T04:17:43.553394+0000","last_clean":"2024-03-29T04:17:43.553394+0000","last_became_active":"2024-03-29T04:17:36.615348+0000","last_became_peered":"2024-03-29T04:17:36.615348+0000","last_unstale":"2024-03-29T04:17:43.553394+0000","last_undegraded":"2024-03-29T04:17:43.553394+0000","last_fullsized":"2024-03-29T04:17:43.553394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:30.729030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656960+0000","last_change":"2024-03-29T04:17:40.653544+0000","last_active":"2024-03-29T04:17:42.656960+0000","last_peered":"2024-03-29T04:17:42.656960+0000","last_clean":"2024-03-29T04:17:42.656960+0000","last_became_active":"2024-03-29T04:17:40.652980+0000","last_became_peered":"2024-03-29T04:17:40.652980+0000","last_unstale":"2024-03-29T04:17:42.656960+0000","last_undegraded":"2024-03-29T04:17:42.656960+0000","last_fullsized":"2024-03-29T04:17:42.656960+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:25:02.717791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634331+0000","last_change":"2024-03-29T04:17:35.604849+0000","last_active":"2024-03-29T04:17:44.634331+0000","last_peered":"2024-03-29T04:17:44.634331+0000","last_clean":"2024-03-29T04:17:44.634331+0000","last_became_active":"2024-03-29T04:17:35.604004+0000","last_became_peered":"2024-03-29T04:17:35.604004+0000","last_unstale":"2024-03-29T04:17:44.634331+0000","last_undegraded":"2024-03-29T04:17:44.634331+0000","last_fullsized":"2024-03-29T04:17:44.634331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:50:11.912084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552984+0000","last_change":"2024-03-29T04:17:36.592055+0000","last_active":"2024-03-29T04:17:43.552984+0000","last_peered":"2024-03-29T04:17:43.552984+0000","last_clean":"2024-03-29T04:17:43.552984+0000","last_became_active":"2024-03-29T04:17:36.588805+0000","last_became_peered":"2024-03-29T04:17:36.588805+0000","last_unstale":"2024-03-29T04:17:43.552984+0000","last_undegraded":"2024-03-29T04:17:43.552984+0000","last_fullsized":"2024-03-29T04:17:43.552984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:13.677092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656938+0000","last_change":"2024-03-29T04:17:40.651131+0000","last_active":"2024-03-29T04:17:42.656938+0000","last_peered":"2024-03-29T04:17:42.656938+0000","last_clean":"2024-03-29T04:17:42.656938+0000","last_became_active":"2024-03-29T04:17:40.650319+0000","last_became_peered":"2024-03-29T04:17:40.650319+0000","last_unstale":"2024-03-29T04:17:42.656938+0000","last_undegraded":"2024-03-29T04:17:42.656938+0000","last_fullsized":"2024-03-29T04:17:42.656938+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:42:25.113757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.269694+0000","last_change":"2024-03-29T04:17:36.620761+0000","last_active":"2024-03-29T04:17:44.269694+0000","last_peered":"2024-03-29T04:17:44.269694+0000","last_clean":"2024-03-29T04:17:44.269694+0000","last_became_active":"2024-03-29T04:17:36.620483+0000","last_became_peered":"2024-03-29T04:17:36.620483+0000","last_unstale":"2024-03-29T04:17:44.269694+0000","last_undegraded":"2024-03-29T04:17:44.269694+0000","last_fullsized":"2024-03-29T04:17:44.269694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:10:00.802551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634192+0000","last_change":"2024-03-29T04:17:35.602583+0000","last_active":"2024-03-29T04:17:44.634192+0000","last_peered":"2024-03-29T04:17:44.634192+0000","last_clean":"2024-03-29T04:17:44.634192+0000","last_became_active":"2024-03-29T04:17:35.601902+0000","last_became_peered":"2024-03-29T04:17:35.601902+0000","last_unstale":"2024-03-29T04:17:44.634192+0000","last_undegraded":"2024-03-29T04:17:44.634192+0000","last_fullsized":"2024-03-29T04:17:44.634192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:43:57.713086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.658621+0000","last_change":"2024-03-29T04:17:40.661925+0000","last_active":"2024-03-29T04:17:42.658621+0000","last_peered":"2024-03-29T04:17:42.658621+0000","last_clean":"2024-03-29T04:17:42.658621+0000","last_became_active":"2024-03-29T04:17:40.661028+0000","last_became_peered":"2024-03-29T04:17:40.661028+0000","last_unstale":"2024-03-29T04:17:42.658621+0000","last_undegraded":"2024-03-29T04:17:42.658621+0000","last_fullsized":"2024-03-29T04:17:42.658621+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:33:27.957344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650137+0000","last_change":"2024-03-29T04:17:36.620569+0000","last_active":"2024-03-29T04:17:42.650137+0000","last_peered":"2024-03-29T04:17:42.650137+0000","last_clean":"2024-03-29T04:17:42.650137+0000","last_became_active":"2024-03-29T04:17:36.620079+0000","last_became_peered":"2024-03-29T04:17:36.620079+0000","last_unstale":"2024-03-29T04:17:42.650137+0000","last_undegraded":"2024-03-29T04:17:42.650137+0000","last_fullsized":"2024-03-29T04:17:42.650137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:41.872399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.469887+0000","last_change":"2024-03-29T04:17:35.577167+0000","last_active":"2024-03-29T04:17:44.469887+0000","last_peered":"2024-03-29T04:17:44.469887+0000","last_clean":"2024-03-29T04:17:44.469887+0000","last_became_active":"2024-03-29T04:17:35.576145+0000","last_became_peered":"2024-03-29T04:17:35.576145+0000","last_unstale":"2024-03-29T04:17:44.469887+0000","last_undegraded":"2024-03-29T04:17:44.469887+0000","last_fullsized":"2024-03-29T04:17:44.469887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:11:43.663978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.657612+0000","last_change":"2024-03-29T04:17:40.664962+0000","last_active":"2024-03-29T04:17:42.657612+0000","last_peered":"2024-03-29T04:17:42.657612+0000","last_clean":"2024-03-29T04:17:42.657612+0000","last_became_active":"2024-03-29T04:17:40.664470+0000","last_became_peered":"2024-03-29T04:17:40.664470+0000","last_unstale":"2024-03-29T04:17:42.657612+0000","last_undegraded":"2024-03-29T04:17:42.657612+0000","last_fullsized":"2024-03-29T04:17:42.657612+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:43:44.093178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650038+0000","last_change":"2024-03-29T04:17:36.617517+0000","last_active":"2024-03-29T04:17:42.650038+0000","last_peered":"2024-03-29T04:17:42.650038+0000","last_clean":"2024-03-29T04:17:42.650038+0000","last_became_active":"2024-03-29T04:17:36.616399+0000","last_became_peered":"2024-03-29T04:17:36.616399+0000","last_unstale":"2024-03-29T04:17:42.650038+0000","last_undegraded":"2024-03-29T04:17:42.650038+0000","last_fullsized":"2024-03-29T04:17:42.650038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:24:47.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470278+0000","last_change":"2024-03-29T04:17:35.576729+0000","last_active":"2024-03-29T04:17:44.470278+0000","last_peered":"2024-03-29T04:17:44.470278+0000","last_clean":"2024-03-29T04:17:44.470278+0000","last_became_active":"2024-03-29T04:17:35.576446+0000","last_became_peered":"2024-03-29T04:17:35.576446+0000","last_unstale":"2024-03-29T04:17:44.470278+0000","last_undegraded":"2024-03-29T04:17:44.470278+0000","last_fullsized":"2024-03-29T04:17:44.470278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:53.508449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470815+0000","last_change":"2024-03-29T04:17:41.170866+0000","last_active":"2024-03-29T04:17:44.470815+0000","last_peered":"2024-03-29T04:17:44.470815+0000","last_clean":"2024-03-29T04:17:44.470815+0000","last_became_active":"2024-03-29T04:17:41.169521+0000","last_became_peered":"2024-03-29T04:17:41.169521+0000","last_unstale":"2024-03-29T04:17:44.470815+0000","last_undegraded":"2024-03-29T04:17:44.470815+0000","last_fullsized":"2024-03-29T04:17:44.470815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:33:27.957344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.657095+0000","last_change":"2024-03-29T04:17:36.624392+0000","last_active":"2024-03-29T04:17:42.657095+0000","last_peered":"2024-03-29T04:17:42.657095+0000","last_clean":"2024-03-29T04:17:42.657095+0000","last_became_active":"2024-03-29T04:17:36.612708+0000","last_became_peered":"2024-03-29T04:17:36.612708+0000","last_unstale":"2024-03-29T04:17:42.657095+0000","last_undegraded":"2024-03-29T04:17:42.657095+0000","last_fullsized":"2024-03-29T04:17:42.657095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:59.355649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634144+0000","last_change":"2024-03-29T04:17:35.596985+0000","last_active":"2024-03-29T04:17:44.634144+0000","last_peered":"2024-03-29T04:17:44.634144+0000","last_clean":"2024-03-29T04:17:44.634144+0000","last_became_active":"2024-03-29T04:17:35.595917+0000","last_became_peered":"2024-03-29T04:17:35.595917+0000","last_unstale":"2024-03-29T04:17:44.634144+0000","last_undegraded":"2024-03-29T04:17:44.634144+0000","last_fullsized":"2024-03-29T04:17:44.634144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:24:46.122034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.658384+0000","last_change":"2024-03-29T04:17:40.664618+0000","last_active":"2024-03-29T04:17:42.658384+0000","last_peered":"2024-03-29T04:17:42.658384+0000","last_clean":"2024-03-29T04:17:42.658384+0000","last_became_active":"2024-03-29T04:17:40.664144+0000","last_became_peered":"2024-03-29T04:17:40.664144+0000","last_unstale":"2024-03-29T04:17:42.658384+0000","last_undegraded":"2024-03-29T04:17:42.658384+0000","last_fullsized":"2024-03-29T04:17:42.658384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:59:47.099511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653827+0000","last_change":"2024-03-29T04:17:36.596925+0000","last_active":"2024-03-29T04:17:42.653827+0000","last_peered":"2024-03-29T04:17:42.653827+0000","last_clean":"2024-03-29T04:17:42.653827+0000","last_became_active":"2024-03-29T04:17:36.596307+0000","last_became_peered":"2024-03-29T04:17:36.596307+0000","last_unstale":"2024-03-29T04:17:42.653827+0000","last_undegraded":"2024-03-29T04:17:42.653827+0000","last_fullsized":"2024-03-29T04:17:42.653827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:39.635039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.271188+0000","last_change":"2024-03-29T04:17:35.598885+0000","last_active":"2024-03-29T04:17:44.271188+0000","last_peered":"2024-03-29T04:17:44.271188+0000","last_clean":"2024-03-29T04:17:44.271188+0000","last_became_active":"2024-03-29T04:17:35.598155+0000","last_became_peered":"2024-03-29T04:17:35.598155+0000","last_unstale":"2024-03-29T04:17:44.271188+0000","last_undegraded":"2024-03-29T04:17:44.271188+0000","last_fullsized":"2024-03-29T04:17:44.271188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:58.346822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656759+0000","last_change":"2024-03-29T04:17:41.164185+0000","last_active":"2024-03-29T04:17:42.656759+0000","last_peered":"2024-03-29T04:17:42.656759+0000","last_clean":"2024-03-29T04:17:42.656759+0000","last_became_active":"2024-03-29T04:17:41.163368+0000","last_became_peered":"2024-03-29T04:17:41.163368+0000","last_unstale":"2024-03-29T04:17:42.656759+0000","last_undegraded":"2024-03-29T04:17:42.656759+0000","last_fullsized":"2024-03-29T04:17:42.656759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:04:15.384842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650724+0000","last_change":"2024-03-29T04:17:36.611583+0000","last_active":"2024-03-29T04:17:42.650724+0000","last_peered":"2024-03-29T04:17:42.650724+0000","last_clean":"2024-03-29T04:17:42.650724+0000","last_became_active":"2024-03-29T04:17:36.611204+0000","last_became_peered":"2024-03-29T04:17:36.611204+0000","last_unstale":"2024-03-29T04:17:42.650724+0000","last_undegraded":"2024-03-29T04:17:42.650724+0000","last_fullsized":"2024-03-29T04:17:42.650724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:13.677092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470861+0000","last_change":"2024-03-29T04:17:35.574541+0000","last_active":"2024-03-29T04:17:44.470861+0000","last_peered":"2024-03-29T04:17:44.470861+0000","last_clean":"2024-03-29T04:17:44.470861+0000","last_became_active":"2024-03-29T04:17:35.573297+0000","last_became_peered":"2024-03-29T04:17:35.573297+0000","last_unstale":"2024-03-29T04:17:44.470861+0000","last_undegraded":"2024-03-29T04:17:44.470861+0000","last_fullsized":"2024-03-29T04:17:44.470861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:58.346822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649700+0000","last_change":"2024-03-29T04:17:41.165155+0000","last_active":"2024-03-29T04:17:42.649700+0000","last_peered":"2024-03-29T04:17:42.649700+0000","last_clean":"2024-03-29T04:17:42.649700+0000","last_became_active":"2024-03-29T04:17:41.163801+0000","last_became_peered":"2024-03-29T04:17:41.163801+0000","last_unstale":"2024-03-29T04:17:42.649700+0000","last_undegraded":"2024-03-29T04:17:42.649700+0000","last_fullsized":"2024-03-29T04:17:42.649700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:05:37.595239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470679+0000","last_change":"2024-03-29T04:17:36.590406+0000","last_active":"2024-03-29T04:17:44.470679+0000","last_peered":"2024-03-29T04:17:44.470679+0000","last_clean":"2024-03-29T04:17:44.470679+0000","last_became_active":"2024-03-29T04:17:36.589975+0000","last_became_peered":"2024-03-29T04:17:36.589975+0000","last_unstale":"2024-03-29T04:17:44.470679+0000","last_undegraded":"2024-03-29T04:17:44.470679+0000","last_fullsized":"2024-03-29T04:17:44.470679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:39.635039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634555+0000","last_change":"2024-03-29T04:17:35.602939+0000","last_active":"2024-03-29T04:17:44.634555+0000","last_peered":"2024-03-29T04:17:44.634555+0000","last_clean":"2024-03-29T04:17:44.634555+0000","last_became_active":"2024-03-29T04:17:35.602250+0000","last_became_peered":"2024-03-29T04:17:35.602250+0000","last_unstale":"2024-03-29T04:17:44.634555+0000","last_undegraded":"2024-03-29T04:17:44.634555+0000","last_fullsized":"2024-03-29T04:17:44.634555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:16:13.431728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653145+0000","last_change":"2024-03-29T04:17:41.165561+0000","last_active":"2024-03-29T04:17:42.653145+0000","last_peered":"2024-03-29T04:17:42.653145+0000","last_clean":"2024-03-29T04:17:42.653145+0000","last_became_active":"2024-03-29T04:17:41.163703+0000","last_became_peered":"2024-03-29T04:17:41.163703+0000","last_unstale":"2024-03-29T04:17:42.653145+0000","last_undegraded":"2024-03-29T04:17:42.653145+0000","last_fullsized":"2024-03-29T04:17:42.653145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:10:04.139549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553157+0000","last_change":"2024-03-29T04:17:36.602865+0000","last_active":"2024-03-29T04:17:43.553157+0000","last_peered":"2024-03-29T04:17:43.553157+0000","last_clean":"2024-03-29T04:17:43.553157+0000","last_became_active":"2024-03-29T04:17:36.602578+0000","last_became_peered":"2024-03-29T04:17:36.602578+0000","last_unstale":"2024-03-29T04:17:43.553157+0000","last_undegraded":"2024-03-29T04:17:43.553157+0000","last_fullsized":"2024-03-29T04:17:43.553157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:55:11.413735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.469877+0000","last_change":"2024-03-29T04:17:35.560382+0000","last_active":"2024-03-29T04:17:44.469877+0000","last_peered":"2024-03-29T04:17:44.469877+0000","last_clean":"2024-03-29T04:17:44.469877+0000","last_became_active":"2024-03-29T04:17:35.560122+0000","last_became_peered":"2024-03-29T04:17:35.560122+0000","last_unstale":"2024-03-29T04:17:44.469877+0000","last_undegraded":"2024-03-29T04:17:44.469877+0000","last_fullsized":"2024-03-29T04:17:44.469877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:52:18.910727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470397+0000","last_change":"2024-03-29T04:17:40.663870+0000","last_active":"2024-03-29T04:17:44.470397+0000","last_peered":"2024-03-29T04:17:44.470397+0000","last_clean":"2024-03-29T04:17:44.470397+0000","last_became_active":"2024-03-29T04:17:40.662957+0000","last_became_peered":"2024-03-29T04:17:40.662957+0000","last_unstale":"2024-03-29T04:17:44.470397+0000","last_undegraded":"2024-03-29T04:17:44.470397+0000","last_fullsized":"2024-03-29T04:17:44.470397+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:10:04.139549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.657408+0000","last_change":"2024-03-29T04:17:36.624632+0000","last_active":"2024-03-29T04:17:42.657408+0000","last_peered":"2024-03-29T04:17:42.657408+0000","last_clean":"2024-03-29T04:17:42.657408+0000","last_became_active":"2024-03-29T04:17:36.621308+0000","last_became_peered":"2024-03-29T04:17:36.621308+0000","last_unstale":"2024-03-29T04:17:42.657408+0000","last_undegraded":"2024-03-29T04:17:42.657408+0000","last_fullsized":"2024-03-29T04:17:42.657408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:11:44.672805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553434+0000","last_change":"2024-03-29T04:17:35.590306+0000","last_active":"2024-03-29T04:17:43.553434+0000","last_peered":"2024-03-29T04:17:43.553434+0000","last_clean":"2024-03-29T04:17:43.553434+0000","last_became_active":"2024-03-29T04:17:35.582422+0000","last_became_peered":"2024-03-29T04:17:35.582422+0000","last_unstale":"2024-03-29T04:17:43.553434+0000","last_undegraded":"2024-03-29T04:17:43.553434+0000","last_fullsized":"2024-03-29T04:17:43.553434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:58.346822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470758+0000","last_change":"2024-03-29T04:17:40.661919+0000","last_active":"2024-03-29T04:17:44.470758+0000","last_peered":"2024-03-29T04:17:44.470758+0000","last_clean":"2024-03-29T04:17:44.470758+0000","last_became_active":"2024-03-29T04:17:40.661251+0000","last_became_peered":"2024-03-29T04:17:40.661251+0000","last_unstale":"2024-03-29T04:17:44.470758+0000","last_undegraded":"2024-03-29T04:17:44.470758+0000","last_fullsized":"2024-03-29T04:17:44.470758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:59:47.099511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470257+0000","last_change":"2024-03-29T04:17:36.590933+0000","last_active":"2024-03-29T04:17:44.470257+0000","last_peered":"2024-03-29T04:17:44.470257+0000","last_clean":"2024-03-29T04:17:44.470257+0000","last_became_active":"2024-03-29T04:17:36.590535+0000","last_became_peered":"2024-03-29T04:17:36.590535+0000","last_unstale":"2024-03-29T04:17:44.470257+0000","last_undegraded":"2024-03-29T04:17:44.470257+0000","last_fullsized":"2024-03-29T04:17:44.470257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:41.872399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578021+0000","last_change":"2024-03-29T04:17:35.565257+0000","last_active":"2024-03-29T04:17:43.578021+0000","last_peered":"2024-03-29T04:17:43.578021+0000","last_clean":"2024-03-29T04:17:43.578021+0000","last_became_active":"2024-03-29T04:17:35.564547+0000","last_became_peered":"2024-03-29T04:17:35.564547+0000","last_unstale":"2024-03-29T04:17:43.578021+0000","last_undegraded":"2024-03-29T04:17:43.578021+0000","last_fullsized":"2024-03-29T04:17:43.578021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:04:44.150824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470921+0000","last_change":"2024-03-29T04:17:40.657201+0000","last_active":"2024-03-29T04:17:44.470921+0000","last_peered":"2024-03-29T04:17:44.470921+0000","last_clean":"2024-03-29T04:17:44.470921+0000","last_became_active":"2024-03-29T04:17:40.655333+0000","last_became_peered":"2024-03-29T04:17:40.655333+0000","last_unstale":"2024-03-29T04:17:44.470921+0000","last_undegraded":"2024-03-29T04:17:44.470921+0000","last_fullsized":"2024-03-29T04:17:44.470921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:37:53.228170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.652829+0000","last_change":"2024-03-29T04:17:36.625393+0000","last_active":"2024-03-29T04:17:42.652829+0000","last_peered":"2024-03-29T04:17:42.652829+0000","last_clean":"2024-03-29T04:17:42.652829+0000","last_became_active":"2024-03-29T04:17:36.625053+0000","last_became_peered":"2024-03-29T04:17:36.625053+0000","last_unstale":"2024-03-29T04:17:42.652829+0000","last_undegraded":"2024-03-29T04:17:42.652829+0000","last_fullsized":"2024-03-29T04:17:42.652829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:16:58.574477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.635249+0000","last_change":"2024-03-29T04:17:35.597695+0000","last_active":"2024-03-29T04:17:44.635249+0000","last_peered":"2024-03-29T04:17:44.635249+0000","last_clean":"2024-03-29T04:17:44.635249+0000","last_became_active":"2024-03-29T04:17:35.596065+0000","last_became_peered":"2024-03-29T04:17:35.596065+0000","last_unstale":"2024-03-29T04:17:44.635249+0000","last_undegraded":"2024-03-29T04:17:44.635249+0000","last_fullsized":"2024-03-29T04:17:44.635249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:11:43.663978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653676+0000","last_change":"2024-03-29T04:17:41.169719+0000","last_active":"2024-03-29T04:17:42.653676+0000","last_peered":"2024-03-29T04:17:42.653676+0000","last_clean":"2024-03-29T04:17:42.653676+0000","last_became_active":"2024-03-29T04:17:41.168241+0000","last_became_peered":"2024-03-29T04:17:41.168241+0000","last_unstale":"2024-03-29T04:17:42.653676+0000","last_undegraded":"2024-03-29T04:17:42.653676+0000","last_fullsized":"2024-03-29T04:17:42.653676+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:37:53.228170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649994+0000","last_change":"2024-03-29T04:17:36.599562+0000","last_active":"2024-03-29T04:17:42.649994+0000","last_peered":"2024-03-29T04:17:42.649994+0000","last_clean":"2024-03-29T04:17:42.649994+0000","last_became_active":"2024-03-29T04:17:36.597078+0000","last_became_peered":"2024-03-29T04:17:36.597078+0000","last_unstale":"2024-03-29T04:17:42.649994+0000","last_undegraded":"2024-03-29T04:17:42.649994+0000","last_fullsized":"2024-03-29T04:17:42.649994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:43:27.568676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553439+0000","last_change":"2024-03-29T04:17:35.589469+0000","last_active":"2024-03-29T04:17:43.553439+0000","last_peered":"2024-03-29T04:17:43.553439+0000","last_clean":"2024-03-29T04:17:43.553439+0000","last_became_active":"2024-03-29T04:17:35.572474+0000","last_became_peered":"2024-03-29T04:17:35.572474+0000","last_unstale":"2024-03-29T04:17:43.553439+0000","last_undegraded":"2024-03-29T04:17:43.553439+0000","last_fullsized":"2024-03-29T04:17:43.553439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:55:50.253366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.635415+0000","last_change":"2024-03-29T04:17:25.361826+0000","last_active":"2024-03-29T04:17:44.635415+0000","last_peered":"2024-03-29T04:17:44.635415+0000","last_clean":"2024-03-29T04:17:44.635415+0000","last_became_active":"2024-03-29T04:17:25.361259+0000","last_became_peered":"2024-03-29T04:17:25.361259+0000","last_unstale":"2024-03-29T04:17:44.635415+0000","last_undegraded":"2024-03-29T04:17:44.635415+0000","last_fullsized":"2024-03-29T04:17:44.635415+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_clean_scrub_stamp":"2024-03-29T04:17:24.334639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:40.027873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270077+0000","last_change":"2024-03-29T04:17:41.167132+0000","last_active":"2024-03-29T04:17:44.270077+0000","last_peered":"2024-03-29T04:17:44.270077+0000","last_clean":"2024-03-29T04:17:44.270077+0000","last_became_active":"2024-03-29T04:17:41.166660+0000","last_became_peered":"2024-03-29T04:17:41.166660+0000","last_unstale":"2024-03-29T04:17:44.270077+0000","last_undegraded":"2024-03-29T04:17:44.270077+0000","last_fullsized":"2024-03-29T04:17:44.270077+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:42:25.113757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634556+0000","last_change":"2024-03-29T04:17:36.612535+0000","last_active":"2024-03-29T04:17:44.634556+0000","last_peered":"2024-03-29T04:17:44.634556+0000","last_clean":"2024-03-29T04:17:44.634556+0000","last_became_active":"2024-03-29T04:17:36.610106+0000","last_became_peered":"2024-03-29T04:17:36.610106+0000","last_unstale":"2024-03-29T04:17:44.634556+0000","last_undegraded":"2024-03-29T04:17:44.634556+0000","last_fullsized":"2024-03-29T04:17:44.634556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:24:58.639606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578939+0000","last_change":"2024-03-29T04:17:35.600209+0000","last_active":"2024-03-29T04:17:43.578939+0000","last_peered":"2024-03-29T04:17:43.578939+0000","last_clean":"2024-03-29T04:17:43.578939+0000","last_became_active":"2024-03-29T04:17:35.599686+0000","last_became_peered":"2024-03-29T04:17:35.599686+0000","last_unstale":"2024-03-29T04:17:43.578939+0000","last_undegraded":"2024-03-29T04:17:43.578939+0000","last_fullsized":"2024-03-29T04:17:43.578939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:55:50.253366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.269503+0000","last_change":"2024-03-29T04:17:41.165901+0000","last_active":"2024-03-29T04:17:44.269503+0000","last_peered":"2024-03-29T04:17:44.269503+0000","last_clean":"2024-03-29T04:17:44.269503+0000","last_became_active":"2024-03-29T04:17:41.165453+0000","last_became_peered":"2024-03-29T04:17:41.165453+0000","last_unstale":"2024-03-29T04:17:44.269503+0000","last_undegraded":"2024-03-29T04:17:44.269503+0000","last_fullsized":"2024-03-29T04:17:44.269503+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:04:15.384842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553553+0000","last_change":"2024-03-29T04:17:36.596650+0000","last_active":"2024-03-29T04:17:43.553553+0000","last_peered":"2024-03-29T04:17:43.553553+0000","last_clean":"2024-03-29T04:17:43.553553+0000","last_became_active":"2024-03-29T04:17:36.596458+0000","last_became_peered":"2024-03-29T04:17:36.596458+0000","last_unstale":"2024-03-29T04:17:43.553553+0000","last_undegraded":"2024-03-29T04:17:43.553553+0000","last_fullsized":"2024-03-29T04:17:43.553553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:42:46.422245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.469894+0000","last_change":"2024-03-29T04:17:35.554693+0000","last_active":"2024-03-29T04:17:44.469894+0000","last_peered":"2024-03-29T04:17:44.469894+0000","last_clean":"2024-03-29T04:17:44.469894+0000","last_became_active":"2024-03-29T04:17:35.554347+0000","last_became_peered":"2024-03-29T04:17:35.554347+0000","last_unstale":"2024-03-29T04:17:44.469894+0000","last_undegraded":"2024-03-29T04:17:44.469894+0000","last_fullsized":"2024-03-29T04:17:44.469894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:24:46.122034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653319+0000","last_change":"2024-03-29T04:17:40.656032+0000","last_active":"2024-03-29T04:17:42.653319+0000","last_peered":"2024-03-29T04:17:42.653319+0000","last_clean":"2024-03-29T04:17:42.653319+0000","last_became_active":"2024-03-29T04:17:40.654947+0000","last_became_peered":"2024-03-29T04:17:40.654947+0000","last_unstale":"2024-03-29T04:17:42.653319+0000","last_undegraded":"2024-03-29T04:17:42.653319+0000","last_fullsized":"2024-03-29T04:17:42.653319+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:59:47.099511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634965+0000","last_change":"2024-03-29T04:17:36.627875+0000","last_active":"2024-03-29T04:17:44.634965+0000","last_peered":"2024-03-29T04:17:44.634965+0000","last_clean":"2024-03-29T04:17:44.634965+0000","last_became_active":"2024-03-29T04:17:36.627498+0000","last_became_peered":"2024-03-29T04:17:36.627498+0000","last_unstale":"2024-03-29T04:17:44.634965+0000","last_undegraded":"2024-03-29T04:17:44.634965+0000","last_fullsized":"2024-03-29T04:17:44.634965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:10:00.061365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578092+0000","last_change":"2024-03-29T04:17:35.568135+0000","last_active":"2024-03-29T04:17:43.578092+0000","last_peered":"2024-03-29T04:17:43.578092+0000","last_clean":"2024-03-29T04:17:43.578092+0000","last_became_active":"2024-03-29T04:17:35.567800+0000","last_became_peered":"2024-03-29T04:17:35.567800+0000","last_unstale":"2024-03-29T04:17:43.578092+0000","last_undegraded":"2024-03-29T04:17:43.578092+0000","last_fullsized":"2024-03-29T04:17:43.578092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:23.283694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552529+0000","last_change":"2024-03-29T04:17:41.166538+0000","last_active":"2024-03-29T04:17:43.552529+0000","last_peered":"2024-03-29T04:17:43.552529+0000","last_clean":"2024-03-29T04:17:43.552529+0000","last_became_active":"2024-03-29T04:17:41.165914+0000","last_became_peered":"2024-03-29T04:17:41.165914+0000","last_unstale":"2024-03-29T04:17:43.552529+0000","last_undegraded":"2024-03-29T04:17:43.552529+0000","last_fullsized":"2024-03-29T04:17:43.552529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:37:53.228170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634358+0000","last_change":"2024-03-29T04:17:36.626068+0000","last_active":"2024-03-29T04:17:44.634358+0000","last_peered":"2024-03-29T04:17:44.634358+0000","last_clean":"2024-03-29T04:17:44.634358+0000","last_became_active":"2024-03-29T04:17:36.625776+0000","last_became_peered":"2024-03-29T04:17:36.625776+0000","last_unstale":"2024-03-29T04:17:44.634358+0000","last_undegraded":"2024-03-29T04:17:44.634358+0000","last_fullsized":"2024-03-29T04:17:44.634358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:55:11.413735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.658456+0000","last_change":"2024-03-29T04:17:35.573245+0000","last_active":"2024-03-29T04:17:42.658456+0000","last_peered":"2024-03-29T04:17:42.658456+0000","last_clean":"2024-03-29T04:17:42.658456+0000","last_became_active":"2024-03-29T04:17:35.572324+0000","last_became_peered":"2024-03-29T04:17:35.572324+0000","last_unstale":"2024-03-29T04:17:42.658456+0000","last_undegraded":"2024-03-29T04:17:42.658456+0000","last_fullsized":"2024-03-29T04:17:42.658456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:07:32.265670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656484+0000","last_change":"2024-03-29T04:17:40.653790+0000","last_active":"2024-03-29T04:17:42.656484+0000","last_peered":"2024-03-29T04:17:42.656484+0000","last_clean":"2024-03-29T04:17:42.656484+0000","last_became_active":"2024-03-29T04:17:40.653115+0000","last_became_peered":"2024-03-29T04:17:40.653115+0000","last_unstale":"2024-03-29T04:17:42.656484+0000","last_undegraded":"2024-03-29T04:17:42.656484+0000","last_fullsized":"2024-03-29T04:17:42.656484+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:55:15.491920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634122+0000","last_change":"2024-03-29T04:17:36.625368+0000","last_active":"2024-03-29T04:17:44.634122+0000","last_peered":"2024-03-29T04:17:44.634122+0000","last_clean":"2024-03-29T04:17:44.634122+0000","last_became_active":"2024-03-29T04:17:36.625073+0000","last_became_peered":"2024-03-29T04:17:36.625073+0000","last_unstale":"2024-03-29T04:17:44.634122+0000","last_undegraded":"2024-03-29T04:17:44.634122+0000","last_fullsized":"2024-03-29T04:17:44.634122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:13.677092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578401+0000","last_change":"2024-03-29T04:17:35.564073+0000","last_active":"2024-03-29T04:17:43.578401+0000","last_peered":"2024-03-29T04:17:43.578401+0000","last_clean":"2024-03-29T04:17:43.578401+0000","last_became_active":"2024-03-29T04:17:35.563521+0000","last_became_peered":"2024-03-29T04:17:35.563521+0000","last_unstale":"2024-03-29T04:17:43.578401+0000","last_undegraded":"2024-03-29T04:17:43.578401+0000","last_fullsized":"2024-03-29T04:17:43.578401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:53.508449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653476+0000","last_change":"2024-03-29T04:17:40.648424+0000","last_active":"2024-03-29T04:17:42.653476+0000","last_peered":"2024-03-29T04:17:42.653476+0000","last_clean":"2024-03-29T04:17:42.653476+0000","last_became_active":"2024-03-29T04:17:40.647665+0000","last_became_peered":"2024-03-29T04:17:40.647665+0000","last_unstale":"2024-03-29T04:17:42.653476+0000","last_undegraded":"2024-03-29T04:17:42.653476+0000","last_fullsized":"2024-03-29T04:17:42.653476+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:33:27.957344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.269744+0000","last_change":"2024-03-29T04:17:36.614419+0000","last_active":"2024-03-29T04:17:44.269744+0000","last_peered":"2024-03-29T04:17:44.269744+0000","last_clean":"2024-03-29T04:17:44.269744+0000","last_became_active":"2024-03-29T04:17:36.614052+0000","last_became_peered":"2024-03-29T04:17:36.614052+0000","last_unstale":"2024-03-29T04:17:44.269744+0000","last_undegraded":"2024-03-29T04:17:44.269744+0000","last_fullsized":"2024-03-29T04:17:44.269744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:41:31.910133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.268906+0000","last_change":"2024-03-29T04:17:35.576576+0000","last_active":"2024-03-29T04:17:44.268906+0000","last_peered":"2024-03-29T04:17:44.268906+0000","last_clean":"2024-03-29T04:17:44.268906+0000","last_became_active":"2024-03-29T04:17:35.575156+0000","last_became_peered":"2024-03-29T04:17:35.575156+0000","last_unstale":"2024-03-29T04:17:44.268906+0000","last_undegraded":"2024-03-29T04:17:44.268906+0000","last_fullsized":"2024-03-29T04:17:44.268906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:52:18.910727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650517+0000","last_change":"2024-03-29T04:17:41.165455+0000","last_active":"2024-03-29T04:17:42.650517+0000","last_peered":"2024-03-29T04:17:42.650517+0000","last_clean":"2024-03-29T04:17:42.650517+0000","last_became_active":"2024-03-29T04:17:41.164305+0000","last_became_peered":"2024-03-29T04:17:41.164305+0000","last_unstale":"2024-03-29T04:17:42.650517+0000","last_undegraded":"2024-03-29T04:17:42.650517+0000","last_fullsized":"2024-03-29T04:17:42.650517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:58:14.948478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634370+0000","last_change":"2024-03-29T04:17:36.626514+0000","last_active":"2024-03-29T04:17:44.634370+0000","last_peered":"2024-03-29T04:17:44.634370+0000","last_clean":"2024-03-29T04:17:44.634370+0000","last_became_active":"2024-03-29T04:17:36.626184+0000","last_became_peered":"2024-03-29T04:17:36.626184+0000","last_unstale":"2024-03-29T04:17:44.634370+0000","last_undegraded":"2024-03-29T04:17:44.634370+0000","last_fullsized":"2024-03-29T04:17:44.634370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:42:46.422245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553262+0000","last_change":"2024-03-29T04:17:35.599574+0000","last_active":"2024-03-29T04:17:43.553262+0000","last_peered":"2024-03-29T04:17:43.553262+0000","last_clean":"2024-03-29T04:17:43.553262+0000","last_became_active":"2024-03-29T04:17:35.589264+0000","last_became_peered":"2024-03-29T04:17:35.589264+0000","last_unstale":"2024-03-29T04:17:43.553262+0000","last_undegraded":"2024-03-29T04:17:43.553262+0000","last_fullsized":"2024-03-29T04:17:43.553262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:52:18.910727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650586+0000","last_change":"2024-03-29T04:17:41.164863+0000","last_active":"2024-03-29T04:17:42.650586+0000","last_peered":"2024-03-29T04:17:42.650586+0000","last_clean":"2024-03-29T04:17:42.650586+0000","last_became_active":"2024-03-29T04:17:41.163756+0000","last_became_peered":"2024-03-29T04:17:41.163756+0000","last_unstale":"2024-03-29T04:17:42.650586+0000","last_undegraded":"2024-03-29T04:17:42.650586+0000","last_fullsized":"2024-03-29T04:17:42.650586+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:54.890778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653149+0000","last_change":"2024-03-29T04:17:36.611832+0000","last_active":"2024-03-29T04:17:42.653149+0000","last_peered":"2024-03-29T04:17:42.653149+0000","last_clean":"2024-03-29T04:17:42.653149+0000","last_became_active":"2024-03-29T04:17:36.611106+0000","last_became_peered":"2024-03-29T04:17:36.611106+0000","last_unstale":"2024-03-29T04:17:42.653149+0000","last_undegraded":"2024-03-29T04:17:42.653149+0000","last_fullsized":"2024-03-29T04:17:42.653149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:10:00.802551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653387+0000","last_change":"2024-03-29T04:17:35.569650+0000","last_active":"2024-03-29T04:17:42.653387+0000","last_peered":"2024-03-29T04:17:42.653387+0000","last_clean":"2024-03-29T04:17:42.653387+0000","last_became_active":"2024-03-29T04:17:35.568995+0000","last_became_peered":"2024-03-29T04:17:35.568995+0000","last_unstale":"2024-03-29T04:17:42.653387+0000","last_undegraded":"2024-03-29T04:17:42.653387+0000","last_fullsized":"2024-03-29T04:17:42.653387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:55:50.253366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552920+0000","last_change":"2024-03-29T04:17:41.166364+0000","last_active":"2024-03-29T04:17:43.552920+0000","last_peered":"2024-03-29T04:17:43.552920+0000","last_clean":"2024-03-29T04:17:43.552920+0000","last_became_active":"2024-03-29T04:17:41.165953+0000","last_became_peered":"2024-03-29T04:17:41.165953+0000","last_unstale":"2024-03-29T04:17:43.552920+0000","last_undegraded":"2024-03-29T04:17:43.552920+0000","last_fullsized":"2024-03-29T04:17:43.552920+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:58.692538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634707+0000","last_change":"2024-03-29T04:17:36.610409+0000","last_active":"2024-03-29T04:17:44.634707+0000","last_peered":"2024-03-29T04:17:44.634707+0000","last_clean":"2024-03-29T04:17:44.634707+0000","last_became_active":"2024-03-29T04:17:36.609783+0000","last_became_peered":"2024-03-29T04:17:36.609783+0000","last_unstale":"2024-03-29T04:17:44.634707+0000","last_undegraded":"2024-03-29T04:17:44.634707+0000","last_fullsized":"2024-03-29T04:17:44.634707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:46:38.204638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:47.106160+0000","last_change":"2024-03-29T04:17:35.576067+0000","last_active":"2024-03-29T04:17:47.106160+0000","last_peered":"2024-03-29T04:17:47.106160+0000","last_clean":"2024-03-29T04:17:47.106160+0000","last_became_active":"2024-03-29T04:17:35.573099+0000","last_became_peered":"2024-03-29T04:17:35.573099+0000","last_unstale":"2024-03-29T04:17:47.106160+0000","last_undegraded":"2024-03-29T04:17:47.106160+0000","last_fullsized":"2024-03-29T04:17:47.106160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:53.508449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470529+0000","last_change":"2024-03-29T04:17:40.658773+0000","last_active":"2024-03-29T04:17:44.470529+0000","last_peered":"2024-03-29T04:17:44.470529+0000","last_clean":"2024-03-29T04:17:44.470529+0000","last_became_active":"2024-03-29T04:17:40.658277+0000","last_became_peered":"2024-03-29T04:17:40.658277+0000","last_unstale":"2024-03-29T04:17:44.470529+0000","last_undegraded":"2024-03-29T04:17:44.470529+0000","last_fullsized":"2024-03-29T04:17:44.470529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:43:44.093178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.658046+0000","last_change":"2024-03-29T04:17:36.620386+0000","last_active":"2024-03-29T04:17:42.658046+0000","last_peered":"2024-03-29T04:17:42.658046+0000","last_clean":"2024-03-29T04:17:42.658046+0000","last_became_active":"2024-03-29T04:17:36.620093+0000","last_became_peered":"2024-03-29T04:17:36.620093+0000","last_unstale":"2024-03-29T04:17:42.658046+0000","last_undegraded":"2024-03-29T04:17:42.658046+0000","last_fullsized":"2024-03-29T04:17:42.658046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:12:37.599253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.469929+0000","last_change":"2024-03-29T04:17:35.592723+0000","last_active":"2024-03-29T04:17:44.469929+0000","last_peered":"2024-03-29T04:17:44.469929+0000","last_clean":"2024-03-29T04:17:44.469929+0000","last_became_active":"2024-03-29T04:17:35.592409+0000","last_became_peered":"2024-03-29T04:17:35.592409+0000","last_unstale":"2024-03-29T04:17:44.469929+0000","last_undegraded":"2024-03-29T04:17:44.469929+0000","last_fullsized":"2024-03-29T04:17:44.469929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:12:36.590426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.657265+0000","last_change":"2024-03-29T04:17:40.654909+0000","last_active":"2024-03-29T04:17:42.657265+0000","last_peered":"2024-03-29T04:17:42.657265+0000","last_clean":"2024-03-29T04:17:42.657265+0000","last_became_active":"2024-03-29T04:17:40.654420+0000","last_became_peered":"2024-03-29T04:17:40.654420+0000","last_unstale":"2024-03-29T04:17:42.657265+0000","last_undegraded":"2024-03-29T04:17:42.657265+0000","last_fullsized":"2024-03-29T04:17:42.657265+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:10:04.139549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653614+0000","last_change":"2024-03-29T04:17:36.599274+0000","last_active":"2024-03-29T04:17:42.653614+0000","last_peered":"2024-03-29T04:17:42.653614+0000","last_clean":"2024-03-29T04:17:42.653614+0000","last_became_active":"2024-03-29T04:17:36.597822+0000","last_became_peered":"2024-03-29T04:17:36.597822+0000","last_unstale":"2024-03-29T04:17:42.653614+0000","last_undegraded":"2024-03-29T04:17:42.653614+0000","last_fullsized":"2024-03-29T04:17:42.653614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:24:47.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553100+0000","last_change":"2024-03-29T04:17:35.592120+0000","last_active":"2024-03-29T04:17:43.553100+0000","last_peered":"2024-03-29T04:17:43.553100+0000","last_clean":"2024-03-29T04:17:43.553100+0000","last_became_active":"2024-03-29T04:17:35.590699+0000","last_became_peered":"2024-03-29T04:17:35.590699+0000","last_unstale":"2024-03-29T04:17:43.553100+0000","last_undegraded":"2024-03-29T04:17:43.553100+0000","last_fullsized":"2024-03-29T04:17:43.553100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:53.508449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649946+0000","last_change":"2024-03-29T04:17:40.661266+0000","last_active":"2024-03-29T04:17:42.649946+0000","last_peered":"2024-03-29T04:17:42.649946+0000","last_clean":"2024-03-29T04:17:42.649946+0000","last_became_active":"2024-03-29T04:17:40.660780+0000","last_became_peered":"2024-03-29T04:17:40.660780+0000","last_unstale":"2024-03-29T04:17:42.649946+0000","last_undegraded":"2024-03-29T04:17:42.649946+0000","last_fullsized":"2024-03-29T04:17:42.649946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:02.859957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552978+0000","last_change":"2024-03-29T04:17:36.610408+0000","last_active":"2024-03-29T04:17:43.552978+0000","last_peered":"2024-03-29T04:17:43.552978+0000","last_clean":"2024-03-29T04:17:43.552978+0000","last_became_active":"2024-03-29T04:17:36.610111+0000","last_became_peered":"2024-03-29T04:17:36.610111+0000","last_unstale":"2024-03-29T04:17:43.552978+0000","last_undegraded":"2024-03-29T04:17:43.552978+0000","last_fullsized":"2024-03-29T04:17:43.552978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:46:38.204638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.269316+0000","last_change":"2024-03-29T04:17:35.601002+0000","last_active":"2024-03-29T04:17:44.269316+0000","last_peered":"2024-03-29T04:17:44.269316+0000","last_clean":"2024-03-29T04:17:44.269316+0000","last_became_active":"2024-03-29T04:17:35.600276+0000","last_became_peered":"2024-03-29T04:17:35.600276+0000","last_unstale":"2024-03-29T04:17:44.269316+0000","last_undegraded":"2024-03-29T04:17:44.269316+0000","last_fullsized":"2024-03-29T04:17:44.269316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:55:50.253366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649873+0000","last_change":"2024-03-29T04:17:41.164850+0000","last_active":"2024-03-29T04:17:42.649873+0000","last_peered":"2024-03-29T04:17:42.649873+0000","last_clean":"2024-03-29T04:17:42.649873+0000","last_became_active":"2024-03-29T04:17:41.163894+0000","last_became_peered":"2024-03-29T04:17:41.163894+0000","last_unstale":"2024-03-29T04:17:42.649873+0000","last_undegraded":"2024-03-29T04:17:42.649873+0000","last_fullsized":"2024-03-29T04:17:42.649873+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:03:39.448893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653276+0000","last_change":"2024-03-29T04:17:41.171200+0000","last_active":"2024-03-29T04:17:42.653276+0000","last_peered":"2024-03-29T04:17:42.653276+0000","last_clean":"2024-03-29T04:17:42.653276+0000","last_became_active":"2024-03-29T04:17:41.169774+0000","last_became_peered":"2024-03-29T04:17:41.169774+0000","last_unstale":"2024-03-29T04:17:42.653276+0000","last_undegraded":"2024-03-29T04:17:42.653276+0000","last_fullsized":"2024-03-29T04:17:42.653276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:43:44.093178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470127+0000","last_change":"2024-03-29T04:17:36.598385+0000","last_active":"2024-03-29T04:17:44.470127+0000","last_peered":"2024-03-29T04:17:44.470127+0000","last_clean":"2024-03-29T04:17:44.470127+0000","last_became_active":"2024-03-29T04:17:36.597989+0000","last_became_peered":"2024-03-29T04:17:36.597989+0000","last_unstale":"2024-03-29T04:17:44.470127+0000","last_undegraded":"2024-03-29T04:17:44.470127+0000","last_fullsized":"2024-03-29T04:17:44.470127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:11.037419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553421+0000","last_change":"2024-03-29T04:17:35.591199+0000","last_active":"2024-03-29T04:17:43.553421+0000","last_peered":"2024-03-29T04:17:43.553421+0000","last_clean":"2024-03-29T04:17:43.553421+0000","last_became_active":"2024-03-29T04:17:35.590444+0000","last_became_peered":"2024-03-29T04:17:35.590444+0000","last_unstale":"2024-03-29T04:17:43.553421+0000","last_undegraded":"2024-03-29T04:17:43.553421+0000","last_fullsized":"2024-03-29T04:17:43.553421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:23.283694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270972+0000","last_change":"2024-03-29T04:17:40.661020+0000","last_active":"2024-03-29T04:17:44.270972+0000","last_peered":"2024-03-29T04:17:44.270972+0000","last_clean":"2024-03-29T04:17:44.270972+0000","last_became_active":"2024-03-29T04:17:40.660552+0000","last_became_peered":"2024-03-29T04:17:40.660552+0000","last_unstale":"2024-03-29T04:17:44.270972+0000","last_undegraded":"2024-03-29T04:17:44.270972+0000","last_fullsized":"2024-03-29T04:17:44.270972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:25:02.717791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649896+0000","last_change":"2024-03-29T04:17:36.619765+0000","last_active":"2024-03-29T04:17:42.649896+0000","last_peered":"2024-03-29T04:17:42.649896+0000","last_clean":"2024-03-29T04:17:42.649896+0000","last_became_active":"2024-03-29T04:17:36.619507+0000","last_became_peered":"2024-03-29T04:17:36.619507+0000","last_unstale":"2024-03-29T04:17:42.649896+0000","last_undegraded":"2024-03-29T04:17:42.649896+0000","last_fullsized":"2024-03-29T04:17:42.649896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:39.635039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649952+0000","last_change":"2024-03-29T04:17:35.575983+0000","last_active":"2024-03-29T04:17:42.649952+0000","last_peered":"2024-03-29T04:17:42.649952+0000","last_clean":"2024-03-29T04:17:42.649952+0000","last_became_active":"2024-03-29T04:17:35.575381+0000","last_became_peered":"2024-03-29T04:17:35.575381+0000","last_unstale":"2024-03-29T04:17:42.649952+0000","last_undegraded":"2024-03-29T04:17:42.649952+0000","last_fullsized":"2024-03-29T04:17:42.649952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:07.631047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.268935+0000","last_change":"2024-03-29T04:17:35.577005+0000","last_active":"2024-03-29T04:17:44.268935+0000","last_peered":"2024-03-29T04:17:44.268935+0000","last_clean":"2024-03-29T04:17:44.268935+0000","last_became_active":"2024-03-29T04:17:35.574501+0000","last_became_peered":"2024-03-29T04:17:35.574501+0000","last_unstale":"2024-03-29T04:17:44.268935+0000","last_undegraded":"2024-03-29T04:17:44.268935+0000","last_fullsized":"2024-03-29T04:17:44.268935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:23.283694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653382+0000","last_change":"2024-03-29T04:17:36.615300+0000","last_active":"2024-03-29T04:17:42.653382+0000","last_peered":"2024-03-29T04:17:42.653382+0000","last_clean":"2024-03-29T04:17:42.653382+0000","last_became_active":"2024-03-29T04:17:36.614767+0000","last_became_peered":"2024-03-29T04:17:36.614767+0000","last_unstale":"2024-03-29T04:17:42.653382+0000","last_undegraded":"2024-03-29T04:17:42.653382+0000","last_fullsized":"2024-03-29T04:17:42.653382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:30.598140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578719+0000","last_change":"2024-03-29T04:17:40.666173+0000","last_active":"2024-03-29T04:17:43.578719+0000","last_peered":"2024-03-29T04:17:43.578719+0000","last_clean":"2024-03-29T04:17:43.578719+0000","last_became_active":"2024-03-29T04:17:40.665205+0000","last_became_peered":"2024-03-29T04:17:40.665205+0000","last_unstale":"2024-03-29T04:17:43.578719+0000","last_undegraded":"2024-03-29T04:17:43.578719+0000","last_fullsized":"2024-03-29T04:17:43.578719+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:10:04.880736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650203+0000","last_change":"2024-03-29T04:17:35.597727+0000","last_active":"2024-03-29T04:17:42.650203+0000","last_peered":"2024-03-29T04:17:42.650203+0000","last_clean":"2024-03-29T04:17:42.650203+0000","last_became_active":"2024-03-29T04:17:35.597476+0000","last_became_peered":"2024-03-29T04:17:35.597476+0000","last_unstale":"2024-03-29T04:17:42.650203+0000","last_undegraded":"2024-03-29T04:17:42.650203+0000","last_fullsized":"2024-03-29T04:17:42.650203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:04:44.150824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656845+0000","last_change":"2024-03-29T04:17:36.593245+0000","last_active":"2024-03-29T04:17:42.656845+0000","last_peered":"2024-03-29T04:17:42.656845+0000","last_clean":"2024-03-29T04:17:42.656845+0000","last_became_active":"2024-03-29T04:17:36.592508+0000","last_became_peered":"2024-03-29T04:17:36.592508+0000","last_unstale":"2024-03-29T04:17:42.656845+0000","last_undegraded":"2024-03-29T04:17:42.656845+0000","last_fullsized":"2024-03-29T04:17:42.656845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:52:19.919554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650486+0000","last_change":"2024-03-29T04:17:40.664734+0000","last_active":"2024-03-29T04:17:42.650486+0000","last_peered":"2024-03-29T04:17:42.650486+0000","last_clean":"2024-03-29T04:17:42.650486+0000","last_became_active":"2024-03-29T04:17:40.664326+0000","last_became_peered":"2024-03-29T04:17:40.664326+0000","last_unstale":"2024-03-29T04:17:42.650486+0000","last_undegraded":"2024-03-29T04:17:42.650486+0000","last_fullsized":"2024-03-29T04:17:42.650486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:03.774120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.269324+0000","last_change":"2024-03-29T04:17:36.618652+0000","last_active":"2024-03-29T04:17:44.269324+0000","last_peered":"2024-03-29T04:17:44.269324+0000","last_clean":"2024-03-29T04:17:44.269324+0000","last_became_active":"2024-03-29T04:17:36.618384+0000","last_became_peered":"2024-03-29T04:17:36.618384+0000","last_unstale":"2024-03-29T04:17:44.269324+0000","last_undegraded":"2024-03-29T04:17:44.269324+0000","last_fullsized":"2024-03-29T04:17:44.269324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:30.598140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578474+0000","last_change":"2024-03-29T04:17:36.606149+0000","last_active":"2024-03-29T04:17:43.578474+0000","last_peered":"2024-03-29T04:17:43.578474+0000","last_clean":"2024-03-29T04:17:43.578474+0000","last_became_active":"2024-03-29T04:17:36.605253+0000","last_became_peered":"2024-03-29T04:17:36.605253+0000","last_unstale":"2024-03-29T04:17:43.578474+0000","last_undegraded":"2024-03-29T04:17:43.578474+0000","last_fullsized":"2024-03-29T04:17:43.578474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:24:47.130861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553107+0000","last_change":"2024-03-29T04:17:36.602472+0000","last_active":"2024-03-29T04:17:43.553107+0000","last_peered":"2024-03-29T04:17:43.553107+0000","last_clean":"2024-03-29T04:17:43.553107+0000","last_became_active":"2024-03-29T04:17:36.601990+0000","last_became_peered":"2024-03-29T04:17:36.601990+0000","last_unstale":"2024-03-29T04:17:43.553107+0000","last_undegraded":"2024-03-29T04:17:43.553107+0000","last_fullsized":"2024-03-29T04:17:43.553107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:38:20.641332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653238+0000","last_change":"2024-03-29T04:17:36.607285+0000","last_active":"2024-03-29T04:17:42.653238+0000","last_peered":"2024-03-29T04:17:42.653238+0000","last_clean":"2024-03-29T04:17:42.653238+0000","last_became_active":"2024-03-29T04:17:36.606419+0000","last_became_peered":"2024-03-29T04:17:36.606419+0000","last_unstale":"2024-03-29T04:17:42.653238+0000","last_undegraded":"2024-03-29T04:17:42.653238+0000","last_fullsized":"2024-03-29T04:17:42.653238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:41.872399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650775+0000","last_change":"2024-03-29T04:17:36.627762+0000","last_active":"2024-03-29T04:17:42.650775+0000","last_peered":"2024-03-29T04:17:42.650775+0000","last_clean":"2024-03-29T04:17:42.650775+0000","last_became_active":"2024-03-29T04:17:36.627476+0000","last_became_peered":"2024-03-29T04:17:36.627476+0000","last_unstale":"2024-03-29T04:17:42.650775+0000","last_undegraded":"2024-03-29T04:17:42.650775+0000","last_fullsized":"2024-03-29T04:17:42.650775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:38:20.641332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470447+0000","last_change":"2024-03-29T04:17:36.597795+0000","last_active":"2024-03-29T04:17:44.470447+0000","last_peered":"2024-03-29T04:17:44.470447+0000","last_clean":"2024-03-29T04:17:44.470447+0000","last_became_active":"2024-03-29T04:17:36.597490+0000","last_became_peered":"2024-03-29T04:17:36.597490+0000","last_unstale":"2024-03-29T04:17:44.470447+0000","last_undegraded":"2024-03-29T04:17:44.470447+0000","last_fullsized":"2024-03-29T04:17:44.470447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:43:27.568676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270661+0000","last_change":"2024-03-29T04:17:36.609886+0000","last_active":"2024-03-29T04:17:44.270661+0000","last_peered":"2024-03-29T04:17:44.270661+0000","last_clean":"2024-03-29T04:17:44.270661+0000","last_became_active":"2024-03-29T04:17:36.609354+0000","last_became_peered":"2024-03-29T04:17:36.609354+0000","last_unstale":"2024-03-29T04:17:44.270661+0000","last_undegraded":"2024-03-29T04:17:44.270661+0000","last_fullsized":"2024-03-29T04:17:44.270661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:24:47.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650587+0000","last_change":"2024-03-29T04:17:36.627964+0000","last_active":"2024-03-29T04:17:42.650587+0000","last_peered":"2024-03-29T04:17:42.650587+0000","last_clean":"2024-03-29T04:17:42.650587+0000","last_became_active":"2024-03-29T04:17:36.627606+0000","last_became_peered":"2024-03-29T04:17:36.627606+0000","last_unstale":"2024-03-29T04:17:42.650587+0000","last_undegraded":"2024-03-29T04:17:42.650587+0000","last_fullsized":"2024-03-29T04:17:42.650587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:44.877184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552870+0000","last_change":"2024-03-29T04:17:36.617738+0000","last_active":"2024-03-29T04:17:43.552870+0000","last_peered":"2024-03-29T04:17:43.552870+0000","last_clean":"2024-03-29T04:17:43.552870+0000","last_became_active":"2024-03-29T04:17:36.617532+0000","last_became_peered":"2024-03-29T04:17:36.617532+0000","last_unstale":"2024-03-29T04:17:43.552870+0000","last_undegraded":"2024-03-29T04:17:43.552870+0000","last_fullsized":"2024-03-29T04:17:43.552870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:41.872399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649836+0000","last_change":"2024-03-29T04:17:36.627025+0000","last_active":"2024-03-29T04:17:42.649836+0000","last_peered":"2024-03-29T04:17:42.649836+0000","last_clean":"2024-03-29T04:17:42.649836+0000","last_became_active":"2024-03-29T04:17:36.626671+0000","last_became_peered":"2024-03-29T04:17:36.626671+0000","last_unstale":"2024-03-29T04:17:42.649836+0000","last_undegraded":"2024-03-29T04:17:42.649836+0000","last_fullsized":"2024-03-29T04:17:42.649836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:31:36.783083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552909+0000","last_change":"2024-03-29T04:17:36.609953+0000","last_active":"2024-03-29T04:17:43.552909+0000","last_peered":"2024-03-29T04:17:43.552909+0000","last_clean":"2024-03-29T04:17:43.552909+0000","last_became_active":"2024-03-29T04:17:36.609455+0000","last_became_peered":"2024-03-29T04:17:36.609455+0000","last_unstale":"2024-03-29T04:17:43.552909+0000","last_undegraded":"2024-03-29T04:17:43.552909+0000","last_fullsized":"2024-03-29T04:17:43.552909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:31:36.783083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653087+0000","last_change":"2024-03-29T04:17:36.618490+0000","last_active":"2024-03-29T04:17:42.653087+0000","last_peered":"2024-03-29T04:17:42.653087+0000","last_clean":"2024-03-29T04:17:42.653087+0000","last_became_active":"2024-03-29T04:17:36.618012+0000","last_became_peered":"2024-03-29T04:17:36.618012+0000","last_unstale":"2024-03-29T04:17:42.653087+0000","last_undegraded":"2024-03-29T04:17:42.653087+0000","last_fullsized":"2024-03-29T04:17:42.653087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:43:27.568676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270104+0000","last_change":"2024-03-29T04:17:36.621359+0000","last_active":"2024-03-29T04:17:44.270104+0000","last_peered":"2024-03-29T04:17:44.270104+0000","last_clean":"2024-03-29T04:17:44.270104+0000","last_became_active":"2024-03-29T04:17:36.620689+0000","last_became_peered":"2024-03-29T04:17:36.620689+0000","last_unstale":"2024-03-29T04:17:44.270104+0000","last_undegraded":"2024-03-29T04:17:44.270104+0000","last_fullsized":"2024-03-29T04:17:44.270104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:43:27.568676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634352+0000","last_change":"2024-03-29T04:17:36.610675+0000","last_active":"2024-03-29T04:17:44.634352+0000","last_peered":"2024-03-29T04:17:44.634352+0000","last_clean":"2024-03-29T04:17:44.634352+0000","last_became_active":"2024-03-29T04:17:36.609797+0000","last_became_peered":"2024-03-29T04:17:36.609797+0000","last_unstale":"2024-03-29T04:17:44.634352+0000","last_undegraded":"2024-03-29T04:17:44.634352+0000","last_fullsized":"2024-03-29T04:17:44.634352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:04:11.306658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.579363+0000","last_change":"2024-03-29T04:17:36.605354+0000","last_active":"2024-03-29T04:17:43.579363+0000","last_peered":"2024-03-29T04:17:43.579363+0000","last_clean":"2024-03-29T04:17:43.579363+0000","last_became_active":"2024-03-29T04:17:36.605028+0000","last_became_peered":"2024-03-29T04:17:36.605028+0000","last_unstale":"2024-03-29T04:17:43.579363+0000","last_undegraded":"2024-03-29T04:17:43.579363+0000","last_fullsized":"2024-03-29T04:17:43.579363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:12:37.599253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553747+0000","last_change":"2024-03-29T04:17:36.611310+0000","last_active":"2024-03-29T04:17:43.553747+0000","last_peered":"2024-03-29T04:17:43.553747+0000","last_clean":"2024-03-29T04:17:43.553747+0000","last_became_active":"2024-03-29T04:17:36.611020+0000","last_became_peered":"2024-03-29T04:17:36.611020+0000","last_unstale":"2024-03-29T04:17:43.553747+0000","last_undegraded":"2024-03-29T04:17:43.553747+0000","last_fullsized":"2024-03-29T04:17:43.553747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:42.915247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552533+0000","last_change":"2024-03-29T04:17:36.618624+0000","last_active":"2024-03-29T04:17:43.552533+0000","last_peered":"2024-03-29T04:17:43.552533+0000","last_clean":"2024-03-29T04:17:43.552533+0000","last_became_active":"2024-03-29T04:17:36.618282+0000","last_became_peered":"2024-03-29T04:17:36.618282+0000","last_unstale":"2024-03-29T04:17:43.552533+0000","last_undegraded":"2024-03-29T04:17:43.552533+0000","last_fullsized":"2024-03-29T04:17:43.552533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:24:47.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470271+0000","last_change":"2024-03-29T04:17:36.587187+0000","last_active":"2024-03-29T04:17:44.470271+0000","last_peered":"2024-03-29T04:17:44.470271+0000","last_clean":"2024-03-29T04:17:44.470271+0000","last_became_active":"2024-03-29T04:17:36.586740+0000","last_became_peered":"2024-03-29T04:17:36.586740+0000","last_unstale":"2024-03-29T04:17:44.470271+0000","last_undegraded":"2024-03-29T04:17:44.470271+0000","last_fullsized":"2024-03-29T04:17:44.470271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:24:47.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653115+0000","last_change":"2024-03-29T04:17:36.615925+0000","last_active":"2024-03-29T04:17:42.653115+0000","last_peered":"2024-03-29T04:17:42.653115+0000","last_clean":"2024-03-29T04:17:42.653115+0000","last_became_active":"2024-03-29T04:17:36.615143+0000","last_became_peered":"2024-03-29T04:17:36.615143+0000","last_unstale":"2024-03-29T04:17:42.653115+0000","last_undegraded":"2024-03-29T04:17:42.653115+0000","last_fullsized":"2024-03-29T04:17:42.653115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:50:12.920911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650032+0000","last_change":"2024-03-29T04:17:36.605090+0000","last_active":"2024-03-29T04:17:42.650032+0000","last_peered":"2024-03-29T04:17:42.650032+0000","last_clean":"2024-03-29T04:17:42.650032+0000","last_became_active":"2024-03-29T04:17:36.604669+0000","last_became_peered":"2024-03-29T04:17:36.604669+0000","last_unstale":"2024-03-29T04:17:42.650032+0000","last_undegraded":"2024-03-29T04:17:42.650032+0000","last_fullsized":"2024-03-29T04:17:42.650032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:30.598140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.657981+0000","last_change":"2024-03-29T04:17:36.614472+0000","last_active":"2024-03-29T04:17:42.657981+0000","last_peered":"2024-03-29T04:17:42.657981+0000","last_clean":"2024-03-29T04:17:42.657981+0000","last_became_active":"2024-03-29T04:17:36.613881+0000","last_became_peered":"2024-03-29T04:17:36.613881+0000","last_unstale":"2024-03-29T04:17:42.657981+0000","last_undegraded":"2024-03-29T04:17:42.657981+0000","last_fullsized":"2024-03-29T04:17:42.657981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:24:47.130861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270375+0000","last_change":"2024-03-29T04:17:36.600125+0000","last_active":"2024-03-29T04:17:44.270375+0000","last_peered":"2024-03-29T04:17:44.270375+0000","last_clean":"2024-03-29T04:17:44.270375+0000","last_became_active":"2024-03-29T04:17:36.599450+0000","last_became_peered":"2024-03-29T04:17:36.599450+0000","last_unstale":"2024-03-29T04:17:44.270375+0000","last_undegraded":"2024-03-29T04:17:44.270375+0000","last_fullsized":"2024-03-29T04:17:44.270375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:11.037419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270626+0000","last_change":"2024-03-29T04:17:36.600177+0000","last_active":"2024-03-29T04:17:44.270626+0000","last_peered":"2024-03-29T04:17:44.270626+0000","last_clean":"2024-03-29T04:17:44.270626+0000","last_became_active":"2024-03-29T04:17:36.599696+0000","last_became_peered":"2024-03-29T04:17:36.599696+0000","last_unstale":"2024-03-29T04:17:44.270626+0000","last_undegraded":"2024-03-29T04:17:44.270626+0000","last_fullsized":"2024-03-29T04:17:44.270626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:39.635039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.635042+0000","last_change":"2024-03-29T04:17:36.610038+0000","last_active":"2024-03-29T04:17:44.635042+0000","last_peered":"2024-03-29T04:17:44.635042+0000","last_clean":"2024-03-29T04:17:44.635042+0000","last_became_active":"2024-03-29T04:17:36.609175+0000","last_became_peered":"2024-03-29T04:17:36.609175+0000","last_unstale":"2024-03-29T04:17:44.635042+0000","last_undegraded":"2024-03-29T04:17:44.635042+0000","last_fullsized":"2024-03-29T04:17:44.635042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:42:21.035572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649728+0000","last_change":"2024-03-29T04:17:36.616578+0000","last_active":"2024-03-29T04:17:42.649728+0000","last_peered":"2024-03-29T04:17:42.649728+0000","last_clean":"2024-03-29T04:17:42.649728+0000","last_became_active":"2024-03-29T04:17:36.612656+0000","last_became_peered":"2024-03-29T04:17:36.612656+0000","last_unstale":"2024-03-29T04:17:42.649728+0000","last_undegraded":"2024-03-29T04:17:42.649728+0000","last_fullsized":"2024-03-29T04:17:42.649728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:11.037419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634140+0000","last_change":"2024-03-29T04:17:36.618859+0000","last_active":"2024-03-29T04:17:44.634140+0000","last_peered":"2024-03-29T04:17:44.634140+0000","last_clean":"2024-03-29T04:17:44.634140+0000","last_became_active":"2024-03-29T04:17:36.618475+0000","last_became_peered":"2024-03-29T04:17:36.618475+0000","last_unstale":"2024-03-29T04:17:44.634140+0000","last_undegraded":"2024-03-29T04:17:44.634140+0000","last_fullsized":"2024-03-29T04:17:44.634140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:30.729030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552858+0000","last_change":"2024-03-29T04:17:36.614982+0000","last_active":"2024-03-29T04:17:43.552858+0000","last_peered":"2024-03-29T04:17:43.552858+0000","last_clean":"2024-03-29T04:17:43.552858+0000","last_became_active":"2024-03-29T04:17:36.614701+0000","last_became_peered":"2024-03-29T04:17:36.614701+0000","last_unstale":"2024-03-29T04:17:43.552858+0000","last_undegraded":"2024-03-29T04:17:43.552858+0000","last_fullsized":"2024-03-29T04:17:43.552858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:30.598140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634536+0000","last_change":"2024-03-29T04:17:36.608699+0000","last_active":"2024-03-29T04:17:44.634536+0000","last_peered":"2024-03-29T04:17:44.634536+0000","last_clean":"2024-03-29T04:17:44.634536+0000","last_became_active":"2024-03-29T04:17:36.608428+0000","last_became_peered":"2024-03-29T04:17:36.608428+0000","last_unstale":"2024-03-29T04:17:44.634536+0000","last_undegraded":"2024-03-29T04:17:44.634536+0000","last_fullsized":"2024-03-29T04:17:44.634536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:44.877184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553129+0000","last_change":"2024-03-29T04:17:36.596870+0000","last_active":"2024-03-29T04:17:43.553129+0000","last_peered":"2024-03-29T04:17:43.553129+0000","last_clean":"2024-03-29T04:17:43.553129+0000","last_became_active":"2024-03-29T04:17:36.596557+0000","last_became_peered":"2024-03-29T04:17:36.596557+0000","last_unstale":"2024-03-29T04:17:43.553129+0000","last_undegraded":"2024-03-29T04:17:43.553129+0000","last_fullsized":"2024-03-29T04:17:43.553129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:44.877184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653260+0000","last_change":"2024-03-29T04:17:36.593793+0000","last_active":"2024-03-29T04:17:42.653260+0000","last_peered":"2024-03-29T04:17:42.653260+0000","last_clean":"2024-03-29T04:17:42.653260+0000","last_became_active":"2024-03-29T04:17:36.592632+0000","last_became_peered":"2024-03-29T04:17:36.592632+0000","last_unstale":"2024-03-29T04:17:42.653260+0000","last_undegraded":"2024-03-29T04:17:42.653260+0000","last_fullsized":"2024-03-29T04:17:42.653260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:41:31.910133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470745+0000","last_change":"2024-03-29T04:17:36.593148+0000","last_active":"2024-03-29T04:17:44.470745+0000","last_peered":"2024-03-29T04:17:44.470745+0000","last_clean":"2024-03-29T04:17:44.470745+0000","last_became_active":"2024-03-29T04:17:36.592754+0000","last_became_peered":"2024-03-29T04:17:36.592754+0000","last_unstale":"2024-03-29T04:17:44.470745+0000","last_undegraded":"2024-03-29T04:17:44.470745+0000","last_fullsized":"2024-03-29T04:17:44.470745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:30.598140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.635518+0000","last_change":"2024-03-29T04:17:36.615787+0000","last_active":"2024-03-29T04:17:44.635518+0000","last_peered":"2024-03-29T04:17:44.635518+0000","last_clean":"2024-03-29T04:17:44.635518+0000","last_became_active":"2024-03-29T04:17:36.615476+0000","last_became_peered":"2024-03-29T04:17:36.615476+0000","last_unstale":"2024-03-29T04:17:44.635518+0000","last_undegraded":"2024-03-29T04:17:44.635518+0000","last_fullsized":"2024-03-29T04:17:44.635518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:42.915247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":876544,"data_stored":703014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":112154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":112190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":278528,"data_stored":112064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":286720,"data_stored":112224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":884736,"data_stored":703070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:17:59.165 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:17:59.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.344+0000 7fd452c7c640 1 Processor -- start 2024-03-29T04:17:59.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.344+0000 7fd452c7c640 1 -- start start 2024-03-29T04:17:59.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.344+0000 7fd452c7c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd44c142440 0x7fd44c0cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:59.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.344+0000 7fd452c7c640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd44c0cf5e0 0x7fd44c0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:59.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.344+0000 7fd452c7c640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c0cd680 0x7fd44c15fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:59.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.344+0000 7fd452c7c640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fd44c0cfb60 con 0x7fd44c142440 2024-03-29T04:17:59.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.344+0000 7fd452c7c640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fd44c0cfe40 con 0x7fd44c0cf5e0 2024-03-29T04:17:59.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.344+0000 7fd452c7c640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd44c0d0120 con 0x7fd44c0cd680 2024-03-29T04:17:59.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.344+0000 7fd450a18640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd44c142440 0x7fd44c0cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:59.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.344+0000 7fd450a18640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd44c142440 0x7fd44c0cf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51694/0 (socket says 172.21.15.22:51694) 2024-03-29T04:17:59.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.344+0000 7fd450a18640 1 -- 172.21.15.22:0/27570311 learned_addr learned my addr 172.21.15.22:0/27570311 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:59.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.344+0000 7fd451219640 1 --2- 172.21.15.22:0/27570311 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c0cd680 0x7fd44c15fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:59.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.344+0000 7fd44bfff640 1 --2- 172.21.15.22:0/27570311 >> v2:172.21.15.151:3300/0 conn(0x7fd44c0cf5e0 0x7fd44c0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:59.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.344+0000 7fd450a18640 1 -- 172.21.15.22:0/27570311 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c0cd680 msgr2=0x7fd44c15fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:59.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.344+0000 7fd450a18640 1 --2- 172.21.15.22:0/27570311 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c0cd680 0x7fd44c15fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.344+0000 7fd450a18640 1 -- 172.21.15.22:0/27570311 >> v2:172.21.15.151:3300/0 conn(0x7fd44c0cf5e0 msgr2=0x7fd44c0cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:59.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.344+0000 7fd450a18640 1 --2- 172.21.15.22:0/27570311 >> v2:172.21.15.151:3300/0 conn(0x7fd44c0cf5e0 0x7fd44c0cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.344+0000 7fd450a18640 1 -- 172.21.15.22:0/27570311 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44c0d0400 con 0x7fd44c142440 2024-03-29T04:17:59.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.344+0000 7fd44bfff640 1 --2- 172.21.15.22:0/27570311 >> v2:172.21.15.151:3300/0 conn(0x7fd44c0cf5e0 0x7fd44c0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:17:59.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd450a18640 1 --2- 172.21.15.22:0/27570311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c142440 0x7fd44c0cf0a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0b3080 tx=0x7fd43c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f58b26c9329b42 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:59.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd44b7fe640 1 -- 172.21.15.22:0/27570311 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0b0070 con 0x7fd44c142440 2024-03-29T04:17:59.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd44b7fe640 1 -- 172.21.15.22:0/27570311 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd43c0ab600 con 0x7fd44c142440 2024-03-29T04:17:59.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd44b7fe640 1 -- 172.21.15.22:0/27570311 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0ab9d0 con 0x7fd44c142440 2024-03-29T04:17:59.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd452c7c640 1 -- 172.21.15.22:0/27570311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c142440 msgr2=0x7fd44c0cf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:59.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd452c7c640 1 --2- 172.21.15.22:0/27570311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c142440 0x7fd44c0cf0a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0b3080 tx=0x7fd43c01c260 comp rx=0 tx=0).stop 2024-03-29T04:17:59.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd452c7c640 1 -- 172.21.15.22:0/27570311 shutdown_connections 2024-03-29T04:17:59.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd452c7c640 1 --2- 172.21.15.22:0/27570311 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c0cd680 0x7fd44c15fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd452c7c640 1 --2- 172.21.15.22:0/27570311 >> v2:172.21.15.151:3300/0 conn(0x7fd44c0cf5e0 0x7fd44c0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd452c7c640 1 --2- 172.21.15.22:0/27570311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c142440 0x7fd44c0cf0a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd452c7c640 1 -- 172.21.15.22:0/27570311 >> 172.21.15.22:0/27570311 conn(0x7fd44c0b2b30 msgr2=0x7fd44c0d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:59.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd452c7c640 1 -- 172.21.15.22:0/27570311 shutdown_connections 2024-03-29T04:17:59.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd452c7c640 1 -- 172.21.15.22:0/27570311 wait complete. 2024-03-29T04:17:59.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd452c7c640 1 Processor -- start 2024-03-29T04:17:59.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd452c7c640 1 -- start start 2024-03-29T04:17:59.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd452c7c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c0cd680 0x7fd44c152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:59.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd452c7c640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c0cf5e0 0x7fd44c15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:59.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd452c7c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd44c142440 0x7fd44c155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:59.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd452c7c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c06bda0 con 0x7fd44c0cd680 2024-03-29T04:17:59.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd452c7c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c06bc20 con 0x7fd44c142440 2024-03-29T04:17:59.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd452c7c640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd44c06bf20 con 0x7fd44c0cf5e0 2024-03-29T04:17:59.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd450a18640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c0cd680 0x7fd44c152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:59.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd450a18640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c0cd680 0x7fd44c152690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51704/0 (socket says 172.21.15.22:51704) 2024-03-29T04:17:59.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd450a18640 1 -- 172.21.15.22:0/3044361572 learned_addr learned my addr 172.21.15.22:0/3044361572 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:59.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd44bfff640 1 --2- 172.21.15.22:0/3044361572 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c0cf5e0 0x7fd44c15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:59.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd451219640 1 --2- 172.21.15.22:0/3044361572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd44c142440 0x7fd44c155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:59.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd450a18640 1 -- 172.21.15.22:0/3044361572 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c0cf5e0 msgr2=0x7fd44c15d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:59.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd450a18640 1 --2- 172.21.15.22:0/3044361572 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c0cf5e0 0x7fd44c15d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd450a18640 1 -- 172.21.15.22:0/3044361572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd44c142440 msgr2=0x7fd44c155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:59.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd450a18640 1 --2- 172.21.15.22:0/3044361572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd44c142440 0x7fd44c155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd450a18640 1 -- 172.21.15.22:0/3044361572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44c15dd30 con 0x7fd44c0cd680 2024-03-29T04:17:59.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd450a18640 1 --2- 172.21.15.22:0/3044361572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c0cd680 0x7fd44c152690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0ae040 tx=0x7fd43c01aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:59.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd4497fa640 1 -- 172.21.15.22:0/3044361572 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0ba020 con 0x7fd44c0cd680 2024-03-29T04:17:59.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd4497fa640 1 -- 172.21.15.22:0/3044361572 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd43c0b6640 con 0x7fd44c0cd680 2024-03-29T04:17:59.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd452c7c640 1 -- 172.21.15.22:0/3044361572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd44c15dfd0 con 0x7fd44c0cd680 2024-03-29T04:17:59.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd4497fa640 1 -- 172.21.15.22:0/3044361572 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0b0040 con 0x7fd44c0cd680 2024-03-29T04:17:59.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.348+0000 7fd452c7c640 1 -- 172.21.15.22:0/3044361572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd44c15e590 con 0x7fd44c0cd680 2024-03-29T04:17:59.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.352+0000 7fd452c7c640 1 -- 172.21.15.22:0/3044361572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd44c068890 con 0x7fd44c0cd680 2024-03-29T04:17:59.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.352+0000 7fd4497fa640 1 -- 172.21.15.22:0/3044361572 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fd43c0bd020 con 0x7fd44c0cd680 2024-03-29T04:17:59.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.352+0000 7fd4497fa640 1 --2- 172.21.15.22:0/3044361572 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd438067fc0 0x7fd43806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:59.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.352+0000 7fd4497fa640 1 -- 172.21.15.22:0/3044361572 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd43c120930 con 0x7fd44c0cd680 2024-03-29T04:17:59.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.352+0000 7fd44bfff640 1 --2- 172.21.15.22:0/3044361572 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd438067fc0 0x7fd43806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:59.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.352+0000 7fd44bfff640 1 --2- 172.21.15.22:0/3044361572 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd438067fc0 0x7fd43806a480 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0ccd90 tx=0x7fd440008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:59.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.356+0000 7fd4497fa640 1 -- 172.21.15.22:0/3044361572 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fd43c0c4070 con 0x7fd44c0cd680 2024-03-29T04:17:59.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.508+0000 7fd452c7c640 1 -- 172.21.15.22:0/3044361572 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd44c15ebe0 con 0x7fd438067fc0 2024-03-29T04:17:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.536+0000 7fd4497fa640 1 -- 172.21.15.22:0/3044361572 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fd44c15ebe0 con 0x7fd438067fc0 2024-03-29T04:17:59.520 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:59.521 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-29T04:17:59.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.540+0000 7fd452c7c640 1 -- 172.21.15.22:0/3044361572 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd438067fc0 msgr2=0x7fd43806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:59.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.540+0000 7fd452c7c640 1 --2- 172.21.15.22:0/3044361572 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd438067fc0 0x7fd43806a480 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0ccd90 tx=0x7fd440008040 comp rx=0 tx=0).stop 2024-03-29T04:17:59.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.540+0000 7fd452c7c640 1 -- 172.21.15.22:0/3044361572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c0cd680 msgr2=0x7fd44c152690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:59.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.540+0000 7fd452c7c640 1 --2- 172.21.15.22:0/3044361572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c0cd680 0x7fd44c152690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0ae040 tx=0x7fd43c01aaa0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.540+0000 7fd452c7c640 1 -- 172.21.15.22:0/3044361572 shutdown_connections 2024-03-29T04:17:59.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.540+0000 7fd452c7c640 1 --2- 172.21.15.22:0/3044361572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd44c142440 0x7fd44c155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.540+0000 7fd452c7c640 1 --2- 172.21.15.22:0/3044361572 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c0cf5e0 0x7fd44c15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.540+0000 7fd452c7c640 1 --2- 172.21.15.22:0/3044361572 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd438067fc0 0x7fd43806a480 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.540+0000 7fd452c7c640 1 --2- 172.21.15.22:0/3044361572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c0cd680 0x7fd44c152690 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.540+0000 7fd452c7c640 1 -- 172.21.15.22:0/3044361572 >> 172.21.15.22:0/3044361572 conn(0x7fd44c0b2b30 msgr2=0x7fd44c1568c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:59.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.540+0000 7fd452c7c640 1 -- 172.21.15.22:0/3044361572 shutdown_connections 2024-03-29T04:17:59.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.540+0000 7fd452c7c640 1 -- 172.21.15.22:0/3044361572 wait complete. 2024-03-29T04:17:59.537 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-03-29T04:17:57.699001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236208,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738512,"statfs":{"total":773094113280,"available":772852236288,"internally_reserved":0,"allocated":3481600,"data_stored":2086528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004600"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653655+0000","last_change":"2024-03-29T04:17:35.587951+0000","last_active":"2024-03-29T04:17:42.653655+0000","last_peered":"2024-03-29T04:17:42.653655+0000","last_clean":"2024-03-29T04:17:42.653655+0000","last_became_active":"2024-03-29T04:17:35.587644+0000","last_became_peered":"2024-03-29T04:17:35.587644+0000","last_unstale":"2024-03-29T04:17:42.653655+0000","last_undegraded":"2024-03-29T04:17:42.653655+0000","last_fullsized":"2024-03-29T04:17:42.653655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:07.631047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650468+0000","last_change":"2024-03-29T04:17:36.604695+0000","last_active":"2024-03-29T04:17:42.650468+0000","last_peered":"2024-03-29T04:17:42.650468+0000","last_clean":"2024-03-29T04:17:42.650468+0000","last_became_active":"2024-03-29T04:17:36.602172+0000","last_became_peered":"2024-03-29T04:17:36.602172+0000","last_unstale":"2024-03-29T04:17:42.650468+0000","last_undegraded":"2024-03-29T04:17:42.650468+0000","last_fullsized":"2024-03-29T04:17:42.650468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:30.729030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649719+0000","last_change":"2024-03-29T04:17:41.168150+0000","last_active":"2024-03-29T04:17:42.649719+0000","last_peered":"2024-03-29T04:17:42.649719+0000","last_clean":"2024-03-29T04:17:42.649719+0000","last_became_active":"2024-03-29T04:17:41.166854+0000","last_became_peered":"2024-03-29T04:17:41.166854+0000","last_unstale":"2024-03-29T04:17:42.649719+0000","last_undegraded":"2024-03-29T04:17:42.649719+0000","last_fullsized":"2024-03-29T04:17:42.649719+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:01:07.401841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650673+0000","last_change":"2024-03-29T04:17:35.575675+0000","last_active":"2024-03-29T04:17:42.650673+0000","last_peered":"2024-03-29T04:17:42.650673+0000","last_clean":"2024-03-29T04:17:42.650673+0000","last_became_active":"2024-03-29T04:17:35.575189+0000","last_became_peered":"2024-03-29T04:17:35.575189+0000","last_unstale":"2024-03-29T04:17:42.650673+0000","last_undegraded":"2024-03-29T04:17:42.650673+0000","last_fullsized":"2024-03-29T04:17:42.650673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:55:50.253366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653534+0000","last_change":"2024-03-29T04:17:36.598621+0000","last_active":"2024-03-29T04:17:42.653534+0000","last_peered":"2024-03-29T04:17:42.653534+0000","last_clean":"2024-03-29T04:17:42.653534+0000","last_became_active":"2024-03-29T04:17:36.596823+0000","last_became_peered":"2024-03-29T04:17:36.596823+0000","last_unstale":"2024-03-29T04:17:42.653534+0000","last_undegraded":"2024-03-29T04:17:42.653534+0000","last_fullsized":"2024-03-29T04:17:42.653534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:11.037419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656537+0000","last_change":"2024-03-29T04:17:40.663898+0000","last_active":"2024-03-29T04:17:42.656537+0000","last_peered":"2024-03-29T04:17:42.656537+0000","last_clean":"2024-03-29T04:17:42.656537+0000","last_became_active":"2024-03-29T04:17:40.663387+0000","last_became_peered":"2024-03-29T04:17:40.663387+0000","last_unstale":"2024-03-29T04:17:42.656537+0000","last_undegraded":"2024-03-29T04:17:42.656537+0000","last_fullsized":"2024-03-29T04:17:42.656537+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:46.993432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.635633+0000","last_change":"2024-03-29T04:17:35.604714+0000","last_active":"2024-03-29T04:17:44.635633+0000","last_peered":"2024-03-29T04:17:44.635633+0000","last_clean":"2024-03-29T04:17:44.635633+0000","last_became_active":"2024-03-29T04:17:35.604080+0000","last_became_peered":"2024-03-29T04:17:35.604080+0000","last_unstale":"2024-03-29T04:17:44.635633+0000","last_undegraded":"2024-03-29T04:17:44.635633+0000","last_fullsized":"2024-03-29T04:17:44.635633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:12:36.590426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553394+0000","last_change":"2024-03-29T04:17:36.615673+0000","last_active":"2024-03-29T04:17:43.553394+0000","last_peered":"2024-03-29T04:17:43.553394+0000","last_clean":"2024-03-29T04:17:43.553394+0000","last_became_active":"2024-03-29T04:17:36.615348+0000","last_became_peered":"2024-03-29T04:17:36.615348+0000","last_unstale":"2024-03-29T04:17:43.553394+0000","last_undegraded":"2024-03-29T04:17:43.553394+0000","last_fullsized":"2024-03-29T04:17:43.553394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:30.729030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656960+0000","last_change":"2024-03-29T04:17:40.653544+0000","last_active":"2024-03-29T04:17:42.656960+0000","last_peered":"2024-03-29T04:17:42.656960+0000","last_clean":"2024-03-29T04:17:42.656960+0000","last_became_active":"2024-03-29T04:17:40.652980+0000","last_became_peered":"2024-03-29T04:17:40.652980+0000","last_unstale":"2024-03-29T04:17:42.656960+0000","last_undegraded":"2024-03-29T04:17:42.656960+0000","last_fullsized":"2024-03-29T04:17:42.656960+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:25:02.717791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634331+0000","last_change":"2024-03-29T04:17:35.604849+0000","last_active":"2024-03-29T04:17:44.634331+0000","last_peered":"2024-03-29T04:17:44.634331+0000","last_clean":"2024-03-29T04:17:44.634331+0000","last_became_active":"2024-03-29T04:17:35.604004+0000","last_became_peered":"2024-03-29T04:17:35.604004+0000","last_unstale":"2024-03-29T04:17:44.634331+0000","last_undegraded":"2024-03-29T04:17:44.634331+0000","last_fullsized":"2024-03-29T04:17:44.634331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:50:11.912084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552984+0000","last_change":"2024-03-29T04:17:36.592055+0000","last_active":"2024-03-29T04:17:43.552984+0000","last_peered":"2024-03-29T04:17:43.552984+0000","last_clean":"2024-03-29T04:17:43.552984+0000","last_became_active":"2024-03-29T04:17:36.588805+0000","last_became_peered":"2024-03-29T04:17:36.588805+0000","last_unstale":"2024-03-29T04:17:43.552984+0000","last_undegraded":"2024-03-29T04:17:43.552984+0000","last_fullsized":"2024-03-29T04:17:43.552984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:13.677092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656938+0000","last_change":"2024-03-29T04:17:40.651131+0000","last_active":"2024-03-29T04:17:42.656938+0000","last_peered":"2024-03-29T04:17:42.656938+0000","last_clean":"2024-03-29T04:17:42.656938+0000","last_became_active":"2024-03-29T04:17:40.650319+0000","last_became_peered":"2024-03-29T04:17:40.650319+0000","last_unstale":"2024-03-29T04:17:42.656938+0000","last_undegraded":"2024-03-29T04:17:42.656938+0000","last_fullsized":"2024-03-29T04:17:42.656938+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:42:25.113757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.269694+0000","last_change":"2024-03-29T04:17:36.620761+0000","last_active":"2024-03-29T04:17:44.269694+0000","last_peered":"2024-03-29T04:17:44.269694+0000","last_clean":"2024-03-29T04:17:44.269694+0000","last_became_active":"2024-03-29T04:17:36.620483+0000","last_became_peered":"2024-03-29T04:17:36.620483+0000","last_unstale":"2024-03-29T04:17:44.269694+0000","last_undegraded":"2024-03-29T04:17:44.269694+0000","last_fullsized":"2024-03-29T04:17:44.269694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:10:00.802551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634192+0000","last_change":"2024-03-29T04:17:35.602583+0000","last_active":"2024-03-29T04:17:44.634192+0000","last_peered":"2024-03-29T04:17:44.634192+0000","last_clean":"2024-03-29T04:17:44.634192+0000","last_became_active":"2024-03-29T04:17:35.601902+0000","last_became_peered":"2024-03-29T04:17:35.601902+0000","last_unstale":"2024-03-29T04:17:44.634192+0000","last_undegraded":"2024-03-29T04:17:44.634192+0000","last_fullsized":"2024-03-29T04:17:44.634192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:43:57.713086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.658621+0000","last_change":"2024-03-29T04:17:40.661925+0000","last_active":"2024-03-29T04:17:42.658621+0000","last_peered":"2024-03-29T04:17:42.658621+0000","last_clean":"2024-03-29T04:17:42.658621+0000","last_became_active":"2024-03-29T04:17:40.661028+0000","last_became_peered":"2024-03-29T04:17:40.661028+0000","last_unstale":"2024-03-29T04:17:42.658621+0000","last_undegraded":"2024-03-29T04:17:42.658621+0000","last_fullsized":"2024-03-29T04:17:42.658621+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:33:27.957344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650137+0000","last_change":"2024-03-29T04:17:36.620569+0000","last_active":"2024-03-29T04:17:42.650137+0000","last_peered":"2024-03-29T04:17:42.650137+0000","last_clean":"2024-03-29T04:17:42.650137+0000","last_became_active":"2024-03-29T04:17:36.620079+0000","last_became_peered":"2024-03-29T04:17:36.620079+0000","last_unstale":"2024-03-29T04:17:42.650137+0000","last_undegraded":"2024-03-29T04:17:42.650137+0000","last_fullsized":"2024-03-29T04:17:42.650137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:41.872399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.469887+0000","last_change":"2024-03-29T04:17:35.577167+0000","last_active":"2024-03-29T04:17:44.469887+0000","last_peered":"2024-03-29T04:17:44.469887+0000","last_clean":"2024-03-29T04:17:44.469887+0000","last_became_active":"2024-03-29T04:17:35.576145+0000","last_became_peered":"2024-03-29T04:17:35.576145+0000","last_unstale":"2024-03-29T04:17:44.469887+0000","last_undegraded":"2024-03-29T04:17:44.469887+0000","last_fullsized":"2024-03-29T04:17:44.469887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:11:43.663978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.657612+0000","last_change":"2024-03-29T04:17:40.664962+0000","last_active":"2024-03-29T04:17:42.657612+0000","last_peered":"2024-03-29T04:17:42.657612+0000","last_clean":"2024-03-29T04:17:42.657612+0000","last_became_active":"2024-03-29T04:17:40.664470+0000","last_became_peered":"2024-03-29T04:17:40.664470+0000","last_unstale":"2024-03-29T04:17:42.657612+0000","last_undegraded":"2024-03-29T04:17:42.657612+0000","last_fullsized":"2024-03-29T04:17:42.657612+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:43:44.093178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650038+0000","last_change":"2024-03-29T04:17:36.617517+0000","last_active":"2024-03-29T04:17:42.650038+0000","last_peered":"2024-03-29T04:17:42.650038+0000","last_clean":"2024-03-29T04:17:42.650038+0000","last_became_active":"2024-03-29T04:17:36.616399+0000","last_became_peered":"2024-03-29T04:17:36.616399+0000","last_unstale":"2024-03-29T04:17:42.650038+0000","last_undegraded":"2024-03-29T04:17:42.650038+0000","last_fullsized":"2024-03-29T04:17:42.650038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:24:47.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470278+0000","last_change":"2024-03-29T04:17:35.576729+0000","last_active":"2024-03-29T04:17:44.470278+0000","last_peered":"2024-03-29T04:17:44.470278+0000","last_clean":"2024-03-29T04:17:44.470278+0000","last_became_active":"2024-03-29T04:17:35.576446+0000","last_became_peered":"2024-03-29T04:17:35.576446+0000","last_unstale":"2024-03-29T04:17:44.470278+0000","last_undegraded":"2024-03-29T04:17:44.470278+0000","last_fullsized":"2024-03-29T04:17:44.470278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:53.508449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470815+0000","last_change":"2024-03-29T04:17:41.170866+0000","last_active":"2024-03-29T04:17:44.470815+0000","last_peered":"2024-03-29T04:17:44.470815+0000","last_clean":"2024-03-29T04:17:44.470815+0000","last_became_active":"2024-03-29T04:17:41.169521+0000","last_became_peered":"2024-03-29T04:17:41.169521+0000","last_unstale":"2024-03-29T04:17:44.470815+0000","last_undegraded":"2024-03-29T04:17:44.470815+0000","last_fullsized":"2024-03-29T04:17:44.470815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:33:27.957344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.657095+0000","last_change":"2024-03-29T04:17:36.624392+0000","last_active":"2024-03-29T04:17:42.657095+0000","last_peered":"2024-03-29T04:17:42.657095+0000","last_clean":"2024-03-29T04:17:42.657095+0000","last_became_active":"2024-03-29T04:17:36.612708+0000","last_became_peered":"2024-03-29T04:17:36.612708+0000","last_unstale":"2024-03-29T04:17:42.657095+0000","last_undegraded":"2024-03-29T04:17:42.657095+0000","last_fullsized":"2024-03-29T04:17:42.657095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:59.355649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634144+0000","last_change":"2024-03-29T04:17:35.596985+0000","last_active":"2024-03-29T04:17:44.634144+0000","last_peered":"2024-03-29T04:17:44.634144+0000","last_clean":"2024-03-29T04:17:44.634144+0000","last_became_active":"2024-03-29T04:17:35.595917+0000","last_became_peered":"2024-03-29T04:17:35.595917+0000","last_unstale":"2024-03-29T04:17:44.634144+0000","last_undegraded":"2024-03-29T04:17:44.634144+0000","last_fullsized":"2024-03-29T04:17:44.634144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:24:46.122034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.658384+0000","last_change":"2024-03-29T04:17:40.664618+0000","last_active":"2024-03-29T04:17:42.658384+0000","last_peered":"2024-03-29T04:17:42.658384+0000","last_clean":"2024-03-29T04:17:42.658384+0000","last_became_active":"2024-03-29T04:17:40.664144+0000","last_became_peered":"2024-03-29T04:17:40.664144+0000","last_unstale":"2024-03-29T04:17:42.658384+0000","last_undegraded":"2024-03-29T04:17:42.658384+0000","last_fullsized":"2024-03-29T04:17:42.658384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:59:47.099511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653827+0000","last_change":"2024-03-29T04:17:36.596925+0000","last_active":"2024-03-29T04:17:42.653827+0000","last_peered":"2024-03-29T04:17:42.653827+0000","last_clean":"2024-03-29T04:17:42.653827+0000","last_became_active":"2024-03-29T04:17:36.596307+0000","last_became_peered":"2024-03-29T04:17:36.596307+0000","last_unstale":"2024-03-29T04:17:42.653827+0000","last_undegraded":"2024-03-29T04:17:42.653827+0000","last_fullsized":"2024-03-29T04:17:42.653827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:39.635039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.271188+0000","last_change":"2024-03-29T04:17:35.598885+0000","last_active":"2024-03-29T04:17:44.271188+0000","last_peered":"2024-03-29T04:17:44.271188+0000","last_clean":"2024-03-29T04:17:44.271188+0000","last_became_active":"2024-03-29T04:17:35.598155+0000","last_became_peered":"2024-03-29T04:17:35.598155+0000","last_unstale":"2024-03-29T04:17:44.271188+0000","last_undegraded":"2024-03-29T04:17:44.271188+0000","last_fullsized":"2024-03-29T04:17:44.271188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:58.346822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656759+0000","last_change":"2024-03-29T04:17:41.164185+0000","last_active":"2024-03-29T04:17:42.656759+0000","last_peered":"2024-03-29T04:17:42.656759+0000","last_clean":"2024-03-29T04:17:42.656759+0000","last_became_active":"2024-03-29T04:17:41.163368+0000","last_became_peered":"2024-03-29T04:17:41.163368+0000","last_unstale":"2024-03-29T04:17:42.656759+0000","last_undegraded":"2024-03-29T04:17:42.656759+0000","last_fullsized":"2024-03-29T04:17:42.656759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:04:15.384842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650724+0000","last_change":"2024-03-29T04:17:36.611583+0000","last_active":"2024-03-29T04:17:42.650724+0000","last_peered":"2024-03-29T04:17:42.650724+0000","last_clean":"2024-03-29T04:17:42.650724+0000","last_became_active":"2024-03-29T04:17:36.611204+0000","last_became_peered":"2024-03-29T04:17:36.611204+0000","last_unstale":"2024-03-29T04:17:42.650724+0000","last_undegraded":"2024-03-29T04:17:42.650724+0000","last_fullsized":"2024-03-29T04:17:42.650724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:13.677092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470861+0000","last_change":"2024-03-29T04:17:35.574541+0000","last_active":"2024-03-29T04:17:44.470861+0000","last_peered":"2024-03-29T04:17:44.470861+0000","last_clean":"2024-03-29T04:17:44.470861+0000","last_became_active":"2024-03-29T04:17:35.573297+0000","last_became_peered":"2024-03-29T04:17:35.573297+0000","last_unstale":"2024-03-29T04:17:44.470861+0000","last_undegraded":"2024-03-29T04:17:44.470861+0000","last_fullsized":"2024-03-29T04:17:44.470861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:58.346822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649700+0000","last_change":"2024-03-29T04:17:41.165155+0000","last_active":"2024-03-29T04:17:42.649700+0000","last_peered":"2024-03-29T04:17:42.649700+0000","last_clean":"2024-03-29T04:17:42.649700+0000","last_became_active":"2024-03-29T04:17:41.163801+0000","last_became_peered":"2024-03-29T04:17:41.163801+0000","last_unstale":"2024-03-29T04:17:42.649700+0000","last_undegraded":"2024-03-29T04:17:42.649700+0000","last_fullsized":"2024-03-29T04:17:42.649700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:05:37.595239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470679+0000","last_change":"2024-03-29T04:17:36.590406+0000","last_active":"2024-03-29T04:17:44.470679+0000","last_peered":"2024-03-29T04:17:44.470679+0000","last_clean":"2024-03-29T04:17:44.470679+0000","last_became_active":"2024-03-29T04:17:36.589975+0000","last_became_peered":"2024-03-29T04:17:36.589975+0000","last_unstale":"2024-03-29T04:17:44.470679+0000","last_undegraded":"2024-03-29T04:17:44.470679+0000","last_fullsized":"2024-03-29T04:17:44.470679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:39.635039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634555+0000","last_change":"2024-03-29T04:17:35.602939+0000","last_active":"2024-03-29T04:17:44.634555+0000","last_peered":"2024-03-29T04:17:44.634555+0000","last_clean":"2024-03-29T04:17:44.634555+0000","last_became_active":"2024-03-29T04:17:35.602250+0000","last_became_peered":"2024-03-29T04:17:35.602250+0000","last_unstale":"2024-03-29T04:17:44.634555+0000","last_undegraded":"2024-03-29T04:17:44.634555+0000","last_fullsized":"2024-03-29T04:17:44.634555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:16:13.431728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653145+0000","last_change":"2024-03-29T04:17:41.165561+0000","last_active":"2024-03-29T04:17:42.653145+0000","last_peered":"2024-03-29T04:17:42.653145+0000","last_clean":"2024-03-29T04:17:42.653145+0000","last_became_active":"2024-03-29T04:17:41.163703+0000","last_became_peered":"2024-03-29T04:17:41.163703+0000","last_unstale":"2024-03-29T04:17:42.653145+0000","last_undegraded":"2024-03-29T04:17:42.653145+0000","last_fullsized":"2024-03-29T04:17:42.653145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:10:04.139549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553157+0000","last_change":"2024-03-29T04:17:36.602865+0000","last_active":"2024-03-29T04:17:43.553157+0000","last_peered":"2024-03-29T04:17:43.553157+0000","last_clean":"2024-03-29T04:17:43.553157+0000","last_became_active":"2024-03-29T04:17:36.602578+0000","last_became_peered":"2024-03-29T04:17:36.602578+0000","last_unstale":"2024-03-29T04:17:43.553157+0000","last_undegraded":"2024-03-29T04:17:43.553157+0000","last_fullsized":"2024-03-29T04:17:43.553157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:55:11.413735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.469877+0000","last_change":"2024-03-29T04:17:35.560382+0000","last_active":"2024-03-29T04:17:44.469877+0000","last_peered":"2024-03-29T04:17:44.469877+0000","last_clean":"2024-03-29T04:17:44.469877+0000","last_became_active":"2024-03-29T04:17:35.560122+0000","last_became_peered":"2024-03-29T04:17:35.560122+0000","last_unstale":"2024-03-29T04:17:44.469877+0000","last_undegraded":"2024-03-29T04:17:44.469877+0000","last_fullsized":"2024-03-29T04:17:44.469877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:52:18.910727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470397+0000","last_change":"2024-03-29T04:17:40.663870+0000","last_active":"2024-03-29T04:17:44.470397+0000","last_peered":"2024-03-29T04:17:44.470397+0000","last_clean":"2024-03-29T04:17:44.470397+0000","last_became_active":"2024-03-29T04:17:40.662957+0000","last_became_peered":"2024-03-29T04:17:40.662957+0000","last_unstale":"2024-03-29T04:17:44.470397+0000","last_undegraded":"2024-03-29T04:17:44.470397+0000","last_fullsized":"2024-03-29T04:17:44.470397+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:10:04.139549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.657408+0000","last_change":"2024-03-29T04:17:36.624632+0000","last_active":"2024-03-29T04:17:42.657408+0000","last_peered":"2024-03-29T04:17:42.657408+0000","last_clean":"2024-03-29T04:17:42.657408+0000","last_became_active":"2024-03-29T04:17:36.621308+0000","last_became_peered":"2024-03-29T04:17:36.621308+0000","last_unstale":"2024-03-29T04:17:42.657408+0000","last_undegraded":"2024-03-29T04:17:42.657408+0000","last_fullsized":"2024-03-29T04:17:42.657408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:11:44.672805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553434+0000","last_change":"2024-03-29T04:17:35.590306+0000","last_active":"2024-03-29T04:17:43.553434+0000","last_peered":"2024-03-29T04:17:43.553434+0000","last_clean":"2024-03-29T04:17:43.553434+0000","last_became_active":"2024-03-29T04:17:35.582422+0000","last_became_peered":"2024-03-29T04:17:35.582422+0000","last_unstale":"2024-03-29T04:17:43.553434+0000","last_undegraded":"2024-03-29T04:17:43.553434+0000","last_fullsized":"2024-03-29T04:17:43.553434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:58.346822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470758+0000","last_change":"2024-03-29T04:17:40.661919+0000","last_active":"2024-03-29T04:17:44.470758+0000","last_peered":"2024-03-29T04:17:44.470758+0000","last_clean":"2024-03-29T04:17:44.470758+0000","last_became_active":"2024-03-29T04:17:40.661251+0000","last_became_peered":"2024-03-29T04:17:40.661251+0000","last_unstale":"2024-03-29T04:17:44.470758+0000","last_undegraded":"2024-03-29T04:17:44.470758+0000","last_fullsized":"2024-03-29T04:17:44.470758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:59:47.099511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470257+0000","last_change":"2024-03-29T04:17:36.590933+0000","last_active":"2024-03-29T04:17:44.470257+0000","last_peered":"2024-03-29T04:17:44.470257+0000","last_clean":"2024-03-29T04:17:44.470257+0000","last_became_active":"2024-03-29T04:17:36.590535+0000","last_became_peered":"2024-03-29T04:17:36.590535+0000","last_unstale":"2024-03-29T04:17:44.470257+0000","last_undegraded":"2024-03-29T04:17:44.470257+0000","last_fullsized":"2024-03-29T04:17:44.470257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:41.872399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578021+0000","last_change":"2024-03-29T04:17:35.565257+0000","last_active":"2024-03-29T04:17:43.578021+0000","last_peered":"2024-03-29T04:17:43.578021+0000","last_clean":"2024-03-29T04:17:43.578021+0000","last_became_active":"2024-03-29T04:17:35.564547+0000","last_became_peered":"2024-03-29T04:17:35.564547+0000","last_unstale":"2024-03-29T04:17:43.578021+0000","last_undegraded":"2024-03-29T04:17:43.578021+0000","last_fullsized":"2024-03-29T04:17:43.578021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:04:44.150824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470921+0000","last_change":"2024-03-29T04:17:40.657201+0000","last_active":"2024-03-29T04:17:44.470921+0000","last_peered":"2024-03-29T04:17:44.470921+0000","last_clean":"2024-03-29T04:17:44.470921+0000","last_became_active":"2024-03-29T04:17:40.655333+0000","last_became_peered":"2024-03-29T04:17:40.655333+0000","last_unstale":"2024-03-29T04:17:44.470921+0000","last_undegraded":"2024-03-29T04:17:44.470921+0000","last_fullsized":"2024-03-29T04:17:44.470921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:37:53.228170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.652829+0000","last_change":"2024-03-29T04:17:36.625393+0000","last_active":"2024-03-29T04:17:42.652829+0000","last_peered":"2024-03-29T04:17:42.652829+0000","last_clean":"2024-03-29T04:17:42.652829+0000","last_became_active":"2024-03-29T04:17:36.625053+0000","last_became_peered":"2024-03-29T04:17:36.625053+0000","last_unstale":"2024-03-29T04:17:42.652829+0000","last_undegraded":"2024-03-29T04:17:42.652829+0000","last_fullsized":"2024-03-29T04:17:42.652829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:16:58.574477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.635249+0000","last_change":"2024-03-29T04:17:35.597695+0000","last_active":"2024-03-29T04:17:44.635249+0000","last_peered":"2024-03-29T04:17:44.635249+0000","last_clean":"2024-03-29T04:17:44.635249+0000","last_became_active":"2024-03-29T04:17:35.596065+0000","last_became_peered":"2024-03-29T04:17:35.596065+0000","last_unstale":"2024-03-29T04:17:44.635249+0000","last_undegraded":"2024-03-29T04:17:44.635249+0000","last_fullsized":"2024-03-29T04:17:44.635249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:11:43.663978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653676+0000","last_change":"2024-03-29T04:17:41.169719+0000","last_active":"2024-03-29T04:17:42.653676+0000","last_peered":"2024-03-29T04:17:42.653676+0000","last_clean":"2024-03-29T04:17:42.653676+0000","last_became_active":"2024-03-29T04:17:41.168241+0000","last_became_peered":"2024-03-29T04:17:41.168241+0000","last_unstale":"2024-03-29T04:17:42.653676+0000","last_undegraded":"2024-03-29T04:17:42.653676+0000","last_fullsized":"2024-03-29T04:17:42.653676+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:37:53.228170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649994+0000","last_change":"2024-03-29T04:17:36.599562+0000","last_active":"2024-03-29T04:17:42.649994+0000","last_peered":"2024-03-29T04:17:42.649994+0000","last_clean":"2024-03-29T04:17:42.649994+0000","last_became_active":"2024-03-29T04:17:36.597078+0000","last_became_peered":"2024-03-29T04:17:36.597078+0000","last_unstale":"2024-03-29T04:17:42.649994+0000","last_undegraded":"2024-03-29T04:17:42.649994+0000","last_fullsized":"2024-03-29T04:17:42.649994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:43:27.568676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553439+0000","last_change":"2024-03-29T04:17:35.589469+0000","last_active":"2024-03-29T04:17:43.553439+0000","last_peered":"2024-03-29T04:17:43.553439+0000","last_clean":"2024-03-29T04:17:43.553439+0000","last_became_active":"2024-03-29T04:17:35.572474+0000","last_became_peered":"2024-03-29T04:17:35.572474+0000","last_unstale":"2024-03-29T04:17:43.553439+0000","last_undegraded":"2024-03-29T04:17:43.553439+0000","last_fullsized":"2024-03-29T04:17:43.553439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:55:50.253366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.635415+0000","last_change":"2024-03-29T04:17:25.361826+0000","last_active":"2024-03-29T04:17:44.635415+0000","last_peered":"2024-03-29T04:17:44.635415+0000","last_clean":"2024-03-29T04:17:44.635415+0000","last_became_active":"2024-03-29T04:17:25.361259+0000","last_became_peered":"2024-03-29T04:17:25.361259+0000","last_unstale":"2024-03-29T04:17:44.635415+0000","last_undegraded":"2024-03-29T04:17:44.635415+0000","last_fullsized":"2024-03-29T04:17:44.635415+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_clean_scrub_stamp":"2024-03-29T04:17:24.334639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:40.027873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270077+0000","last_change":"2024-03-29T04:17:41.167132+0000","last_active":"2024-03-29T04:17:44.270077+0000","last_peered":"2024-03-29T04:17:44.270077+0000","last_clean":"2024-03-29T04:17:44.270077+0000","last_became_active":"2024-03-29T04:17:41.166660+0000","last_became_peered":"2024-03-29T04:17:41.166660+0000","last_unstale":"2024-03-29T04:17:44.270077+0000","last_undegraded":"2024-03-29T04:17:44.270077+0000","last_fullsized":"2024-03-29T04:17:44.270077+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:42:25.113757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634556+0000","last_change":"2024-03-29T04:17:36.612535+0000","last_active":"2024-03-29T04:17:44.634556+0000","last_peered":"2024-03-29T04:17:44.634556+0000","last_clean":"2024-03-29T04:17:44.634556+0000","last_became_active":"2024-03-29T04:17:36.610106+0000","last_became_peered":"2024-03-29T04:17:36.610106+0000","last_unstale":"2024-03-29T04:17:44.634556+0000","last_undegraded":"2024-03-29T04:17:44.634556+0000","last_fullsized":"2024-03-29T04:17:44.634556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:24:58.639606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578939+0000","last_change":"2024-03-29T04:17:35.600209+0000","last_active":"2024-03-29T04:17:43.578939+0000","last_peered":"2024-03-29T04:17:43.578939+0000","last_clean":"2024-03-29T04:17:43.578939+0000","last_became_active":"2024-03-29T04:17:35.599686+0000","last_became_peered":"2024-03-29T04:17:35.599686+0000","last_unstale":"2024-03-29T04:17:43.578939+0000","last_undegraded":"2024-03-29T04:17:43.578939+0000","last_fullsized":"2024-03-29T04:17:43.578939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:55:50.253366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.269503+0000","last_change":"2024-03-29T04:17:41.165901+0000","last_active":"2024-03-29T04:17:44.269503+0000","last_peered":"2024-03-29T04:17:44.269503+0000","last_clean":"2024-03-29T04:17:44.269503+0000","last_became_active":"2024-03-29T04:17:41.165453+0000","last_became_peered":"2024-03-29T04:17:41.165453+0000","last_unstale":"2024-03-29T04:17:44.269503+0000","last_undegraded":"2024-03-29T04:17:44.269503+0000","last_fullsized":"2024-03-29T04:17:44.269503+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:04:15.384842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553553+0000","last_change":"2024-03-29T04:17:36.596650+0000","last_active":"2024-03-29T04:17:43.553553+0000","last_peered":"2024-03-29T04:17:43.553553+0000","last_clean":"2024-03-29T04:17:43.553553+0000","last_became_active":"2024-03-29T04:17:36.596458+0000","last_became_peered":"2024-03-29T04:17:36.596458+0000","last_unstale":"2024-03-29T04:17:43.553553+0000","last_undegraded":"2024-03-29T04:17:43.553553+0000","last_fullsized":"2024-03-29T04:17:43.553553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:42:46.422245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.469894+0000","last_change":"2024-03-29T04:17:35.554693+0000","last_active":"2024-03-29T04:17:44.469894+0000","last_peered":"2024-03-29T04:17:44.469894+0000","last_clean":"2024-03-29T04:17:44.469894+0000","last_became_active":"2024-03-29T04:17:35.554347+0000","last_became_peered":"2024-03-29T04:17:35.554347+0000","last_unstale":"2024-03-29T04:17:44.469894+0000","last_undegraded":"2024-03-29T04:17:44.469894+0000","last_fullsized":"2024-03-29T04:17:44.469894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:24:46.122034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653319+0000","last_change":"2024-03-29T04:17:40.656032+0000","last_active":"2024-03-29T04:17:42.653319+0000","last_peered":"2024-03-29T04:17:42.653319+0000","last_clean":"2024-03-29T04:17:42.653319+0000","last_became_active":"2024-03-29T04:17:40.654947+0000","last_became_peered":"2024-03-29T04:17:40.654947+0000","last_unstale":"2024-03-29T04:17:42.653319+0000","last_undegraded":"2024-03-29T04:17:42.653319+0000","last_fullsized":"2024-03-29T04:17:42.653319+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:59:47.099511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634965+0000","last_change":"2024-03-29T04:17:36.627875+0000","last_active":"2024-03-29T04:17:44.634965+0000","last_peered":"2024-03-29T04:17:44.634965+0000","last_clean":"2024-03-29T04:17:44.634965+0000","last_became_active":"2024-03-29T04:17:36.627498+0000","last_became_peered":"2024-03-29T04:17:36.627498+0000","last_unstale":"2024-03-29T04:17:44.634965+0000","last_undegraded":"2024-03-29T04:17:44.634965+0000","last_fullsized":"2024-03-29T04:17:44.634965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:10:00.061365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578092+0000","last_change":"2024-03-29T04:17:35.568135+0000","last_active":"2024-03-29T04:17:43.578092+0000","last_peered":"2024-03-29T04:17:43.578092+0000","last_clean":"2024-03-29T04:17:43.578092+0000","last_became_active":"2024-03-29T04:17:35.567800+0000","last_became_peered":"2024-03-29T04:17:35.567800+0000","last_unstale":"2024-03-29T04:17:43.578092+0000","last_undegraded":"2024-03-29T04:17:43.578092+0000","last_fullsized":"2024-03-29T04:17:43.578092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:23.283694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552529+0000","last_change":"2024-03-29T04:17:41.166538+0000","last_active":"2024-03-29T04:17:43.552529+0000","last_peered":"2024-03-29T04:17:43.552529+0000","last_clean":"2024-03-29T04:17:43.552529+0000","last_became_active":"2024-03-29T04:17:41.165914+0000","last_became_peered":"2024-03-29T04:17:41.165914+0000","last_unstale":"2024-03-29T04:17:43.552529+0000","last_undegraded":"2024-03-29T04:17:43.552529+0000","last_fullsized":"2024-03-29T04:17:43.552529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:37:53.228170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634358+0000","last_change":"2024-03-29T04:17:36.626068+0000","last_active":"2024-03-29T04:17:44.634358+0000","last_peered":"2024-03-29T04:17:44.634358+0000","last_clean":"2024-03-29T04:17:44.634358+0000","last_became_active":"2024-03-29T04:17:36.625776+0000","last_became_peered":"2024-03-29T04:17:36.625776+0000","last_unstale":"2024-03-29T04:17:44.634358+0000","last_undegraded":"2024-03-29T04:17:44.634358+0000","last_fullsized":"2024-03-29T04:17:44.634358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:55:11.413735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.658456+0000","last_change":"2024-03-29T04:17:35.573245+0000","last_active":"2024-03-29T04:17:42.658456+0000","last_peered":"2024-03-29T04:17:42.658456+0000","last_clean":"2024-03-29T04:17:42.658456+0000","last_became_active":"2024-03-29T04:17:35.572324+0000","last_became_peered":"2024-03-29T04:17:35.572324+0000","last_unstale":"2024-03-29T04:17:42.658456+0000","last_undegraded":"2024-03-29T04:17:42.658456+0000","last_fullsized":"2024-03-29T04:17:42.658456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:07:32.265670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656484+0000","last_change":"2024-03-29T04:17:40.653790+0000","last_active":"2024-03-29T04:17:42.656484+0000","last_peered":"2024-03-29T04:17:42.656484+0000","last_clean":"2024-03-29T04:17:42.656484+0000","last_became_active":"2024-03-29T04:17:40.653115+0000","last_became_peered":"2024-03-29T04:17:40.653115+0000","last_unstale":"2024-03-29T04:17:42.656484+0000","last_undegraded":"2024-03-29T04:17:42.656484+0000","last_fullsized":"2024-03-29T04:17:42.656484+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:55:15.491920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634122+0000","last_change":"2024-03-29T04:17:36.625368+0000","last_active":"2024-03-29T04:17:44.634122+0000","last_peered":"2024-03-29T04:17:44.634122+0000","last_clean":"2024-03-29T04:17:44.634122+0000","last_became_active":"2024-03-29T04:17:36.625073+0000","last_became_peered":"2024-03-29T04:17:36.625073+0000","last_unstale":"2024-03-29T04:17:44.634122+0000","last_undegraded":"2024-03-29T04:17:44.634122+0000","last_fullsized":"2024-03-29T04:17:44.634122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:13.677092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578401+0000","last_change":"2024-03-29T04:17:35.564073+0000","last_active":"2024-03-29T04:17:43.578401+0000","last_peered":"2024-03-29T04:17:43.578401+0000","last_clean":"2024-03-29T04:17:43.578401+0000","last_became_active":"2024-03-29T04:17:35.563521+0000","last_became_peered":"2024-03-29T04:17:35.563521+0000","last_unstale":"2024-03-29T04:17:43.578401+0000","last_undegraded":"2024-03-29T04:17:43.578401+0000","last_fullsized":"2024-03-29T04:17:43.578401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:53.508449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653476+0000","last_change":"2024-03-29T04:17:40.648424+0000","last_active":"2024-03-29T04:17:42.653476+0000","last_peered":"2024-03-29T04:17:42.653476+0000","last_clean":"2024-03-29T04:17:42.653476+0000","last_became_active":"2024-03-29T04:17:40.647665+0000","last_became_peered":"2024-03-29T04:17:40.647665+0000","last_unstale":"2024-03-29T04:17:42.653476+0000","last_undegraded":"2024-03-29T04:17:42.653476+0000","last_fullsized":"2024-03-29T04:17:42.653476+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:33:27.957344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.269744+0000","last_change":"2024-03-29T04:17:36.614419+0000","last_active":"2024-03-29T04:17:44.269744+0000","last_peered":"2024-03-29T04:17:44.269744+0000","last_clean":"2024-03-29T04:17:44.269744+0000","last_became_active":"2024-03-29T04:17:36.614052+0000","last_became_peered":"2024-03-29T04:17:36.614052+0000","last_unstale":"2024-03-29T04:17:44.269744+0000","last_undegraded":"2024-03-29T04:17:44.269744+0000","last_fullsized":"2024-03-29T04:17:44.269744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:41:31.910133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.268906+0000","last_change":"2024-03-29T04:17:35.576576+0000","last_active":"2024-03-29T04:17:44.268906+0000","last_peered":"2024-03-29T04:17:44.268906+0000","last_clean":"2024-03-29T04:17:44.268906+0000","last_became_active":"2024-03-29T04:17:35.575156+0000","last_became_peered":"2024-03-29T04:17:35.575156+0000","last_unstale":"2024-03-29T04:17:44.268906+0000","last_undegraded":"2024-03-29T04:17:44.268906+0000","last_fullsized":"2024-03-29T04:17:44.268906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:52:18.910727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650517+0000","last_change":"2024-03-29T04:17:41.165455+0000","last_active":"2024-03-29T04:17:42.650517+0000","last_peered":"2024-03-29T04:17:42.650517+0000","last_clean":"2024-03-29T04:17:42.650517+0000","last_became_active":"2024-03-29T04:17:41.164305+0000","last_became_peered":"2024-03-29T04:17:41.164305+0000","last_unstale":"2024-03-29T04:17:42.650517+0000","last_undegraded":"2024-03-29T04:17:42.650517+0000","last_fullsized":"2024-03-29T04:17:42.650517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:58:14.948478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634370+0000","last_change":"2024-03-29T04:17:36.626514+0000","last_active":"2024-03-29T04:17:44.634370+0000","last_peered":"2024-03-29T04:17:44.634370+0000","last_clean":"2024-03-29T04:17:44.634370+0000","last_became_active":"2024-03-29T04:17:36.626184+0000","last_became_peered":"2024-03-29T04:17:36.626184+0000","last_unstale":"2024-03-29T04:17:44.634370+0000","last_undegraded":"2024-03-29T04:17:44.634370+0000","last_fullsized":"2024-03-29T04:17:44.634370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:42:46.422245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553262+0000","last_change":"2024-03-29T04:17:35.599574+0000","last_active":"2024-03-29T04:17:43.553262+0000","last_peered":"2024-03-29T04:17:43.553262+0000","last_clean":"2024-03-29T04:17:43.553262+0000","last_became_active":"2024-03-29T04:17:35.589264+0000","last_became_peered":"2024-03-29T04:17:35.589264+0000","last_unstale":"2024-03-29T04:17:43.553262+0000","last_undegraded":"2024-03-29T04:17:43.553262+0000","last_fullsized":"2024-03-29T04:17:43.553262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:52:18.910727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650586+0000","last_change":"2024-03-29T04:17:41.164863+0000","last_active":"2024-03-29T04:17:42.650586+0000","last_peered":"2024-03-29T04:17:42.650586+0000","last_clean":"2024-03-29T04:17:42.650586+0000","last_became_active":"2024-03-29T04:17:41.163756+0000","last_became_peered":"2024-03-29T04:17:41.163756+0000","last_unstale":"2024-03-29T04:17:42.650586+0000","last_undegraded":"2024-03-29T04:17:42.650586+0000","last_fullsized":"2024-03-29T04:17:42.650586+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:54.890778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653149+0000","last_change":"2024-03-29T04:17:36.611832+0000","last_active":"2024-03-29T04:17:42.653149+0000","last_peered":"2024-03-29T04:17:42.653149+0000","last_clean":"2024-03-29T04:17:42.653149+0000","last_became_active":"2024-03-29T04:17:36.611106+0000","last_became_peered":"2024-03-29T04:17:36.611106+0000","last_unstale":"2024-03-29T04:17:42.653149+0000","last_undegraded":"2024-03-29T04:17:42.653149+0000","last_fullsized":"2024-03-29T04:17:42.653149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:10:00.802551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653387+0000","last_change":"2024-03-29T04:17:35.569650+0000","last_active":"2024-03-29T04:17:42.653387+0000","last_peered":"2024-03-29T04:17:42.653387+0000","last_clean":"2024-03-29T04:17:42.653387+0000","last_became_active":"2024-03-29T04:17:35.568995+0000","last_became_peered":"2024-03-29T04:17:35.568995+0000","last_unstale":"2024-03-29T04:17:42.653387+0000","last_undegraded":"2024-03-29T04:17:42.653387+0000","last_fullsized":"2024-03-29T04:17:42.653387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:55:50.253366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552920+0000","last_change":"2024-03-29T04:17:41.166364+0000","last_active":"2024-03-29T04:17:43.552920+0000","last_peered":"2024-03-29T04:17:43.552920+0000","last_clean":"2024-03-29T04:17:43.552920+0000","last_became_active":"2024-03-29T04:17:41.165953+0000","last_became_peered":"2024-03-29T04:17:41.165953+0000","last_unstale":"2024-03-29T04:17:43.552920+0000","last_undegraded":"2024-03-29T04:17:43.552920+0000","last_fullsized":"2024-03-29T04:17:43.552920+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:58.692538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634707+0000","last_change":"2024-03-29T04:17:36.610409+0000","last_active":"2024-03-29T04:17:44.634707+0000","last_peered":"2024-03-29T04:17:44.634707+0000","last_clean":"2024-03-29T04:17:44.634707+0000","last_became_active":"2024-03-29T04:17:36.609783+0000","last_became_peered":"2024-03-29T04:17:36.609783+0000","last_unstale":"2024-03-29T04:17:44.634707+0000","last_undegraded":"2024-03-29T04:17:44.634707+0000","last_fullsized":"2024-03-29T04:17:44.634707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:46:38.204638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:47.106160+0000","last_change":"2024-03-29T04:17:35.576067+0000","last_active":"2024-03-29T04:17:47.106160+0000","last_peered":"2024-03-29T04:17:47.106160+0000","last_clean":"2024-03-29T04:17:47.106160+0000","last_became_active":"2024-03-29T04:17:35.573099+0000","last_became_peered":"2024-03-29T04:17:35.573099+0000","last_unstale":"2024-03-29T04:17:47.106160+0000","last_undegraded":"2024-03-29T04:17:47.106160+0000","last_fullsized":"2024-03-29T04:17:47.106160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:53.508449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470529+0000","last_change":"2024-03-29T04:17:40.658773+0000","last_active":"2024-03-29T04:17:44.470529+0000","last_peered":"2024-03-29T04:17:44.470529+0000","last_clean":"2024-03-29T04:17:44.470529+0000","last_became_active":"2024-03-29T04:17:40.658277+0000","last_became_peered":"2024-03-29T04:17:40.658277+0000","last_unstale":"2024-03-29T04:17:44.470529+0000","last_undegraded":"2024-03-29T04:17:44.470529+0000","last_fullsized":"2024-03-29T04:17:44.470529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:43:44.093178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.658046+0000","last_change":"2024-03-29T04:17:36.620386+0000","last_active":"2024-03-29T04:17:42.658046+0000","last_peered":"2024-03-29T04:17:42.658046+0000","last_clean":"2024-03-29T04:17:42.658046+0000","last_became_active":"2024-03-29T04:17:36.620093+0000","last_became_peered":"2024-03-29T04:17:36.620093+0000","last_unstale":"2024-03-29T04:17:42.658046+0000","last_undegraded":"2024-03-29T04:17:42.658046+0000","last_fullsized":"2024-03-29T04:17:42.658046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:12:37.599253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.469929+0000","last_change":"2024-03-29T04:17:35.592723+0000","last_active":"2024-03-29T04:17:44.469929+0000","last_peered":"2024-03-29T04:17:44.469929+0000","last_clean":"2024-03-29T04:17:44.469929+0000","last_became_active":"2024-03-29T04:17:35.592409+0000","last_became_peered":"2024-03-29T04:17:35.592409+0000","last_unstale":"2024-03-29T04:17:44.469929+0000","last_undegraded":"2024-03-29T04:17:44.469929+0000","last_fullsized":"2024-03-29T04:17:44.469929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:12:36.590426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.657265+0000","last_change":"2024-03-29T04:17:40.654909+0000","last_active":"2024-03-29T04:17:42.657265+0000","last_peered":"2024-03-29T04:17:42.657265+0000","last_clean":"2024-03-29T04:17:42.657265+0000","last_became_active":"2024-03-29T04:17:40.654420+0000","last_became_peered":"2024-03-29T04:17:40.654420+0000","last_unstale":"2024-03-29T04:17:42.657265+0000","last_undegraded":"2024-03-29T04:17:42.657265+0000","last_fullsized":"2024-03-29T04:17:42.657265+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:10:04.139549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653614+0000","last_change":"2024-03-29T04:17:36.599274+0000","last_active":"2024-03-29T04:17:42.653614+0000","last_peered":"2024-03-29T04:17:42.653614+0000","last_clean":"2024-03-29T04:17:42.653614+0000","last_became_active":"2024-03-29T04:17:36.597822+0000","last_became_peered":"2024-03-29T04:17:36.597822+0000","last_unstale":"2024-03-29T04:17:42.653614+0000","last_undegraded":"2024-03-29T04:17:42.653614+0000","last_fullsized":"2024-03-29T04:17:42.653614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:24:47.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553100+0000","last_change":"2024-03-29T04:17:35.592120+0000","last_active":"2024-03-29T04:17:43.553100+0000","last_peered":"2024-03-29T04:17:43.553100+0000","last_clean":"2024-03-29T04:17:43.553100+0000","last_became_active":"2024-03-29T04:17:35.590699+0000","last_became_peered":"2024-03-29T04:17:35.590699+0000","last_unstale":"2024-03-29T04:17:43.553100+0000","last_undegraded":"2024-03-29T04:17:43.553100+0000","last_fullsized":"2024-03-29T04:17:43.553100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:53.508449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649946+0000","last_change":"2024-03-29T04:17:40.661266+0000","last_active":"2024-03-29T04:17:42.649946+0000","last_peered":"2024-03-29T04:17:42.649946+0000","last_clean":"2024-03-29T04:17:42.649946+0000","last_became_active":"2024-03-29T04:17:40.660780+0000","last_became_peered":"2024-03-29T04:17:40.660780+0000","last_unstale":"2024-03-29T04:17:42.649946+0000","last_undegraded":"2024-03-29T04:17:42.649946+0000","last_fullsized":"2024-03-29T04:17:42.649946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:02.859957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552978+0000","last_change":"2024-03-29T04:17:36.610408+0000","last_active":"2024-03-29T04:17:43.552978+0000","last_peered":"2024-03-29T04:17:43.552978+0000","last_clean":"2024-03-29T04:17:43.552978+0000","last_became_active":"2024-03-29T04:17:36.610111+0000","last_became_peered":"2024-03-29T04:17:36.610111+0000","last_unstale":"2024-03-29T04:17:43.552978+0000","last_undegraded":"2024-03-29T04:17:43.552978+0000","last_fullsized":"2024-03-29T04:17:43.552978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:46:38.204638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.269316+0000","last_change":"2024-03-29T04:17:35.601002+0000","last_active":"2024-03-29T04:17:44.269316+0000","last_peered":"2024-03-29T04:17:44.269316+0000","last_clean":"2024-03-29T04:17:44.269316+0000","last_became_active":"2024-03-29T04:17:35.600276+0000","last_became_peered":"2024-03-29T04:17:35.600276+0000","last_unstale":"2024-03-29T04:17:44.269316+0000","last_undegraded":"2024-03-29T04:17:44.269316+0000","last_fullsized":"2024-03-29T04:17:44.269316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:55:50.253366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649873+0000","last_change":"2024-03-29T04:17:41.164850+0000","last_active":"2024-03-29T04:17:42.649873+0000","last_peered":"2024-03-29T04:17:42.649873+0000","last_clean":"2024-03-29T04:17:42.649873+0000","last_became_active":"2024-03-29T04:17:41.163894+0000","last_became_peered":"2024-03-29T04:17:41.163894+0000","last_unstale":"2024-03-29T04:17:42.649873+0000","last_undegraded":"2024-03-29T04:17:42.649873+0000","last_fullsized":"2024-03-29T04:17:42.649873+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:03:39.448893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653276+0000","last_change":"2024-03-29T04:17:41.171200+0000","last_active":"2024-03-29T04:17:42.653276+0000","last_peered":"2024-03-29T04:17:42.653276+0000","last_clean":"2024-03-29T04:17:42.653276+0000","last_became_active":"2024-03-29T04:17:41.169774+0000","last_became_peered":"2024-03-29T04:17:41.169774+0000","last_unstale":"2024-03-29T04:17:42.653276+0000","last_undegraded":"2024-03-29T04:17:42.653276+0000","last_fullsized":"2024-03-29T04:17:42.653276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:43:44.093178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470127+0000","last_change":"2024-03-29T04:17:36.598385+0000","last_active":"2024-03-29T04:17:44.470127+0000","last_peered":"2024-03-29T04:17:44.470127+0000","last_clean":"2024-03-29T04:17:44.470127+0000","last_became_active":"2024-03-29T04:17:36.597989+0000","last_became_peered":"2024-03-29T04:17:36.597989+0000","last_unstale":"2024-03-29T04:17:44.470127+0000","last_undegraded":"2024-03-29T04:17:44.470127+0000","last_fullsized":"2024-03-29T04:17:44.470127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:11.037419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553421+0000","last_change":"2024-03-29T04:17:35.591199+0000","last_active":"2024-03-29T04:17:43.553421+0000","last_peered":"2024-03-29T04:17:43.553421+0000","last_clean":"2024-03-29T04:17:43.553421+0000","last_became_active":"2024-03-29T04:17:35.590444+0000","last_became_peered":"2024-03-29T04:17:35.590444+0000","last_unstale":"2024-03-29T04:17:43.553421+0000","last_undegraded":"2024-03-29T04:17:43.553421+0000","last_fullsized":"2024-03-29T04:17:43.553421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:23.283694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270972+0000","last_change":"2024-03-29T04:17:40.661020+0000","last_active":"2024-03-29T04:17:44.270972+0000","last_peered":"2024-03-29T04:17:44.270972+0000","last_clean":"2024-03-29T04:17:44.270972+0000","last_became_active":"2024-03-29T04:17:40.660552+0000","last_became_peered":"2024-03-29T04:17:40.660552+0000","last_unstale":"2024-03-29T04:17:44.270972+0000","last_undegraded":"2024-03-29T04:17:44.270972+0000","last_fullsized":"2024-03-29T04:17:44.270972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:25:02.717791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649896+0000","last_change":"2024-03-29T04:17:36.619765+0000","last_active":"2024-03-29T04:17:42.649896+0000","last_peered":"2024-03-29T04:17:42.649896+0000","last_clean":"2024-03-29T04:17:42.649896+0000","last_became_active":"2024-03-29T04:17:36.619507+0000","last_became_peered":"2024-03-29T04:17:36.619507+0000","last_unstale":"2024-03-29T04:17:42.649896+0000","last_undegraded":"2024-03-29T04:17:42.649896+0000","last_fullsized":"2024-03-29T04:17:42.649896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:39.635039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649952+0000","last_change":"2024-03-29T04:17:35.575983+0000","last_active":"2024-03-29T04:17:42.649952+0000","last_peered":"2024-03-29T04:17:42.649952+0000","last_clean":"2024-03-29T04:17:42.649952+0000","last_became_active":"2024-03-29T04:17:35.575381+0000","last_became_peered":"2024-03-29T04:17:35.575381+0000","last_unstale":"2024-03-29T04:17:42.649952+0000","last_undegraded":"2024-03-29T04:17:42.649952+0000","last_fullsized":"2024-03-29T04:17:42.649952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:07.631047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.268935+0000","last_change":"2024-03-29T04:17:35.577005+0000","last_active":"2024-03-29T04:17:44.268935+0000","last_peered":"2024-03-29T04:17:44.268935+0000","last_clean":"2024-03-29T04:17:44.268935+0000","last_became_active":"2024-03-29T04:17:35.574501+0000","last_became_peered":"2024-03-29T04:17:35.574501+0000","last_unstale":"2024-03-29T04:17:44.268935+0000","last_undegraded":"2024-03-29T04:17:44.268935+0000","last_fullsized":"2024-03-29T04:17:44.268935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:23.283694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653382+0000","last_change":"2024-03-29T04:17:36.615300+0000","last_active":"2024-03-29T04:17:42.653382+0000","last_peered":"2024-03-29T04:17:42.653382+0000","last_clean":"2024-03-29T04:17:42.653382+0000","last_became_active":"2024-03-29T04:17:36.614767+0000","last_became_peered":"2024-03-29T04:17:36.614767+0000","last_unstale":"2024-03-29T04:17:42.653382+0000","last_undegraded":"2024-03-29T04:17:42.653382+0000","last_fullsized":"2024-03-29T04:17:42.653382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:30.598140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578719+0000","last_change":"2024-03-29T04:17:40.666173+0000","last_active":"2024-03-29T04:17:43.578719+0000","last_peered":"2024-03-29T04:17:43.578719+0000","last_clean":"2024-03-29T04:17:43.578719+0000","last_became_active":"2024-03-29T04:17:40.665205+0000","last_became_peered":"2024-03-29T04:17:40.665205+0000","last_unstale":"2024-03-29T04:17:43.578719+0000","last_undegraded":"2024-03-29T04:17:43.578719+0000","last_fullsized":"2024-03-29T04:17:43.578719+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:10:04.880736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650203+0000","last_change":"2024-03-29T04:17:35.597727+0000","last_active":"2024-03-29T04:17:42.650203+0000","last_peered":"2024-03-29T04:17:42.650203+0000","last_clean":"2024-03-29T04:17:42.650203+0000","last_became_active":"2024-03-29T04:17:35.597476+0000","last_became_peered":"2024-03-29T04:17:35.597476+0000","last_unstale":"2024-03-29T04:17:42.650203+0000","last_undegraded":"2024-03-29T04:17:42.650203+0000","last_fullsized":"2024-03-29T04:17:42.650203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:34.529007+0000","last_clean_scrub_stamp":"2024-03-29T04:17:34.529007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:04:44.150824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.656845+0000","last_change":"2024-03-29T04:17:36.593245+0000","last_active":"2024-03-29T04:17:42.656845+0000","last_peered":"2024-03-29T04:17:42.656845+0000","last_clean":"2024-03-29T04:17:42.656845+0000","last_became_active":"2024-03-29T04:17:36.592508+0000","last_became_peered":"2024-03-29T04:17:36.592508+0000","last_unstale":"2024-03-29T04:17:42.656845+0000","last_undegraded":"2024-03-29T04:17:42.656845+0000","last_fullsized":"2024-03-29T04:17:42.656845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:52:19.919554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650486+0000","last_change":"2024-03-29T04:17:40.664734+0000","last_active":"2024-03-29T04:17:42.650486+0000","last_peered":"2024-03-29T04:17:42.650486+0000","last_clean":"2024-03-29T04:17:42.650486+0000","last_became_active":"2024-03-29T04:17:40.664326+0000","last_became_peered":"2024-03-29T04:17:40.664326+0000","last_unstale":"2024-03-29T04:17:42.650486+0000","last_undegraded":"2024-03-29T04:17:42.650486+0000","last_fullsized":"2024-03-29T04:17:42.650486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:39.616018+0000","last_clean_scrub_stamp":"2024-03-29T04:17:39.616018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:03.774120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.269324+0000","last_change":"2024-03-29T04:17:36.618652+0000","last_active":"2024-03-29T04:17:44.269324+0000","last_peered":"2024-03-29T04:17:44.269324+0000","last_clean":"2024-03-29T04:17:44.269324+0000","last_became_active":"2024-03-29T04:17:36.618384+0000","last_became_peered":"2024-03-29T04:17:36.618384+0000","last_unstale":"2024-03-29T04:17:44.269324+0000","last_undegraded":"2024-03-29T04:17:44.269324+0000","last_fullsized":"2024-03-29T04:17:44.269324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:30.598140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.578474+0000","last_change":"2024-03-29T04:17:36.606149+0000","last_active":"2024-03-29T04:17:43.578474+0000","last_peered":"2024-03-29T04:17:43.578474+0000","last_clean":"2024-03-29T04:17:43.578474+0000","last_became_active":"2024-03-29T04:17:36.605253+0000","last_became_peered":"2024-03-29T04:17:36.605253+0000","last_unstale":"2024-03-29T04:17:43.578474+0000","last_undegraded":"2024-03-29T04:17:43.578474+0000","last_fullsized":"2024-03-29T04:17:43.578474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:24:47.130861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553107+0000","last_change":"2024-03-29T04:17:36.602472+0000","last_active":"2024-03-29T04:17:43.553107+0000","last_peered":"2024-03-29T04:17:43.553107+0000","last_clean":"2024-03-29T04:17:43.553107+0000","last_became_active":"2024-03-29T04:17:36.601990+0000","last_became_peered":"2024-03-29T04:17:36.601990+0000","last_unstale":"2024-03-29T04:17:43.553107+0000","last_undegraded":"2024-03-29T04:17:43.553107+0000","last_fullsized":"2024-03-29T04:17:43.553107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:38:20.641332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653238+0000","last_change":"2024-03-29T04:17:36.607285+0000","last_active":"2024-03-29T04:17:42.653238+0000","last_peered":"2024-03-29T04:17:42.653238+0000","last_clean":"2024-03-29T04:17:42.653238+0000","last_became_active":"2024-03-29T04:17:36.606419+0000","last_became_peered":"2024-03-29T04:17:36.606419+0000","last_unstale":"2024-03-29T04:17:42.653238+0000","last_undegraded":"2024-03-29T04:17:42.653238+0000","last_fullsized":"2024-03-29T04:17:42.653238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:41.872399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650775+0000","last_change":"2024-03-29T04:17:36.627762+0000","last_active":"2024-03-29T04:17:42.650775+0000","last_peered":"2024-03-29T04:17:42.650775+0000","last_clean":"2024-03-29T04:17:42.650775+0000","last_became_active":"2024-03-29T04:17:36.627476+0000","last_became_peered":"2024-03-29T04:17:36.627476+0000","last_unstale":"2024-03-29T04:17:42.650775+0000","last_undegraded":"2024-03-29T04:17:42.650775+0000","last_fullsized":"2024-03-29T04:17:42.650775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:38:20.641332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470447+0000","last_change":"2024-03-29T04:17:36.597795+0000","last_active":"2024-03-29T04:17:44.470447+0000","last_peered":"2024-03-29T04:17:44.470447+0000","last_clean":"2024-03-29T04:17:44.470447+0000","last_became_active":"2024-03-29T04:17:36.597490+0000","last_became_peered":"2024-03-29T04:17:36.597490+0000","last_unstale":"2024-03-29T04:17:44.470447+0000","last_undegraded":"2024-03-29T04:17:44.470447+0000","last_fullsized":"2024-03-29T04:17:44.470447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:43:27.568676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270661+0000","last_change":"2024-03-29T04:17:36.609886+0000","last_active":"2024-03-29T04:17:44.270661+0000","last_peered":"2024-03-29T04:17:44.270661+0000","last_clean":"2024-03-29T04:17:44.270661+0000","last_became_active":"2024-03-29T04:17:36.609354+0000","last_became_peered":"2024-03-29T04:17:36.609354+0000","last_unstale":"2024-03-29T04:17:44.270661+0000","last_undegraded":"2024-03-29T04:17:44.270661+0000","last_fullsized":"2024-03-29T04:17:44.270661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:24:47.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650587+0000","last_change":"2024-03-29T04:17:36.627964+0000","last_active":"2024-03-29T04:17:42.650587+0000","last_peered":"2024-03-29T04:17:42.650587+0000","last_clean":"2024-03-29T04:17:42.650587+0000","last_became_active":"2024-03-29T04:17:36.627606+0000","last_became_peered":"2024-03-29T04:17:36.627606+0000","last_unstale":"2024-03-29T04:17:42.650587+0000","last_undegraded":"2024-03-29T04:17:42.650587+0000","last_fullsized":"2024-03-29T04:17:42.650587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:44.877184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552870+0000","last_change":"2024-03-29T04:17:36.617738+0000","last_active":"2024-03-29T04:17:43.552870+0000","last_peered":"2024-03-29T04:17:43.552870+0000","last_clean":"2024-03-29T04:17:43.552870+0000","last_became_active":"2024-03-29T04:17:36.617532+0000","last_became_peered":"2024-03-29T04:17:36.617532+0000","last_unstale":"2024-03-29T04:17:43.552870+0000","last_undegraded":"2024-03-29T04:17:43.552870+0000","last_fullsized":"2024-03-29T04:17:43.552870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:41.872399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649836+0000","last_change":"2024-03-29T04:17:36.627025+0000","last_active":"2024-03-29T04:17:42.649836+0000","last_peered":"2024-03-29T04:17:42.649836+0000","last_clean":"2024-03-29T04:17:42.649836+0000","last_became_active":"2024-03-29T04:17:36.626671+0000","last_became_peered":"2024-03-29T04:17:36.626671+0000","last_unstale":"2024-03-29T04:17:42.649836+0000","last_undegraded":"2024-03-29T04:17:42.649836+0000","last_fullsized":"2024-03-29T04:17:42.649836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:31:36.783083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552909+0000","last_change":"2024-03-29T04:17:36.609953+0000","last_active":"2024-03-29T04:17:43.552909+0000","last_peered":"2024-03-29T04:17:43.552909+0000","last_clean":"2024-03-29T04:17:43.552909+0000","last_became_active":"2024-03-29T04:17:36.609455+0000","last_became_peered":"2024-03-29T04:17:36.609455+0000","last_unstale":"2024-03-29T04:17:43.552909+0000","last_undegraded":"2024-03-29T04:17:43.552909+0000","last_fullsized":"2024-03-29T04:17:43.552909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:31:36.783083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653087+0000","last_change":"2024-03-29T04:17:36.618490+0000","last_active":"2024-03-29T04:17:42.653087+0000","last_peered":"2024-03-29T04:17:42.653087+0000","last_clean":"2024-03-29T04:17:42.653087+0000","last_became_active":"2024-03-29T04:17:36.618012+0000","last_became_peered":"2024-03-29T04:17:36.618012+0000","last_unstale":"2024-03-29T04:17:42.653087+0000","last_undegraded":"2024-03-29T04:17:42.653087+0000","last_fullsized":"2024-03-29T04:17:42.653087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:43:27.568676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270104+0000","last_change":"2024-03-29T04:17:36.621359+0000","last_active":"2024-03-29T04:17:44.270104+0000","last_peered":"2024-03-29T04:17:44.270104+0000","last_clean":"2024-03-29T04:17:44.270104+0000","last_became_active":"2024-03-29T04:17:36.620689+0000","last_became_peered":"2024-03-29T04:17:36.620689+0000","last_unstale":"2024-03-29T04:17:44.270104+0000","last_undegraded":"2024-03-29T04:17:44.270104+0000","last_fullsized":"2024-03-29T04:17:44.270104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:43:27.568676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634352+0000","last_change":"2024-03-29T04:17:36.610675+0000","last_active":"2024-03-29T04:17:44.634352+0000","last_peered":"2024-03-29T04:17:44.634352+0000","last_clean":"2024-03-29T04:17:44.634352+0000","last_became_active":"2024-03-29T04:17:36.609797+0000","last_became_peered":"2024-03-29T04:17:36.609797+0000","last_unstale":"2024-03-29T04:17:44.634352+0000","last_undegraded":"2024-03-29T04:17:44.634352+0000","last_fullsized":"2024-03-29T04:17:44.634352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:04:11.306658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.579363+0000","last_change":"2024-03-29T04:17:36.605354+0000","last_active":"2024-03-29T04:17:43.579363+0000","last_peered":"2024-03-29T04:17:43.579363+0000","last_clean":"2024-03-29T04:17:43.579363+0000","last_became_active":"2024-03-29T04:17:36.605028+0000","last_became_peered":"2024-03-29T04:17:36.605028+0000","last_unstale":"2024-03-29T04:17:43.579363+0000","last_undegraded":"2024-03-29T04:17:43.579363+0000","last_fullsized":"2024-03-29T04:17:43.579363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:12:37.599253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553747+0000","last_change":"2024-03-29T04:17:36.611310+0000","last_active":"2024-03-29T04:17:43.553747+0000","last_peered":"2024-03-29T04:17:43.553747+0000","last_clean":"2024-03-29T04:17:43.553747+0000","last_became_active":"2024-03-29T04:17:36.611020+0000","last_became_peered":"2024-03-29T04:17:36.611020+0000","last_unstale":"2024-03-29T04:17:43.553747+0000","last_undegraded":"2024-03-29T04:17:43.553747+0000","last_fullsized":"2024-03-29T04:17:43.553747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:42.915247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552533+0000","last_change":"2024-03-29T04:17:36.618624+0000","last_active":"2024-03-29T04:17:43.552533+0000","last_peered":"2024-03-29T04:17:43.552533+0000","last_clean":"2024-03-29T04:17:43.552533+0000","last_became_active":"2024-03-29T04:17:36.618282+0000","last_became_peered":"2024-03-29T04:17:36.618282+0000","last_unstale":"2024-03-29T04:17:43.552533+0000","last_undegraded":"2024-03-29T04:17:43.552533+0000","last_fullsized":"2024-03-29T04:17:43.552533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:24:47.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470271+0000","last_change":"2024-03-29T04:17:36.587187+0000","last_active":"2024-03-29T04:17:44.470271+0000","last_peered":"2024-03-29T04:17:44.470271+0000","last_clean":"2024-03-29T04:17:44.470271+0000","last_became_active":"2024-03-29T04:17:36.586740+0000","last_became_peered":"2024-03-29T04:17:36.586740+0000","last_unstale":"2024-03-29T04:17:44.470271+0000","last_undegraded":"2024-03-29T04:17:44.470271+0000","last_fullsized":"2024-03-29T04:17:44.470271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:24:47.519285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653115+0000","last_change":"2024-03-29T04:17:36.615925+0000","last_active":"2024-03-29T04:17:42.653115+0000","last_peered":"2024-03-29T04:17:42.653115+0000","last_clean":"2024-03-29T04:17:42.653115+0000","last_became_active":"2024-03-29T04:17:36.615143+0000","last_became_peered":"2024-03-29T04:17:36.615143+0000","last_unstale":"2024-03-29T04:17:42.653115+0000","last_undegraded":"2024-03-29T04:17:42.653115+0000","last_fullsized":"2024-03-29T04:17:42.653115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:50:12.920911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.650032+0000","last_change":"2024-03-29T04:17:36.605090+0000","last_active":"2024-03-29T04:17:42.650032+0000","last_peered":"2024-03-29T04:17:42.650032+0000","last_clean":"2024-03-29T04:17:42.650032+0000","last_became_active":"2024-03-29T04:17:36.604669+0000","last_became_peered":"2024-03-29T04:17:36.604669+0000","last_unstale":"2024-03-29T04:17:42.650032+0000","last_undegraded":"2024-03-29T04:17:42.650032+0000","last_fullsized":"2024-03-29T04:17:42.650032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:30.598140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.657981+0000","last_change":"2024-03-29T04:17:36.614472+0000","last_active":"2024-03-29T04:17:42.657981+0000","last_peered":"2024-03-29T04:17:42.657981+0000","last_clean":"2024-03-29T04:17:42.657981+0000","last_became_active":"2024-03-29T04:17:36.613881+0000","last_became_peered":"2024-03-29T04:17:36.613881+0000","last_unstale":"2024-03-29T04:17:42.657981+0000","last_undegraded":"2024-03-29T04:17:42.657981+0000","last_fullsized":"2024-03-29T04:17:42.657981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:24:47.130861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270375+0000","last_change":"2024-03-29T04:17:36.600125+0000","last_active":"2024-03-29T04:17:44.270375+0000","last_peered":"2024-03-29T04:17:44.270375+0000","last_clean":"2024-03-29T04:17:44.270375+0000","last_became_active":"2024-03-29T04:17:36.599450+0000","last_became_peered":"2024-03-29T04:17:36.599450+0000","last_unstale":"2024-03-29T04:17:44.270375+0000","last_undegraded":"2024-03-29T04:17:44.270375+0000","last_fullsized":"2024-03-29T04:17:44.270375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:11.037419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.270626+0000","last_change":"2024-03-29T04:17:36.600177+0000","last_active":"2024-03-29T04:17:44.270626+0000","last_peered":"2024-03-29T04:17:44.270626+0000","last_clean":"2024-03-29T04:17:44.270626+0000","last_became_active":"2024-03-29T04:17:36.599696+0000","last_became_peered":"2024-03-29T04:17:36.599696+0000","last_unstale":"2024-03-29T04:17:44.270626+0000","last_undegraded":"2024-03-29T04:17:44.270626+0000","last_fullsized":"2024-03-29T04:17:44.270626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:39.635039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.635042+0000","last_change":"2024-03-29T04:17:36.610038+0000","last_active":"2024-03-29T04:17:44.635042+0000","last_peered":"2024-03-29T04:17:44.635042+0000","last_clean":"2024-03-29T04:17:44.635042+0000","last_became_active":"2024-03-29T04:17:36.609175+0000","last_became_peered":"2024-03-29T04:17:36.609175+0000","last_unstale":"2024-03-29T04:17:44.635042+0000","last_undegraded":"2024-03-29T04:17:44.635042+0000","last_fullsized":"2024-03-29T04:17:44.635042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:42:21.035572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.649728+0000","last_change":"2024-03-29T04:17:36.616578+0000","last_active":"2024-03-29T04:17:42.649728+0000","last_peered":"2024-03-29T04:17:42.649728+0000","last_clean":"2024-03-29T04:17:42.649728+0000","last_became_active":"2024-03-29T04:17:36.612656+0000","last_became_peered":"2024-03-29T04:17:36.612656+0000","last_unstale":"2024-03-29T04:17:42.649728+0000","last_undegraded":"2024-03-29T04:17:42.649728+0000","last_fullsized":"2024-03-29T04:17:42.649728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:11.037419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634140+0000","last_change":"2024-03-29T04:17:36.618859+0000","last_active":"2024-03-29T04:17:44.634140+0000","last_peered":"2024-03-29T04:17:44.634140+0000","last_clean":"2024-03-29T04:17:44.634140+0000","last_became_active":"2024-03-29T04:17:36.618475+0000","last_became_peered":"2024-03-29T04:17:36.618475+0000","last_unstale":"2024-03-29T04:17:44.634140+0000","last_undegraded":"2024-03-29T04:17:44.634140+0000","last_fullsized":"2024-03-29T04:17:44.634140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:30.729030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.552858+0000","last_change":"2024-03-29T04:17:36.614982+0000","last_active":"2024-03-29T04:17:43.552858+0000","last_peered":"2024-03-29T04:17:43.552858+0000","last_clean":"2024-03-29T04:17:43.552858+0000","last_became_active":"2024-03-29T04:17:36.614701+0000","last_became_peered":"2024-03-29T04:17:36.614701+0000","last_unstale":"2024-03-29T04:17:43.552858+0000","last_undegraded":"2024-03-29T04:17:43.552858+0000","last_fullsized":"2024-03-29T04:17:43.552858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:30.598140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.634536+0000","last_change":"2024-03-29T04:17:36.608699+0000","last_active":"2024-03-29T04:17:44.634536+0000","last_peered":"2024-03-29T04:17:44.634536+0000","last_clean":"2024-03-29T04:17:44.634536+0000","last_became_active":"2024-03-29T04:17:36.608428+0000","last_became_peered":"2024-03-29T04:17:36.608428+0000","last_unstale":"2024-03-29T04:17:44.634536+0000","last_undegraded":"2024-03-29T04:17:44.634536+0000","last_fullsized":"2024-03-29T04:17:44.634536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:44.877184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:43.553129+0000","last_change":"2024-03-29T04:17:36.596870+0000","last_active":"2024-03-29T04:17:43.553129+0000","last_peered":"2024-03-29T04:17:43.553129+0000","last_clean":"2024-03-29T04:17:43.553129+0000","last_became_active":"2024-03-29T04:17:36.596557+0000","last_became_peered":"2024-03-29T04:17:36.596557+0000","last_unstale":"2024-03-29T04:17:43.553129+0000","last_undegraded":"2024-03-29T04:17:43.553129+0000","last_fullsized":"2024-03-29T04:17:43.553129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:44.877184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:42.653260+0000","last_change":"2024-03-29T04:17:36.593793+0000","last_active":"2024-03-29T04:17:42.653260+0000","last_peered":"2024-03-29T04:17:42.653260+0000","last_clean":"2024-03-29T04:17:42.653260+0000","last_became_active":"2024-03-29T04:17:36.592632+0000","last_became_peered":"2024-03-29T04:17:36.592632+0000","last_unstale":"2024-03-29T04:17:42.653260+0000","last_undegraded":"2024-03-29T04:17:42.653260+0000","last_fullsized":"2024-03-29T04:17:42.653260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:41:31.910133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.470745+0000","last_change":"2024-03-29T04:17:36.593148+0000","last_active":"2024-03-29T04:17:44.470745+0000","last_peered":"2024-03-29T04:17:44.470745+0000","last_clean":"2024-03-29T04:17:44.470745+0000","last_became_active":"2024-03-29T04:17:36.592754+0000","last_became_peered":"2024-03-29T04:17:36.592754+0000","last_unstale":"2024-03-29T04:17:44.470745+0000","last_undegraded":"2024-03-29T04:17:44.470745+0000","last_fullsized":"2024-03-29T04:17:44.470745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:30.598140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-29T04:17:44.635518+0000","last_change":"2024-03-29T04:17:36.615787+0000","last_active":"2024-03-29T04:17:44.635518+0000","last_peered":"2024-03-29T04:17:44.635518+0000","last_clean":"2024-03-29T04:17:44.635518+0000","last_became_active":"2024-03-29T04:17:36.615476+0000","last_became_peered":"2024-03-29T04:17:36.615476+0000","last_unstale":"2024-03-29T04:17:44.635518+0000","last_undegraded":"2024-03-29T04:17:44.635518+0000","last_fullsized":"2024-03-29T04:17:44.635518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:35.537834+0000","last_clean_scrub_stamp":"2024-03-29T04:17:35.537834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:42.915247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":876544,"data_stored":703014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":112154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":112190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":278528,"data_stored":112064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":286720,"data_stored":112224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":884736,"data_stored":703070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:17:59.542 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-29T04:17:59.542 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-29T04:17:59.542 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-29T04:17:59.542 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-29T04:17:59.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.720+0000 7f3141fa7640 1 Processor -- start 2024-03-29T04:17:59.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.720+0000 7f3141fa7640 1 -- start start 2024-03-29T04:17:59.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.720+0000 7f3141fa7640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f313c144270 0x7f313c144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:59.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.720+0000 7f3141fa7640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f313c0b2ef0 0x7f313c15f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:59.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.720+0000 7f3141fa7640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f313c0b33f0 con 0x7f313c0b4490 2024-03-29T04:17:59.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.720+0000 7f3141fa7640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f313c144fa0 con 0x7f313c0b2ef0 2024-03-29T04:17:59.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.720+0000 7f3141fa7640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f313c15fce0 con 0x7f313c144270 2024-03-29T04:17:59.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313bfff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f313c0b4490 0x7f313c06c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44316/0 (socket says 172.21.15.22:44316) 2024-03-29T04:17:59.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313bfff640 1 -- 172.21.15.22:0/2408990827 learned_addr learned my addr 172.21.15.22:0/2408990827 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:17:59.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313b7fe640 1 --2- 172.21.15.22:0/2408990827 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f313c144270 0x7f313c144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:59.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313affd640 1 --2- 172.21.15.22:0/2408990827 >> v2:172.21.15.151:3300/0 conn(0x7f313c0b2ef0 0x7f313c15f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:59.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313a7fc640 1 -- 172.21.15.22:0/2408990827 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2578964114 0 0) 0x7f313c0b33f0 con 0x7f313c0b4490 2024-03-29T04:17:59.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313a7fc640 1 -- 172.21.15.22:0/2408990827 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3124003530 con 0x7f313c0b4490 2024-03-29T04:17:59.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313a7fc640 1 -- 172.21.15.22:0/2408990827 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1584942744 0 0) 0x7f3124003530 con 0x7f313c0b4490 2024-03-29T04:17:59.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313a7fc640 1 -- 172.21.15.22:0/2408990827 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f313c144270 msgr2=0x7f313c144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:59.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313a7fc640 1 --2- 172.21.15.22:0/2408990827 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f313c144270 0x7f313c144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313a7fc640 1 -- 172.21.15.22:0/2408990827 >> v2:172.21.15.151:3300/0 conn(0x7f313c0b2ef0 msgr2=0x7f313c15f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:59.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313a7fc640 1 --2- 172.21.15.22:0/2408990827 >> v2:172.21.15.151:3300/0 conn(0x7f313c0b2ef0 0x7f313c15f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313a7fc640 1 -- 172.21.15.22:0/2408990827 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313c0b36d0 con 0x7f313c0b4490 2024-03-29T04:17:59.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313a7fc640 1 -- 172.21.15.22:0/2408990827 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f3134001a00 con 0x7f313c0b4490 2024-03-29T04:17:59.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313a7fc640 1 -- 172.21.15.22:0/2408990827 >> v1:172.21.15.22:6789/0 conn(0x7f313c0b4490 legacy=0x7f313c06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:59.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313a7fc640 1 --2- 172.21.15.22:0/2408990827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31240039c0 0x7f3124003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:59.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313a7fc640 1 --2- 172.21.15.22:0/2408990827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3124004810 0x7f3124004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:59.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313a7fc640 1 --2- 172.21.15.22:0/2408990827 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f313c144270 0x7f31240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:59.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313a7fc640 1 -- 172.21.15.22:0/2408990827 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f313c15fce0 con 0x7f3124004810 2024-03-29T04:17:59.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313a7fc640 1 -- 172.21.15.22:0/2408990827 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f312400b9d0 con 0x7f31240039c0 2024-03-29T04:17:59.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313a7fc640 1 -- 172.21.15.22:0/2408990827 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f312400bb50 con 0x7f313c144270 2024-03-29T04:17:59.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313affd640 1 --2- 172.21.15.22:0/2408990827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3124004810 0x7f3124004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:59.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313bfff640 1 --2- 172.21.15.22:0/2408990827 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f313c144270 0x7f31240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:59.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313b7fe640 1 --2- 172.21.15.22:0/2408990827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31240039c0 0x7f3124003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:59.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313affd640 1 -- 172.21.15.22:0/2408990827 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f313c144270 msgr2=0x7f31240051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:59.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313affd640 1 --2- 172.21.15.22:0/2408990827 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f313c144270 0x7f31240051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313affd640 1 -- 172.21.15.22:0/2408990827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31240039c0 msgr2=0x7f3124003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:17:59.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313affd640 1 --2- 172.21.15.22:0/2408990827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31240039c0 0x7f3124003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313affd640 1 -- 172.21.15.22:0/2408990827 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3124003530 con 0x7f3124004810 2024-03-29T04:17:59.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313affd640 1 --2- 172.21.15.22:0/2408990827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3124004810 0x7f3124004c90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3130009ae0 tx=0x7f313001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eff52e9fd3e13dd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:59.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313a7fc640 1 -- 172.21.15.22:0/2408990827 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31300096b0 con 0x7f3124004810 2024-03-29T04:17:59.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313a7fc640 1 -- 172.21.15.22:0/2408990827 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31300a64c0 con 0x7f3124004810 2024-03-29T04:17:59.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.724+0000 7f313a7fc640 1 -- 172.21.15.22:0/2408990827 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31300a68b0 con 0x7f3124004810 2024-03-29T04:17:59.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f3141fa7640 1 -- 172.21.15.22:0/2408990827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3124004810 msgr2=0x7f3124004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:59.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f3141fa7640 1 --2- 172.21.15.22:0/2408990827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3124004810 0x7f3124004c90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3130009ae0 tx=0x7f313001cab0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f313bfff640 1 -- 172.21.15.22:0/2408990827 reap_dead start 2024-03-29T04:17:59.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f3141fa7640 1 -- 172.21.15.22:0/2408990827 shutdown_connections 2024-03-29T04:17:59.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f3141fa7640 1 -- 172.21.15.22:0/2408990827 >> 172.21.15.22:0/2408990827 conn(0x7f313c0cfee0 msgr2=0x7f313c0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:59.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f3141fa7640 1 -- 172.21.15.22:0/2408990827 shutdown_connections 2024-03-29T04:17:59.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f3141fa7640 1 -- 172.21.15.22:0/2408990827 wait complete. 2024-03-29T04:17:59.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f3141fa7640 1 Processor -- start 2024-03-29T04:17:59.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f3141fa7640 1 -- start start 2024-03-29T04:17:59.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f3141fa7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f313c142b30 0x7f313c1407a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:59.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f3141fa7640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f313c142f10 0x7f313c06ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:59.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f3141fa7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f313c06f140 0x7f313c1f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:59.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f3141fa7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f313c06cd80 con 0x7f313c142b30 2024-03-29T04:17:59.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f3141fa7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f313c0b3f70 con 0x7f313c06f140 2024-03-29T04:17:59.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f3141fa7640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f313c06cf00 con 0x7f313c142f10 2024-03-29T04:17:59.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f313b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f313c142b30 0x7f313c1407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:59.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f313b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f313c142b30 0x7f313c1407a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51726/0 (socket says 172.21.15.22:51726) 2024-03-29T04:17:59.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f313b7fe640 1 -- 172.21.15.22:0/3149462188 learned_addr learned my addr 172.21.15.22:0/3149462188 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:17:59.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f313bfff640 1 --2- 172.21.15.22:0/3149462188 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f313c06f140 0x7f313c1f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:59.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f313affd640 1 --2- 172.21.15.22:0/3149462188 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f313c142f10 0x7f313c06ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:59.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f313b7fe640 1 -- 172.21.15.22:0/3149462188 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f313c142f10 msgr2=0x7f313c06ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:59.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f313b7fe640 1 --2- 172.21.15.22:0/3149462188 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f313c142f10 0x7f313c06ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f313b7fe640 1 -- 172.21.15.22:0/3149462188 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f313c06f140 msgr2=0x7f313c1f7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:59.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f313b7fe640 1 --2- 172.21.15.22:0/3149462188 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f313c06f140 0x7f313c1f7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f313b7fe640 1 -- 172.21.15.22:0/3149462188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313c151290 con 0x7f313c142b30 2024-03-29T04:17:59.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f313b7fe640 1 --2- 172.21.15.22:0/3149462188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f313c142b30 0x7f313c1407a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f312c0115e0 tx=0x7f312c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:59.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f3123fff640 1 -- 172.21.15.22:0/3149462188 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c00f8f0 con 0x7f313c142b30 2024-03-29T04:17:59.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f3123fff640 1 -- 172.21.15.22:0/3149462188 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f312c00fa90 con 0x7f313c142b30 2024-03-29T04:17:59.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f3141fa7640 1 -- 172.21.15.22:0/3149462188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313c151570 con 0x7f313c142b30 2024-03-29T04:17:59.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f3123fff640 1 -- 172.21.15.22:0/3149462188 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c00fe40 con 0x7f313c142b30 2024-03-29T04:17:59.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f3141fa7640 1 -- 172.21.15.22:0/3149462188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f313c151b30 con 0x7f313c142b30 2024-03-29T04:17:59.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.728+0000 7f3141fa7640 1 -- 172.21.15.22:0/3149462188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f313c06f550 con 0x7f313c142b30 2024-03-29T04:17:59.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.732+0000 7f3123fff640 1 -- 172.21.15.22:0/3149462188 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f312c015460 con 0x7f313c142b30 2024-03-29T04:17:59.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.732+0000 7f3123fff640 1 --2- 172.21.15.22:0/3149462188 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3128067fc0 0x7f312806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:17:59.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.732+0000 7f3123fff640 1 -- 172.21.15.22:0/3149462188 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f312c087170 con 0x7f313c142b30 2024-03-29T04:17:59.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.732+0000 7f313affd640 1 --2- 172.21.15.22:0/3149462188 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3128067fc0 0x7f312806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:17:59.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.732+0000 7f313affd640 1 --2- 172.21.15.22:0/3149462188 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3128067fc0 0x7f312806a480 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f31300af000 tx=0x7f3130008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:17:59.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.736+0000 7f3123fff640 1 -- 172.21.15.22:0/3149462188 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f313c06f550 con 0x7f313c142b30 2024-03-29T04:17:59.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.928+0000 7f3141fa7640 1 -- 172.21.15.22:0/3149462188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f313c0722d0 con 0x7f313c142b30 2024-03-29T04:17:59.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.928+0000 7f3123fff640 1 -- 172.21.15.22:0/3149462188 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f312c019030 con 0x7f313c142b30 2024-03-29T04:17:59.908 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:17:59.908 INFO:teuthology.orchestra.run.smithi022.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-29T04:17:59.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.928+0000 7f3141fa7640 1 -- 172.21.15.22:0/3149462188 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3128067fc0 msgr2=0x7f312806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:59.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.928+0000 7f3141fa7640 1 --2- 172.21.15.22:0/3149462188 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3128067fc0 0x7f312806a480 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f31300af000 tx=0x7f3130008040 comp rx=0 tx=0).stop 2024-03-29T04:17:59.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.928+0000 7f3141fa7640 1 -- 172.21.15.22:0/3149462188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f313c142b30 msgr2=0x7f313c1407a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:17:59.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.928+0000 7f3141fa7640 1 --2- 172.21.15.22:0/3149462188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f313c142b30 0x7f313c1407a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f312c0115e0 tx=0x7f312c011aa0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.928+0000 7f3141fa7640 1 -- 172.21.15.22:0/3149462188 shutdown_connections 2024-03-29T04:17:59.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.928+0000 7f3141fa7640 1 --2- 172.21.15.22:0/3149462188 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f313c06f140 0x7f313c1f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.928+0000 7f3141fa7640 1 --2- 172.21.15.22:0/3149462188 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f313c142f10 0x7f313c06ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.928+0000 7f3141fa7640 1 --2- 172.21.15.22:0/3149462188 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3128067fc0 0x7f312806a480 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.928+0000 7f3141fa7640 1 --2- 172.21.15.22:0/3149462188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f313c142b30 0x7f313c1407a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:17:59.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.928+0000 7f3141fa7640 1 -- 172.21.15.22:0/3149462188 >> 172.21.15.22:0/3149462188 conn(0x7f313c0cfee0 msgr2=0x7f313c06c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:17:59.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.928+0000 7f3141fa7640 1 -- 172.21.15.22:0/3149462188 shutdown_connections 2024-03-29T04:17:59.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:17:59.932+0000 7f3141fa7640 1 -- 172.21.15.22:0/3149462188 wait complete. 2024-03-29T04:17:59.927 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-29T04:17:59.927 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:00.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.056+0000 7fceec203640 1 Processor -- start 2024-03-29T04:18:00.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.056+0000 7fceec203640 1 -- start start 2024-03-29T04:18:00.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.060+0000 7fceec203640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcee4144270 0x7fcee4144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.060+0000 7fceec203640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcee40b2ef0 0x7fcee415f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.060+0000 7fceec203640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fcee40b4490 0x7fcee406c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.060+0000 7fceec203640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fcee4144fa0 con 0x7fcee40b2ef0 2024-03-29T04:18:00.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.060+0000 7fceec203640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fcee415fce0 con 0x7fcee40b4490 2024-03-29T04:18:00.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.060+0000 7fceec203640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fcee40b33f0 con 0x7fcee4144270 2024-03-29T04:18:00.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.060+0000 7fcee979e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcee40b2ef0 0x7fcee415f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.060+0000 7fcee979e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcee40b2ef0 0x7fcee415f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51734/0 (socket says 172.21.15.22:51734) 2024-03-29T04:18:00.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.060+0000 7fcee979e640 1 -- 172.21.15.22:0/3141382722 learned_addr learned my addr 172.21.15.22:0/3141382722 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:00.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.060+0000 7fcee9f9f640 1 --2- 172.21.15.22:0/3141382722 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcee4144270 0x7fcee4144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.060+0000 7fceea7a0640 1 --2- 172.21.15.22:0/3141382722 >> v2:172.21.15.151:3300/0 conn(0x7fcee40b4490 0x7fcee406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.060+0000 7fcee979e640 1 -- 172.21.15.22:0/3141382722 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcee4144270 msgr2=0x7fcee4144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:00.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.060+0000 7fcee979e640 1 --2- 172.21.15.22:0/3141382722 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcee4144270 0x7fcee4144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.060+0000 7fcee979e640 1 -- 172.21.15.22:0/3141382722 >> v2:172.21.15.151:3300/0 conn(0x7fcee40b4490 msgr2=0x7fcee406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:00.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.060+0000 7fcee979e640 1 --2- 172.21.15.22:0/3141382722 >> v2:172.21.15.151:3300/0 conn(0x7fcee40b4490 0x7fcee406c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.060+0000 7fcee979e640 1 -- 172.21.15.22:0/3141382722 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee40b36d0 con 0x7fcee40b2ef0 2024-03-29T04:18:00.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.060+0000 7fcee979e640 1 --2- 172.21.15.22:0/3141382722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcee40b2ef0 0x7fcee415f7a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcee40b5430 tx=0x7fced401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1142c6a3a8f3691e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:00.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.060+0000 7fcee8f9d640 1 -- 172.21.15.22:0/3141382722 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced40b1070 con 0x7fcee40b2ef0 2024-03-29T04:18:00.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.060+0000 7fcee8f9d640 1 -- 172.21.15.22:0/3141382722 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced40ac620 con 0x7fcee40b2ef0 2024-03-29T04:18:00.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.060+0000 7fcee8f9d640 1 -- 172.21.15.22:0/3141382722 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced40ac9f0 con 0x7fcee40b2ef0 2024-03-29T04:18:00.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceec203640 1 -- 172.21.15.22:0/3141382722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcee40b2ef0 msgr2=0x7fcee415f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceec203640 1 --2- 172.21.15.22:0/3141382722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcee40b2ef0 0x7fcee415f7a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcee40b5430 tx=0x7fced401c4f0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceec203640 1 -- 172.21.15.22:0/3141382722 shutdown_connections 2024-03-29T04:18:00.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceec203640 1 --2- 172.21.15.22:0/3141382722 >> v2:172.21.15.151:3300/0 conn(0x7fcee40b4490 0x7fcee406c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceec203640 1 --2- 172.21.15.22:0/3141382722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcee40b2ef0 0x7fcee415f7a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceec203640 1 --2- 172.21.15.22:0/3141382722 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcee4144270 0x7fcee4144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceec203640 1 -- 172.21.15.22:0/3141382722 >> 172.21.15.22:0/3141382722 conn(0x7fcee40cfee0 msgr2=0x7fcee40d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:00.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceec203640 1 -- 172.21.15.22:0/3141382722 shutdown_connections 2024-03-29T04:18:00.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceec203640 1 -- 172.21.15.22:0/3141382722 wait complete. 2024-03-29T04:18:00.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceec203640 1 Processor -- start 2024-03-29T04:18:00.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceec203640 1 -- start start 2024-03-29T04:18:00.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceec203640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcee40b2ef0 0x7fcee4162a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceec203640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcee40b4490 0x7fcee413dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceec203640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcee4144270 0x7fcee4143a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceec203640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcee40cde90 con 0x7fcee4144270 2024-03-29T04:18:00.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceec203640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcee40cdd10 con 0x7fcee40b4490 2024-03-29T04:18:00.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceec203640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fcee4163c30 con 0x7fcee40b2ef0 2024-03-29T04:18:00.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceea7a0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcee4144270 0x7fcee4143a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fcee979e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcee40b4490 0x7fcee413dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceea7a0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcee4144270 0x7fcee4143a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51736/0 (socket says 172.21.15.22:51736) 2024-03-29T04:18:00.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceea7a0640 1 -- 172.21.15.22:0/3704172474 learned_addr learned my addr 172.21.15.22:0/3704172474 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:00.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fcee9f9f640 1 --2- 172.21.15.22:0/3704172474 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcee40b2ef0 0x7fcee4162a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceea7a0640 1 -- 172.21.15.22:0/3704172474 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcee40b2ef0 msgr2=0x7fcee4162a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceea7a0640 1 --2- 172.21.15.22:0/3704172474 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcee40b2ef0 0x7fcee4162a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceea7a0640 1 -- 172.21.15.22:0/3704172474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcee40b4490 msgr2=0x7fcee413dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceea7a0640 1 --2- 172.21.15.22:0/3704172474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcee40b4490 0x7fcee413dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceea7a0640 1 -- 172.21.15.22:0/3704172474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee4163020 con 0x7fcee4144270 2024-03-29T04:18:00.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fcee9f9f640 1 --2- 172.21.15.22:0/3704172474 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcee40b2ef0 0x7fcee4162a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:00.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceea7a0640 1 --2- 172.21.15.22:0/3704172474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcee4144270 0x7fcee4143a00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcee0007e60 tx=0x7fcee000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:00.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceda7fc640 1 -- 172.21.15.22:0/3704172474 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee0014620 con 0x7fcee4144270 2024-03-29T04:18:00.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceda7fc640 1 -- 172.21.15.22:0/3704172474 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcee0019070 con 0x7fcee4144270 2024-03-29T04:18:00.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceda7fc640 1 -- 172.21.15.22:0/3704172474 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee0014a20 con 0x7fcee4144270 2024-03-29T04:18:00.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceec203640 1 -- 172.21.15.22:0/3704172474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee4163300 con 0x7fcee4144270 2024-03-29T04:18:00.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.064+0000 7fceec203640 1 -- 172.21.15.22:0/3704172474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee41638c0 con 0x7fcee4144270 2024-03-29T04:18:00.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.068+0000 7fceec203640 1 -- 172.21.15.22:0/3704172474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcee4068890 con 0x7fcee4144270 2024-03-29T04:18:00.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.068+0000 7fceda7fc640 1 -- 172.21.15.22:0/3704172474 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fcee0014bc0 con 0x7fcee4144270 2024-03-29T04:18:00.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.068+0000 7fceda7fc640 1 --2- 172.21.15.22:0/3704172474 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcec8067fc0 0x7fcec806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.068+0000 7fceda7fc640 1 -- 172.21.15.22:0/3704172474 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fcee0087110 con 0x7fcee4144270 2024-03-29T04:18:00.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.068+0000 7fcee9f9f640 1 --2- 172.21.15.22:0/3704172474 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcec8067fc0 0x7fcec806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.068+0000 7fcee9f9f640 1 --2- 172.21.15.22:0/3704172474 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcec8067fc0 0x7fcec806a480 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcee4165190 tx=0x7fcedc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:00.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.072+0000 7fceda7fc640 1 -- 172.21.15.22:0/3704172474 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fcee00574e0 con 0x7fcee4144270 2024-03-29T04:18:00.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.228+0000 7fceec203640 1 -- 172.21.15.22:0/3704172474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcee40722d0 con 0x7fcee4144270 2024-03-29T04:18:00.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.228+0000 7fceda7fc640 1 -- 172.21.15.22:0/3704172474 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18514 (secure 0 0 0) 0x7fcee0056de0 con 0x7fcee4144270 2024-03-29T04:18:00.209 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:00.209 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:17:42.645632+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:00.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.232+0000 7fceec203640 1 -- 172.21.15.22:0/3704172474 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcec8067fc0 msgr2=0x7fcec806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.232+0000 7fceec203640 1 --2- 172.21.15.22:0/3704172474 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcec8067fc0 0x7fcec806a480 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcee4165190 tx=0x7fcedc008040 comp rx=0 tx=0).stop 2024-03-29T04:18:00.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.232+0000 7fceec203640 1 -- 172.21.15.22:0/3704172474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcee4144270 msgr2=0x7fcee4143a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.232+0000 7fceec203640 1 --2- 172.21.15.22:0/3704172474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcee4144270 0x7fcee4143a00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcee0007e60 tx=0x7fcee000e4b0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.232+0000 7fceec203640 1 -- 172.21.15.22:0/3704172474 shutdown_connections 2024-03-29T04:18:00.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.232+0000 7fceec203640 1 --2- 172.21.15.22:0/3704172474 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcec8067fc0 0x7fcec806a480 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.232+0000 7fceec203640 1 --2- 172.21.15.22:0/3704172474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcee4144270 0x7fcee4143a00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.232+0000 7fceec203640 1 --2- 172.21.15.22:0/3704172474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcee40b4490 0x7fcee413dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.232+0000 7fceec203640 1 --2- 172.21.15.22:0/3704172474 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcee40b2ef0 0x7fcee4162a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.232+0000 7fceec203640 1 -- 172.21.15.22:0/3704172474 >> 172.21.15.22:0/3704172474 conn(0x7fcee40cfee0 msgr2=0x7fcee4141dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:00.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.232+0000 7fceec203640 1 -- 172.21.15.22:0/3704172474 shutdown_connections 2024-03-29T04:18:00.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.232+0000 7fceec203640 1 -- 172.21.15.22:0/3704172474 wait complete. 2024-03-29T04:18:00.228 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:17:34.034142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:17:34.886659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:17:38.628632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:18:00.229 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:18:00.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.368+0000 7f58ff7fa640 1 Processor -- start 2024-03-29T04:18:00.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.368+0000 7f58ff7fa640 1 -- start start 2024-03-29T04:18:00.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.368+0000 7f58ff7fa640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f58f80ba090 0x7f58f80cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.368+0000 7f58ff7fa640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58f80d3ab0 con 0x7f58f815e540 2024-03-29T04:18:00.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.368+0000 7f58ff7fa640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58f80d3d90 con 0x7f58f80cf670 2024-03-29T04:18:00.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.368+0000 7f58ff7fa640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f58f80cfae0 con 0x7f58f80ba090 2024-03-29T04:18:00.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.368+0000 7f58fd596640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f58f815e540 0x7f58f80d3140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44330/0 (socket says 172.21.15.22:44330) 2024-03-29T04:18:00.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.368+0000 7f58fd596640 1 -- 172.21.15.22:0/2538371604 learned_addr learned my addr 172.21.15.22:0/2538371604 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:00.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.368+0000 7f58fcd95640 1 --2- 172.21.15.22:0/2538371604 >> v2:172.21.15.22:3300/0 conn(0x7f58f80ba090 0x7f58f80cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.368+0000 7f58fcd95640 1 -- 172.21.15.22:0/2538371604 >> v1:172.21.15.151:6789/0 conn(0x7f58f80cf670 legacy=0x7f58f806bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.368+0000 7f58fcd95640 1 -- 172.21.15.22:0/2538371604 >> v1:172.21.15.22:6789/0 conn(0x7f58f815e540 legacy=0x7f58f80d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.368+0000 7f58effff640 1 -- 172.21.15.22:0/2538371604 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3685186977 0 0) 0x7f58f80d3ab0 con 0x7f58f815e540 2024-03-29T04:18:00.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.368+0000 7f58fcd95640 1 -- 172.21.15.22:0/2538371604 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f806b790 con 0x7f58f80ba090 2024-03-29T04:18:00.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.372+0000 7f58fcd95640 1 --2- 172.21.15.22:0/2538371604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f80ba090 0x7f58f80cf130 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f58e80097c0 tx=0x7f58e801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d65e4f363c692f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:00.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.372+0000 7f58effff640 1 -- 172.21.15.22:0/2538371604 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e80b1070 con 0x7f58f80ba090 2024-03-29T04:18:00.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.372+0000 7f58effff640 1 -- 172.21.15.22:0/2538371604 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e80a4df0 con 0x7f58f80ba090 2024-03-29T04:18:00.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.372+0000 7f58effff640 1 -- 172.21.15.22:0/2538371604 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e80ac660 con 0x7f58f80ba090 2024-03-29T04:18:00.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.372+0000 7f58ff7fa640 1 -- 172.21.15.22:0/2538371604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f80ba090 msgr2=0x7f58f80cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.372+0000 7f58ff7fa640 1 --2- 172.21.15.22:0/2538371604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f80ba090 0x7f58f80cf130 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f58e80097c0 tx=0x7f58e801c410 comp rx=0 tx=0).stop 2024-03-29T04:18:00.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.372+0000 7f58ff7fa640 1 -- 172.21.15.22:0/2538371604 shutdown_connections 2024-03-29T04:18:00.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.372+0000 7f58ff7fa640 1 --2- 172.21.15.22:0/2538371604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f80ba090 0x7f58f80cf130 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.372+0000 7f58ff7fa640 1 -- 172.21.15.22:0/2538371604 >> 172.21.15.22:0/2538371604 conn(0x7f58f80d4360 msgr2=0x7f58f80d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:00.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.372+0000 7f58ff7fa640 1 -- 172.21.15.22:0/2538371604 shutdown_connections 2024-03-29T04:18:00.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.372+0000 7f58ff7fa640 1 -- 172.21.15.22:0/2538371604 wait complete. 2024-03-29T04:18:00.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58ff7fa640 1 Processor -- start 2024-03-29T04:18:00.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58ff7fa640 1 -- start start 2024-03-29T04:18:00.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58ff7fa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f80ba090 0x7f58f81573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58ff7fa640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58f80cf670 0x7f58f8167620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58ff7fa640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f815e540 0x7f58f81692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58ff7fa640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f58f816c6b0 con 0x7f58f80ba090 2024-03-29T04:18:00.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58ff7fa640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f58f80d3850 con 0x7f58f80cf670 2024-03-29T04:18:00.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58ff7fa640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f58f816c830 con 0x7f58f815e540 2024-03-29T04:18:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58fd596640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f80ba090 0x7f58f81573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58fd596640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f80ba090 0x7f58f81573d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51752/0 (socket says 172.21.15.22:51752) 2024-03-29T04:18:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58fd596640 1 -- 172.21.15.22:0/1700749874 learned_addr learned my addr 172.21.15.22:0/1700749874 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58fcd95640 1 --2- 172.21.15.22:0/1700749874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58f80cf670 0x7f58f8167620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58fd596640 1 -- 172.21.15.22:0/1700749874 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f815e540 msgr2=0x7f58f81692c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58fd596640 1 --2- 172.21.15.22:0/1700749874 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f815e540 0x7f58f81692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58fd596640 1 -- 172.21.15.22:0/1700749874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58f80cf670 msgr2=0x7f58f8167620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58fd596640 1 --2- 172.21.15.22:0/1700749874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58f80cf670 0x7f58f8167620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58fd596640 1 -- 172.21.15.22:0/1700749874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f8164e20 con 0x7f58f80ba090 2024-03-29T04:18:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58fd596640 1 --2- 172.21.15.22:0/1700749874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f80ba090 0x7f58f81573d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f58f400c440 tx=0x7f58f400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58edffb640 1 -- 172.21.15.22:0/1700749874 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4018070 con 0x7f58f80ba090 2024-03-29T04:18:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58edffb640 1 -- 172.21.15.22:0/1700749874 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f40074e0 con 0x7f58f80ba090 2024-03-29T04:18:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58ff7fa640 1 -- 172.21.15.22:0/1700749874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f8165100 con 0x7f58f80ba090 2024-03-29T04:18:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58edffb640 1 -- 172.21.15.22:0/1700749874 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4004a10 con 0x7f58f80ba090 2024-03-29T04:18:00.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58ff7fa640 1 -- 172.21.15.22:0/1700749874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f81656c0 con 0x7f58f80ba090 2024-03-29T04:18:00.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58ff7fa640 1 -- 172.21.15.22:0/1700749874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f8068900 con 0x7f58f80ba090 2024-03-29T04:18:00.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58edffb640 1 -- 172.21.15.22:0/1700749874 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f58f401b070 con 0x7f58f80ba090 2024-03-29T04:18:00.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58edffb640 1 --2- 172.21.15.22:0/1700749874 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f58dc067fc0 0x7f58dc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58edffb640 1 -- 172.21.15.22:0/1700749874 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f58f4085a60 con 0x7f58f80ba090 2024-03-29T04:18:00.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.376+0000 7f58fcd95640 1 --2- 172.21.15.22:0/1700749874 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f58dc067fc0 0x7f58dc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.380+0000 7f58fcd95640 1 --2- 172.21.15.22:0/1700749874 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f58dc067fc0 0x7f58dc06a480 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f58e80ae000 tx=0x7f58e8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:00.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.380+0000 7f58edffb640 1 -- 172.21.15.22:0/1700749874 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f58f4055f60 con 0x7f58f80ba090 2024-03-29T04:18:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.536+0000 7f58ff7fa640 1 -- 172.21.15.22:0/1700749874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f58f80d4950 con 0x7f58f80ba090 2024-03-29T04:18:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.540+0000 7f58edffb640 1 -- 172.21.15.22:0/1700749874 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f58f4055860 con 0x7f58f80ba090 2024-03-29T04:18:00.520 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:18:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.540+0000 7f58ff7fa640 1 -- 172.21.15.22:0/1700749874 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f58dc067fc0 msgr2=0x7f58dc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.540+0000 7f58ff7fa640 1 --2- 172.21.15.22:0/1700749874 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f58dc067fc0 0x7f58dc06a480 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f58e80ae000 tx=0x7f58e8008040 comp rx=0 tx=0).stop 2024-03-29T04:18:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.540+0000 7f58ff7fa640 1 -- 172.21.15.22:0/1700749874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f80ba090 msgr2=0x7f58f81573d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.540+0000 7f58ff7fa640 1 --2- 172.21.15.22:0/1700749874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f80ba090 0x7f58f81573d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f58f400c440 tx=0x7f58f400c900 comp rx=0 tx=0).stop 2024-03-29T04:18:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.540+0000 7f58ff7fa640 1 -- 172.21.15.22:0/1700749874 shutdown_connections 2024-03-29T04:18:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.540+0000 7f58ff7fa640 1 --2- 172.21.15.22:0/1700749874 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f815e540 0x7f58f81692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.540+0000 7f58ff7fa640 1 --2- 172.21.15.22:0/1700749874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58f80cf670 0x7f58f8167620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.540+0000 7f58ff7fa640 1 --2- 172.21.15.22:0/1700749874 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f58dc067fc0 0x7f58dc06a480 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.540+0000 7f58ff7fa640 1 --2- 172.21.15.22:0/1700749874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f80ba090 0x7f58f81573d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.540+0000 7f58ff7fa640 1 -- 172.21.15.22:0/1700749874 >> 172.21.15.22:0/1700749874 conn(0x7f58f80d4360 msgr2=0x7f58f815f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.540+0000 7f58ff7fa640 1 -- 172.21.15.22:0/1700749874 shutdown_connections 2024-03-29T04:18:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.540+0000 7f58ff7fa640 1 -- 172.21.15.22:0/1700749874 wait complete. 2024-03-29T04:18:00.538 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:18:00.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61e3826640 1 Processor -- start 2024-03-29T04:18:00.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61e3826640 1 -- start start 2024-03-29T04:18:00.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61e3826640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61dc1627c0 0x7f61dc06b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61e3826640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f61dc070790 0x7f61dc06f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61e3826640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61dc0b4080 con 0x7f61dc0cda80 2024-03-29T04:18:00.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61e3826640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f61dc0b4c50 con 0x7f61dc070790 2024-03-29T04:18:00.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61e3826640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f61dc0b4320 con 0x7f61dc1627c0 2024-03-29T04:18:00.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61e1dc3640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f61dc0cda80 0x7f61dc0b50e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44342/0 (socket says 172.21.15.22:44342) 2024-03-29T04:18:00.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61e1dc3640 1 -- 172.21.15.22:0/2281630344 learned_addr learned my addr 172.21.15.22:0/2281630344 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:00.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61e15c2640 1 --2- 172.21.15.22:0/2281630344 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61dc1627c0 0x7f61dc06b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61e0dc1640 1 --2- 172.21.15.22:0/2281630344 >> v2:172.21.15.151:3300/0 conn(0x7f61dc070790 0x7f61dc06f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61dbfff640 1 -- 172.21.15.22:0/2281630344 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2111096406 0 0) 0x7f61dc0b4080 con 0x7f61dc0cda80 2024-03-29T04:18:00.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61dbfff640 1 -- 172.21.15.22:0/2281630344 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61c4003530 con 0x7f61dc0cda80 2024-03-29T04:18:00.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61dbfff640 1 -- 172.21.15.22:0/2281630344 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2384373094 0 0) 0x7f61c4003530 con 0x7f61dc0cda80 2024-03-29T04:18:00.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61dbfff640 1 -- 172.21.15.22:0/2281630344 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61dc1627c0 msgr2=0x7f61dc06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:00.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61dbfff640 1 --2- 172.21.15.22:0/2281630344 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61dc1627c0 0x7f61dc06b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61dbfff640 1 -- 172.21.15.22:0/2281630344 >> v2:172.21.15.151:3300/0 conn(0x7f61dc070790 msgr2=0x7f61dc06f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:00.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61dbfff640 1 --2- 172.21.15.22:0/2281630344 >> v2:172.21.15.151:3300/0 conn(0x7f61dc070790 0x7f61dc06f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61dbfff640 1 -- 172.21.15.22:0/2281630344 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc0b4600 con 0x7f61dc0cda80 2024-03-29T04:18:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61dbfff640 1 -- 172.21.15.22:0/2281630344 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f61d00016a0 con 0x7f61dc0cda80 2024-03-29T04:18:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61e15c2640 1 --2- 172.21.15.22:0/2281630344 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61dc1627c0 0x7f61dc06b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:18:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61dbfff640 1 -- 172.21.15.22:0/2281630344 >> v1:172.21.15.22:6789/0 conn(0x7f61dc0cda80 legacy=0x7f61dc0b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61dbfff640 1 --2- 172.21.15.22:0/2281630344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61c40039c0 0x7f61c4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61dbfff640 1 --2- 172.21.15.22:0/2281630344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61c4004810 0x7f61c4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61dbfff640 1 --2- 172.21.15.22:0/2281630344 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61dc1627c0 0x7f61c40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61dbfff640 1 -- 172.21.15.22:0/2281630344 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc0b4c50 con 0x7f61c40039c0 2024-03-29T04:18:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61dbfff640 1 -- 172.21.15.22:0/2281630344 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc0b4320 con 0x7f61c4004810 2024-03-29T04:18:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61dbfff640 1 -- 172.21.15.22:0/2281630344 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f61c400b9d0 con 0x7f61dc1627c0 2024-03-29T04:18:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61e15c2640 1 --2- 172.21.15.22:0/2281630344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61c40039c0 0x7f61c4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61e0dc1640 1 --2- 172.21.15.22:0/2281630344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61c4004810 0x7f61c4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61e1dc3640 1 --2- 172.21.15.22:0/2281630344 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61dc1627c0 0x7f61c40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61e15c2640 1 -- 172.21.15.22:0/2281630344 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61dc1627c0 msgr2=0x7f61c40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61e15c2640 1 --2- 172.21.15.22:0/2281630344 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61dc1627c0 0x7f61c40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61e15c2640 1 -- 172.21.15.22:0/2281630344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61c4004810 msgr2=0x7f61c4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:00.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61e15c2640 1 --2- 172.21.15.22:0/2281630344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61c4004810 0x7f61c4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.684+0000 7f61e15c2640 1 -- 172.21.15.22:0/2281630344 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c4003530 con 0x7f61c40039c0 2024-03-29T04:18:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.688+0000 7f61e15c2640 1 --2- 172.21.15.22:0/2281630344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61c40039c0 0x7f61c4003de0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00e350 tx=0x7f61cc01cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eab3cb17d8fc783 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.688+0000 7f61dbfff640 1 -- 172.21.15.22:0/2281630344 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc0b2070 con 0x7f61c40039c0 2024-03-29T04:18:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.688+0000 7f61dbfff640 1 -- 172.21.15.22:0/2281630344 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61cc0a5b60 con 0x7f61c40039c0 2024-03-29T04:18:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.688+0000 7f61dbfff640 1 -- 172.21.15.22:0/2281630344 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc009de0 con 0x7f61c40039c0 2024-03-29T04:18:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.688+0000 7f61e3826640 1 -- 172.21.15.22:0/2281630344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61c40039c0 msgr2=0x7f61c4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.688+0000 7f61e3826640 1 --2- 172.21.15.22:0/2281630344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61c40039c0 0x7f61c4003de0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00e350 tx=0x7f61cc01cdd0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.688+0000 7f61e1dc3640 1 -- 172.21.15.22:0/2281630344 reap_dead start 2024-03-29T04:18:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.688+0000 7f61e3826640 1 -- 172.21.15.22:0/2281630344 shutdown_connections 2024-03-29T04:18:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.688+0000 7f61e3826640 1 -- 172.21.15.22:0/2281630344 >> 172.21.15.22:0/2281630344 conn(0x7f61dc0d1070 msgr2=0x7f61dc0d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:00.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.688+0000 7f61e3826640 1 -- 172.21.15.22:0/2281630344 shutdown_connections 2024-03-29T04:18:00.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.688+0000 7f61e3826640 1 -- 172.21.15.22:0/2281630344 wait complete. 2024-03-29T04:18:00.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.688+0000 7f61e3826640 1 Processor -- start 2024-03-29T04:18:00.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.688+0000 7f61e3826640 1 -- start start 2024-03-29T04:18:00.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e3826640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61c40039c0 0x7f61dc1637b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e3826640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61dc0b1ed0 0x7f61dc1fb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e3826640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61dc0b2350 0x7f61dc14f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e3826640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc0b5b80 con 0x7f61dc0b2350 2024-03-29T04:18:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e3826640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc0b5a00 con 0x7f61c40039c0 2024-03-29T04:18:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e3826640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f61dc0b5d00 con 0x7f61dc0b1ed0 2024-03-29T04:18:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e1dc3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61dc0b2350 0x7f61dc14f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e1dc3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61dc0b2350 0x7f61dc14f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51772/0 (socket says 172.21.15.22:51772) 2024-03-29T04:18:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e1dc3640 1 -- 172.21.15.22:0/2368938292 learned_addr learned my addr 172.21.15.22:0/2368938292 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e15c2640 1 --2- 172.21.15.22:0/2368938292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61c40039c0 0x7f61dc1637b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e0dc1640 1 --2- 172.21.15.22:0/2368938292 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61dc0b1ed0 0x7f61dc1fb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e1dc3640 1 -- 172.21.15.22:0/2368938292 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61dc0b1ed0 msgr2=0x7f61dc1fb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e1dc3640 1 --2- 172.21.15.22:0/2368938292 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61dc0b1ed0 0x7f61dc1fb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e1dc3640 1 -- 172.21.15.22:0/2368938292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61c40039c0 msgr2=0x7f61dc1637b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e1dc3640 1 --2- 172.21.15.22:0/2368938292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61c40039c0 0x7f61dc1637b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e1dc3640 1 -- 172.21.15.22:0/2368938292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc1fc000 con 0x7f61dc0b2350 2024-03-29T04:18:00.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e15c2640 1 --2- 172.21.15.22:0/2368938292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61c40039c0 0x7f61dc1637b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:00.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e1dc3640 1 --2- 172.21.15.22:0/2368938292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61dc0b2350 0x7f61dc14f210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f61dc073e30 tx=0x7f61d000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:00.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61d9ffb640 1 -- 172.21.15.22:0/2368938292 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d000c690 con 0x7f61dc0b2350 2024-03-29T04:18:00.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61d9ffb640 1 -- 172.21.15.22:0/2368938292 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d0018070 con 0x7f61dc0b2350 2024-03-29T04:18:00.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e3826640 1 -- 172.21.15.22:0/2368938292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61dc1fc2e0 con 0x7f61dc0b2350 2024-03-29T04:18:00.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61d9ffb640 1 -- 172.21.15.22:0/2368938292 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d000ca70 con 0x7f61dc0b2350 2024-03-29T04:18:00.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e3826640 1 -- 172.21.15.22:0/2368938292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61dc163290 con 0x7f61dc0b2350 2024-03-29T04:18:00.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e3826640 1 -- 172.21.15.22:0/2368938292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61dc068900 con 0x7f61dc0b2350 2024-03-29T04:18:00.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61d9ffb640 1 -- 172.21.15.22:0/2368938292 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f61d000cc10 con 0x7f61dc0b2350 2024-03-29T04:18:00.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61d9ffb640 1 --2- 172.21.15.22:0/2368938292 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f61c8067fc0 0x7f61c806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61d9ffb640 1 -- 172.21.15.22:0/2368938292 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f61d00863e0 con 0x7f61dc0b2350 2024-03-29T04:18:00.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.692+0000 7f61e15c2640 1 --2- 172.21.15.22:0/2368938292 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f61c8067fc0 0x7f61c806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.696+0000 7f61e15c2640 1 --2- 172.21.15.22:0/2368938292 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f61c8067fc0 0x7f61c806a480 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0af000 tx=0x7f61cc004ac0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:00.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.696+0000 7f61d9ffb640 1 -- 172.21.15.22:0/2368938292 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f61d0056850 con 0x7f61dc0b2350 2024-03-29T04:18:00.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.860+0000 7f61e3826640 1 -- 172.21.15.22:0/2368938292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f61dc163460 con 0x7f61dc0b2350 2024-03-29T04:18:00.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.860+0000 7f61d9ffb640 1 -- 172.21.15.22:0/2368938292 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f61d0056150 con 0x7f61dc0b2350 2024-03-29T04:18:00.843 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:00.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.864+0000 7f61e3826640 1 -- 172.21.15.22:0/2368938292 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f61c8067fc0 msgr2=0x7f61c806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.864+0000 7f61e3826640 1 --2- 172.21.15.22:0/2368938292 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f61c8067fc0 0x7f61c806a480 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0af000 tx=0x7f61cc004ac0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.864+0000 7f61e3826640 1 -- 172.21.15.22:0/2368938292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61dc0b2350 msgr2=0x7f61dc14f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.864+0000 7f61e3826640 1 --2- 172.21.15.22:0/2368938292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61dc0b2350 0x7f61dc14f210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f61dc073e30 tx=0x7f61d000eb90 comp rx=0 tx=0).stop 2024-03-29T04:18:00.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.864+0000 7f61e3826640 1 -- 172.21.15.22:0/2368938292 shutdown_connections 2024-03-29T04:18:00.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.864+0000 7f61e3826640 1 --2- 172.21.15.22:0/2368938292 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f61c8067fc0 0x7f61c806a480 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.864+0000 7f61e3826640 1 --2- 172.21.15.22:0/2368938292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61dc0b2350 0x7f61dc14f210 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.864+0000 7f61e3826640 1 --2- 172.21.15.22:0/2368938292 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f61dc0b1ed0 0x7f61dc1fb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.864+0000 7f61e3826640 1 --2- 172.21.15.22:0/2368938292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61c40039c0 0x7f61dc1637b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.864+0000 7f61e3826640 1 -- 172.21.15.22:0/2368938292 >> 172.21.15.22:0/2368938292 conn(0x7f61dc0d1070 msgr2=0x7f61dc0d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:00.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.864+0000 7f61e3826640 1 -- 172.21.15.22:0/2368938292 shutdown_connections 2024-03-29T04:18:00.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:00.864+0000 7f61e3826640 1 -- 172.21.15.22:0/2368938292 wait complete. 2024-03-29T04:18:00.863 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:18:00.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.004+0000 7efd9a7ec640 1 Processor -- start 2024-03-29T04:18:00.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.004+0000 7efd9a7ec640 1 -- start start 2024-03-29T04:18:00.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.004+0000 7efd9a7ec640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efd94147d90 0x7efd940cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.004+0000 7efd9a7ec640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd9406e960 con 0x7efd940caa00 2024-03-29T04:18:00.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.004+0000 7efd9a7ec640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd9406ec40 con 0x7efd941479b0 2024-03-29T04:18:00.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.004+0000 7efd9a7ec640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7efd9406ef20 con 0x7efd94147d90 2024-03-29T04:18:00.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.004+0000 7efd937fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7efd940caa00 0x7efd940ccbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44358/0 (socket says 172.21.15.22:44358) 2024-03-29T04:18:00.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.004+0000 7efd937fe640 1 -- 172.21.15.22:0/2074629498 learned_addr learned my addr 172.21.15.22:0/2074629498 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:00.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.004+0000 7efd98d89640 1 --2- 172.21.15.22:0/2074629498 >> v2:172.21.15.22:3300/0 conn(0x7efd94147d90 0x7efd940cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.004+0000 7efd92ffd640 1 -- 172.21.15.22:0/2074629498 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3882748863 0 0) 0x7efd9406e960 con 0x7efd940caa00 2024-03-29T04:18:00.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.004+0000 7efd92ffd640 1 -- 172.21.15.22:0/2074629498 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd80003530 con 0x7efd940caa00 2024-03-29T04:18:00.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.004+0000 7efd98d89640 1 -- 172.21.15.22:0/2074629498 >> v1:172.21.15.151:6789/0 conn(0x7efd941479b0 legacy=0x7efd940ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.004+0000 7efd98d89640 1 -- 172.21.15.22:0/2074629498 >> v1:172.21.15.22:6789/0 conn(0x7efd940caa00 legacy=0x7efd940ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.004+0000 7efd98d89640 1 -- 172.21.15.22:0/2074629498 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9406f200 con 0x7efd94147d90 2024-03-29T04:18:00.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd98d89640 1 --2- 172.21.15.22:0/2074629498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd94147d90 0x7efd940cd300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0097f0 tx=0x7efd8c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b61a412c143747 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:00.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd92ffd640 1 -- 172.21.15.22:0/2074629498 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c0b1070 con 0x7efd94147d90 2024-03-29T04:18:00.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd92ffd640 1 -- 172.21.15.22:0/2074629498 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd8c0ac410 con 0x7efd94147d90 2024-03-29T04:18:00.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd92ffd640 1 -- 172.21.15.22:0/2074629498 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c0ac7e0 con 0x7efd94147d90 2024-03-29T04:18:00.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd9a7ec640 1 -- 172.21.15.22:0/2074629498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd94147d90 msgr2=0x7efd940cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd9a7ec640 1 --2- 172.21.15.22:0/2074629498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd94147d90 0x7efd940cd300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0097f0 tx=0x7efd8c01c4a0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd9a7ec640 1 -- 172.21.15.22:0/2074629498 shutdown_connections 2024-03-29T04:18:00.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd9a7ec640 1 --2- 172.21.15.22:0/2074629498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd94147d90 0x7efd940cd300 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd9a7ec640 1 -- 172.21.15.22:0/2074629498 >> 172.21.15.22:0/2074629498 conn(0x7efd940b2c70 msgr2=0x7efd940b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:00.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd9a7ec640 1 -- 172.21.15.22:0/2074629498 shutdown_connections 2024-03-29T04:18:00.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd9a7ec640 1 -- 172.21.15.22:0/2074629498 wait complete. 2024-03-29T04:18:00.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd9a7ec640 1 Processor -- start 2024-03-29T04:18:00.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd9a7ec640 1 -- start start 2024-03-29T04:18:00.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd9a7ec640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd940caa00 0x7efd940d0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd9a7ec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd941479b0 0x7efd9413f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd9a7ec640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efd94147d90 0x7efd9413f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd9a7ec640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efd941606f0 con 0x7efd941479b0 2024-03-29T04:18:00.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd9a7ec640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efd94160570 con 0x7efd940caa00 2024-03-29T04:18:00.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd9a7ec640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7efd9416c170 con 0x7efd94147d90 2024-03-29T04:18:00.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd941479b0 0x7efd9413f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd941479b0 0x7efd9413f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51796/0 (socket says 172.21.15.22:51796) 2024-03-29T04:18:00.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd937fe640 1 -- 172.21.15.22:0/10295202 learned_addr learned my addr 172.21.15.22:0/10295202 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:00.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd98d89640 1 --2- 172.21.15.22:0/10295202 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efd94147d90 0x7efd9413f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd93fff640 1 --2- 172.21.15.22:0/10295202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd940caa00 0x7efd940d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd937fe640 1 -- 172.21.15.22:0/10295202 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efd94147d90 msgr2=0x7efd9413f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd937fe640 1 --2- 172.21.15.22:0/10295202 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efd94147d90 0x7efd9413f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd937fe640 1 -- 172.21.15.22:0/10295202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd940caa00 msgr2=0x7efd940d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.008+0000 7efd937fe640 1 --2- 172.21.15.22:0/10295202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd940caa00 0x7efd940d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.012+0000 7efd937fe640 1 -- 172.21.15.22:0/10295202 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd941435d0 con 0x7efd941479b0 2024-03-29T04:18:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.012+0000 7efd93fff640 1 --2- 172.21.15.22:0/10295202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd940caa00 0x7efd940d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.012+0000 7efd937fe640 1 --2- 172.21.15.22:0/10295202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd941479b0 0x7efd9413f3c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00cbc0 tx=0x7efd7c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.012+0000 7efd90ff9640 1 -- 172.21.15.22:0/10295202 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c017070 con 0x7efd941479b0 2024-03-29T04:18:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.012+0000 7efd90ff9640 1 -- 172.21.15.22:0/10295202 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7c00ad40 con 0x7efd941479b0 2024-03-29T04:18:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.012+0000 7efd9a7ec640 1 -- 172.21.15.22:0/10295202 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd94142a50 con 0x7efd941479b0 2024-03-29T04:18:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.012+0000 7efd90ff9640 1 -- 172.21.15.22:0/10295202 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c012600 con 0x7efd941479b0 2024-03-29T04:18:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.012+0000 7efd9a7ec640 1 -- 172.21.15.22:0/10295202 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd94143010 con 0x7efd941479b0 2024-03-29T04:18:00.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.012+0000 7efd9a7ec640 1 -- 172.21.15.22:0/10295202 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd94068900 con 0x7efd941479b0 2024-03-29T04:18:00.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.012+0000 7efd90ff9640 1 -- 172.21.15.22:0/10295202 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7efd7c01a070 con 0x7efd941479b0 2024-03-29T04:18:00.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.012+0000 7efd90ff9640 1 --2- 172.21.15.22:0/10295202 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efd74067fc0 0x7efd7406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:00.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.012+0000 7efd90ff9640 1 -- 172.21.15.22:0/10295202 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7efd7c086550 con 0x7efd941479b0 2024-03-29T04:18:00.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.012+0000 7efd93fff640 1 --2- 172.21.15.22:0/10295202 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efd74067fc0 0x7efd7406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:00.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.012+0000 7efd93fff640 1 --2- 172.21.15.22:0/10295202 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efd74067fc0 0x7efd7406a480 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efd840016f0 tx=0x7efd84009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:00.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.016+0000 7efd90ff9640 1 -- 172.21.15.22:0/10295202 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7efd7c056370 con 0x7efd941479b0 2024-03-29T04:18:01.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.176+0000 7efd9a7ec640 1 -- 172.21.15.22:0/10295202 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efd94158b40 con 0x7efd941479b0 2024-03-29T04:18:01.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.176+0000 7efd90ff9640 1 -- 172.21.15.22:0/10295202 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7efd7c05a670 con 0x7efd941479b0 2024-03-29T04:18:01.157 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-29T04:18:01.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.176+0000 7efd9a7ec640 1 -- 172.21.15.22:0/10295202 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efd74067fc0 msgr2=0x7efd7406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:01.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.176+0000 7efd9a7ec640 1 --2- 172.21.15.22:0/10295202 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efd74067fc0 0x7efd7406a480 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efd840016f0 tx=0x7efd84009040 comp rx=0 tx=0).stop 2024-03-29T04:18:01.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.176+0000 7efd9a7ec640 1 -- 172.21.15.22:0/10295202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd941479b0 msgr2=0x7efd9413f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:01.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.176+0000 7efd9a7ec640 1 --2- 172.21.15.22:0/10295202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd941479b0 0x7efd9413f3c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00cbc0 tx=0x7efd7c00a430 comp rx=0 tx=0).stop 2024-03-29T04:18:01.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.176+0000 7efd9a7ec640 1 -- 172.21.15.22:0/10295202 shutdown_connections 2024-03-29T04:18:01.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.176+0000 7efd9a7ec640 1 --2- 172.21.15.22:0/10295202 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efd94147d90 0x7efd9413f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.176+0000 7efd9a7ec640 1 --2- 172.21.15.22:0/10295202 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efd74067fc0 0x7efd7406a480 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.176+0000 7efd9a7ec640 1 --2- 172.21.15.22:0/10295202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd941479b0 0x7efd9413f3c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.176+0000 7efd9a7ec640 1 --2- 172.21.15.22:0/10295202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd940caa00 0x7efd940d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.176+0000 7efd9a7ec640 1 -- 172.21.15.22:0/10295202 >> 172.21.15.22:0/10295202 conn(0x7efd940b2c70 msgr2=0x7efd940b0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:01.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.180+0000 7efd9a7ec640 1 -- 172.21.15.22:0/10295202 shutdown_connections 2024-03-29T04:18:01.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.180+0000 7efd9a7ec640 1 -- 172.21.15.22:0/10295202 wait complete. 2024-03-29T04:18:01.180 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:18:01.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.312+0000 7f05565f4640 1 Processor -- start 2024-03-29T04:18:01.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.312+0000 7f05565f4640 1 -- start start 2024-03-29T04:18:01.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.316+0000 7f05565f4640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f055015e540 0x7f05500d3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:01.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.316+0000 7f05565f4640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f05500ba090 0x7f05500cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:01.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.316+0000 7f05565f4640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05500d3ab0 con 0x7f05500cf670 2024-03-29T04:18:01.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.316+0000 7f05565f4640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f05500d4070 con 0x7f05500ba090 2024-03-29T04:18:01.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.316+0000 7f05565f4640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f05500d3d90 con 0x7f055015e540 2024-03-29T04:18:01.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.316+0000 7f054f7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f05500ba090 0x7f05500cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:01.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.316+0000 7f0554b91640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f05500cf670 0x7f05501620f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44360/0 (socket says 172.21.15.22:44360) 2024-03-29T04:18:01.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.316+0000 7f0554b91640 1 -- 172.21.15.22:0/2315566248 learned_addr learned my addr 172.21.15.22:0/2315566248 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:01.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.316+0000 7f054ffff640 1 --2- 172.21.15.22:0/2315566248 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f055015e540 0x7f05500d3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:01.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.316+0000 7f054f7fe640 1 -- 172.21.15.22:0/2315566248 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f055015e540 msgr2=0x7f05500d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:01.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.316+0000 7f054f7fe640 1 --2- 172.21.15.22:0/2315566248 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f055015e540 0x7f05500d3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.316+0000 7f054f7fe640 1 -- 172.21.15.22:0/2315566248 >> v1:172.21.15.22:6789/0 conn(0x7f05500cf670 legacy=0x7f05501620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:01.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.316+0000 7f054f7fe640 1 -- 172.21.15.22:0/2315566248 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055006b790 con 0x7f05500ba090 2024-03-29T04:18:01.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.316+0000 7f054f7fe640 1 --2- 172.21.15.22:0/2315566248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05500ba090 0x7f05500cf130 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f05440097c0 tx=0x7f054401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b23a2061438940f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:01.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.316+0000 7f054effd640 1 -- 172.21.15.22:0/2315566248 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05440b0070 con 0x7f05500ba090 2024-03-29T04:18:01.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.316+0000 7f054effd640 1 -- 172.21.15.22:0/2315566248 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05440ab600 con 0x7f05500ba090 2024-03-29T04:18:01.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.316+0000 7f054effd640 1 -- 172.21.15.22:0/2315566248 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05440ab9d0 con 0x7f05500ba090 2024-03-29T04:18:01.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f05565f4640 1 -- 172.21.15.22:0/2315566248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05500ba090 msgr2=0x7f05500cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:01.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f05565f4640 1 --2- 172.21.15.22:0/2315566248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05500ba090 0x7f05500cf130 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f05440097c0 tx=0x7f054401c410 comp rx=0 tx=0).stop 2024-03-29T04:18:01.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f05565f4640 1 -- 172.21.15.22:0/2315566248 shutdown_connections 2024-03-29T04:18:01.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f05565f4640 1 --2- 172.21.15.22:0/2315566248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05500ba090 0x7f05500cf130 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f05565f4640 1 --2- 172.21.15.22:0/2315566248 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f055015e540 0x7f05500d3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f05565f4640 1 -- 172.21.15.22:0/2315566248 >> 172.21.15.22:0/2315566248 conn(0x7f05500d4360 msgr2=0x7f05500d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:01.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f05565f4640 1 -- 172.21.15.22:0/2315566248 shutdown_connections 2024-03-29T04:18:01.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f05565f4640 1 -- 172.21.15.22:0/2315566248 wait complete. 2024-03-29T04:18:01.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f05565f4640 1 Processor -- start 2024-03-29T04:18:01.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f05565f4640 1 -- start start 2024-03-29T04:18:01.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f05565f4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05500ba090 0x7f05501515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:01.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f05565f4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05500cf670 0x7f05500b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:01.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f05565f4640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f055015e540 0x7f0550157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:01.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f05565f4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f05500d3800 con 0x7f05500cf670 2024-03-29T04:18:01.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f05565f4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f05500d3680 con 0x7f05500ba090 2024-03-29T04:18:01.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f05565f4640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f055016b490 con 0x7f055015e540 2024-03-29T04:18:01.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f054f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05500cf670 0x7f05500b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:01.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f054f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05500cf670 0x7f05500b80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51820/0 (socket says 172.21.15.22:51820) 2024-03-29T04:18:01.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f054f7fe640 1 -- 172.21.15.22:0/282141347 learned_addr learned my addr 172.21.15.22:0/282141347 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:01.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f054ffff640 1 --2- 172.21.15.22:0/282141347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05500ba090 0x7f05501515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:01.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f0554b91640 1 --2- 172.21.15.22:0/282141347 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f055015e540 0x7f0550157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:01.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f054f7fe640 1 -- 172.21.15.22:0/282141347 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f055015e540 msgr2=0x7f0550157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:01.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f054f7fe640 1 --2- 172.21.15.22:0/282141347 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f055015e540 0x7f0550157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f054f7fe640 1 -- 172.21.15.22:0/282141347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05500ba090 msgr2=0x7f05501515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:01.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f054f7fe640 1 --2- 172.21.15.22:0/282141347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05500ba090 0x7f05501515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f054f7fe640 1 -- 172.21.15.22:0/282141347 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0550152d50 con 0x7f05500cf670 2024-03-29T04:18:01.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f054f7fe640 1 --2- 172.21.15.22:0/282141347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05500cf670 0x7f05500b80a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f05440097c0 tx=0x7f05440191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:01.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f054cff9640 1 -- 172.21.15.22:0/282141347 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05440b8030 con 0x7f05500cf670 2024-03-29T04:18:01.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f054cff9640 1 -- 172.21.15.22:0/282141347 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05440ab400 con 0x7f05500cf670 2024-03-29T04:18:01.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f05565f4640 1 -- 172.21.15.22:0/282141347 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0550153030 con 0x7f05500cf670 2024-03-29T04:18:01.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f054cff9640 1 -- 172.21.15.22:0/282141347 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05440b0040 con 0x7f05500cf670 2024-03-29T04:18:01.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f05565f4640 1 -- 172.21.15.22:0/282141347 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05501535f0 con 0x7f05500cf670 2024-03-29T04:18:01.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.320+0000 7f05565f4640 1 -- 172.21.15.22:0/282141347 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0550068900 con 0x7f05500cf670 2024-03-29T04:18:01.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.324+0000 7f054cff9640 1 -- 172.21.15.22:0/282141347 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f05440bb070 con 0x7f05500cf670 2024-03-29T04:18:01.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.324+0000 7f054cff9640 1 --2- 172.21.15.22:0/282141347 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f053c067fc0 0x7f053c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:01.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.324+0000 7f054cff9640 1 -- 172.21.15.22:0/282141347 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f054411fc40 con 0x7f05500cf670 2024-03-29T04:18:01.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.324+0000 7f054ffff640 1 --2- 172.21.15.22:0/282141347 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f053c067fc0 0x7f053c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:01.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.324+0000 7f054ffff640 1 --2- 172.21.15.22:0/282141347 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f053c067fc0 0x7f053c06a480 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f05500ce6f0 tx=0x7f0540008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:01.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.328+0000 7f054cff9640 1 -- 172.21.15.22:0/282141347 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f05440c3020 con 0x7f05500cf670 2024-03-29T04:18:01.357 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:18:01.360 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:18:01.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.484+0000 7f05565f4640 1 -- 172.21.15.22:0/282141347 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f05500d1fa0 con 0x7f05500cf670 2024-03-29T04:18:01.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.484+0000 7f054cff9640 1 -- 172.21.15.22:0/282141347 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f0544016090 con 0x7f05500cf670 2024-03-29T04:18:01.465 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:01.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.484+0000 7f05565f4640 1 -- 172.21.15.22:0/282141347 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f053c067fc0 msgr2=0x7f053c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:01.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.484+0000 7f05565f4640 1 --2- 172.21.15.22:0/282141347 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f053c067fc0 0x7f053c06a480 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f05500ce6f0 tx=0x7f0540008040 comp rx=0 tx=0).stop 2024-03-29T04:18:01.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.484+0000 7f05565f4640 1 -- 172.21.15.22:0/282141347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05500cf670 msgr2=0x7f05500b80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:01.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.484+0000 7f05565f4640 1 --2- 172.21.15.22:0/282141347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05500cf670 0x7f05500b80a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f05440097c0 tx=0x7f05440191b0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.484+0000 7f05565f4640 1 -- 172.21.15.22:0/282141347 shutdown_connections 2024-03-29T04:18:01.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.484+0000 7f05565f4640 1 --2- 172.21.15.22:0/282141347 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f055015e540 0x7f0550157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.484+0000 7f05565f4640 1 --2- 172.21.15.22:0/282141347 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f053c067fc0 0x7f053c06a480 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.484+0000 7f05565f4640 1 --2- 172.21.15.22:0/282141347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05500cf670 0x7f05500b80a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.484+0000 7f05565f4640 1 --2- 172.21.15.22:0/282141347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05500ba090 0x7f05501515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.484+0000 7f05565f4640 1 -- 172.21.15.22:0/282141347 >> 172.21.15.22:0/282141347 conn(0x7f05500d4360 msgr2=0x7f055015f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:01.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.484+0000 7f05565f4640 1 -- 172.21.15.22:0/282141347 shutdown_connections 2024-03-29T04:18:01.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.484+0000 7f05565f4640 1 -- 172.21.15.22:0/282141347 wait complete. 2024-03-29T04:18:01.482 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:18:01.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.624+0000 7efc45351640 1 Processor -- start 2024-03-29T04:18:01.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.624+0000 7efc45351640 1 -- start start 2024-03-29T04:18:01.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.624+0000 7efc45351640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efc4015c0e0 0x7efc400d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:01.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.624+0000 7efc45351640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efc400cfc40 0x7efc400d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:01.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.624+0000 7efc45351640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc400cc6c0 con 0x7efc4015bd00 2024-03-29T04:18:01.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.624+0000 7efc45351640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7efc400c7190 con 0x7efc400cfc40 2024-03-29T04:18:01.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.624+0000 7efc45351640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7efc400cc980 con 0x7efc4015c0e0 2024-03-29T04:18:01.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.624+0000 7efc3f7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efc400cfc40 0x7efc400d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:01.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.624+0000 7efc3f7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efc400cfc40 0x7efc400d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51832/0 (socket says 172.21.15.22:51832) 2024-03-29T04:18:01.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.624+0000 7efc3f7fe640 1 -- 172.21.15.22:0/3195636856 learned_addr learned my addr 172.21.15.22:0/3195636856 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:01.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.624+0000 7efc37fff640 1 --2- 172.21.15.22:0/3195636856 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efc4015c0e0 0x7efc400d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:01.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.624+0000 7efc3f7fe640 1 -- 172.21.15.22:0/3195636856 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efc4015c0e0 msgr2=0x7efc400d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:01.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.624+0000 7efc3f7fe640 1 --2- 172.21.15.22:0/3195636856 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efc4015c0e0 0x7efc400d1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.624+0000 7efc3f7fe640 1 -- 172.21.15.22:0/3195636856 >> v1:172.21.15.151:6789/0 conn(0x7efc4015bd00 legacy=0x7efc400cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:01.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.624+0000 7efc3f7fe640 1 -- 172.21.15.22:0/3195636856 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc400ccc60 con 0x7efc400cfc40 2024-03-29T04:18:01.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.624+0000 7efc37fff640 1 --2- 172.21.15.22:0/3195636856 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efc4015c0e0 0x7efc400d1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:01.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc3f7fe640 1 --2- 172.21.15.22:0/3195636856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc400cfc40 0x7efc400d00d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efc38009e50 tx=0x7efc3801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d41c6503d93372 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:01.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc3e7fc640 1 -- 172.21.15.22:0/3195636856 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc380a56e0 con 0x7efc400cfc40 2024-03-29T04:18:01.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc3e7fc640 1 -- 172.21.15.22:0/3195636856 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc380a5880 con 0x7efc400cfc40 2024-03-29T04:18:01.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc3e7fc640 1 -- 172.21.15.22:0/3195636856 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc380a5c50 con 0x7efc400cfc40 2024-03-29T04:18:01.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc45351640 1 -- 172.21.15.22:0/3195636856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc400cfc40 msgr2=0x7efc400d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:01.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc45351640 1 --2- 172.21.15.22:0/3195636856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc400cfc40 0x7efc400d00d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efc38009e50 tx=0x7efc3801ca30 comp rx=0 tx=0).stop 2024-03-29T04:18:01.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc45351640 1 -- 172.21.15.22:0/3195636856 shutdown_connections 2024-03-29T04:18:01.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc45351640 1 --2- 172.21.15.22:0/3195636856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc400cfc40 0x7efc400d00d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc45351640 1 --2- 172.21.15.22:0/3195636856 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efc4015c0e0 0x7efc400d1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc45351640 1 -- 172.21.15.22:0/3195636856 >> 172.21.15.22:0/3195636856 conn(0x7efc400b3c80 msgr2=0x7efc400b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:01.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc45351640 1 -- 172.21.15.22:0/3195636856 shutdown_connections 2024-03-29T04:18:01.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc45351640 1 -- 172.21.15.22:0/3195636856 wait complete. 2024-03-29T04:18:01.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc45351640 1 Processor -- start 2024-03-29T04:18:01.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc45351640 1 -- start start 2024-03-29T04:18:01.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc45351640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efc400cfc40 0x7efc400b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc45351640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc4015bd00 0x7efc4013c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc45351640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc4015c0e0 0x7efc4015b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc45351640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efc401622e0 con 0x7efc4015c0e0 2024-03-29T04:18:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc45351640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc40162160 con 0x7efc4015bd00 2024-03-29T04:18:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc45351640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7efc40162460 con 0x7efc400cfc40 2024-03-29T04:18:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc3f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc4015c0e0 0x7efc4015b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc3f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc4015c0e0 0x7efc4015b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51840/0 (socket says 172.21.15.22:51840) 2024-03-29T04:18:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc3f7fe640 1 -- 172.21.15.22:0/1899544766 learned_addr learned my addr 172.21.15.22:0/1899544766 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc3effd640 1 --2- 172.21.15.22:0/1899544766 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efc400cfc40 0x7efc400b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc37fff640 1 --2- 172.21.15.22:0/1899544766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc4015bd00 0x7efc4013c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc3f7fe640 1 -- 172.21.15.22:0/1899544766 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efc400cfc40 msgr2=0x7efc400b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc3f7fe640 1 --2- 172.21.15.22:0/1899544766 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efc400cfc40 0x7efc400b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc3f7fe640 1 -- 172.21.15.22:0/1899544766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc4015bd00 msgr2=0x7efc4013c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc3f7fe640 1 --2- 172.21.15.22:0/1899544766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc4015bd00 0x7efc4013c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc3f7fe640 1 -- 172.21.15.22:0/1899544766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc400c9af0 con 0x7efc4015c0e0 2024-03-29T04:18:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc3f7fe640 1 --2- 172.21.15.22:0/1899544766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc4015c0e0 0x7efc4015b0d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efc3801a660 tx=0x7efc380197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc377fe640 1 -- 172.21.15.22:0/1899544766 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc380a56e0 con 0x7efc4015c0e0 2024-03-29T04:18:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc377fe640 1 -- 172.21.15.22:0/1899544766 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc380be070 con 0x7efc4015c0e0 2024-03-29T04:18:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc377fe640 1 -- 172.21.15.22:0/1899544766 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc38009510 con 0x7efc4015c0e0 2024-03-29T04:18:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc45351640 1 -- 172.21.15.22:0/1899544766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc401578c0 con 0x7efc4015c0e0 2024-03-29T04:18:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.628+0000 7efc45351640 1 -- 172.21.15.22:0/1899544766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc400ca090 con 0x7efc4015c0e0 2024-03-29T04:18:01.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.632+0000 7efc45351640 1 -- 172.21.15.22:0/1899544766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc40068890 con 0x7efc4015c0e0 2024-03-29T04:18:01.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.632+0000 7efc377fe640 1 -- 172.21.15.22:0/1899544766 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7efc380b8470 con 0x7efc4015c0e0 2024-03-29T04:18:01.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.632+0000 7efc377fe640 1 --2- 172.21.15.22:0/1899544766 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efc20067fc0 0x7efc2006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:01.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.632+0000 7efc377fe640 1 -- 172.21.15.22:0/1899544766 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7efc38121cf0 con 0x7efc4015c0e0 2024-03-29T04:18:01.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.632+0000 7efc3effd640 1 --2- 172.21.15.22:0/1899544766 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efc20067fc0 0x7efc2006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:01.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.632+0000 7efc3effd640 1 --2- 172.21.15.22:0/1899544766 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efc20067fc0 0x7efc2006a480 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efc30001650 tx=0x7efc30009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:01.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.636+0000 7efc377fe640 1 -- 172.21.15.22:0/1899544766 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7efc38126070 con 0x7efc4015c0e0 2024-03-29T04:18:01.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.832+0000 7efc45351640 1 -- 172.21.15.22:0/1899544766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc400cc200 con 0x7efc4015c0e0 2024-03-29T04:18:01.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.836+0000 7efc377fe640 1 -- 172.21.15.22:0/1899544766 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7efc38016620 con 0x7efc4015c0e0 2024-03-29T04:18:01.816 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:01.816 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3094991863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3094991863},{"type":"v1","addr":"172.21.15.22:6834","nonce":3094991863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2464050026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2464050026},{"type":"v1","addr":"172.21.15.151:6835","nonce":2464050026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2999052384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2999052384},{"type":"v1","addr":"172.21.15.22:6835","nonce":2999052384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:17:35.918069+0000","modified":"2024-03-29T04:17:53.156365+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6837/1466916671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":1466916671},{"type":"v1","addr":"172.21.15.151:6837","nonce":1466916671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-03-29T04:18:01.816 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2024-03-29T04:18:01.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.836+0000 7efc45351640 1 -- 172.21.15.22:0/1899544766 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efc20067fc0 msgr2=0x7efc2006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:01.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.836+0000 7efc45351640 1 --2- 172.21.15.22:0/1899544766 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efc20067fc0 0x7efc2006a480 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efc30001650 tx=0x7efc30009040 comp rx=0 tx=0).stop 2024-03-29T04:18:01.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.836+0000 7efc45351640 1 -- 172.21.15.22:0/1899544766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc4015c0e0 msgr2=0x7efc4015b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:01.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.836+0000 7efc45351640 1 --2- 172.21.15.22:0/1899544766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc4015c0e0 0x7efc4015b0d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efc3801a660 tx=0x7efc380197d0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.836+0000 7efc45351640 1 -- 172.21.15.22:0/1899544766 shutdown_connections 2024-03-29T04:18:01.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.836+0000 7efc45351640 1 --2- 172.21.15.22:0/1899544766 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efc20067fc0 0x7efc2006a480 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.836+0000 7efc45351640 1 --2- 172.21.15.22:0/1899544766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc4015c0e0 0x7efc4015b0d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.836+0000 7efc45351640 1 --2- 172.21.15.22:0/1899544766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc4015bd00 0x7efc4013c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.836+0000 7efc45351640 1 --2- 172.21.15.22:0/1899544766 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efc400cfc40 0x7efc400b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.836+0000 7efc45351640 1 -- 172.21.15.22:0/1899544766 >> 172.21.15.22:0/1899544766 conn(0x7efc400b3c80 msgr2=0x7efc400ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:01.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.836+0000 7efc45351640 1 -- 172.21.15.22:0/1899544766 shutdown_connections 2024-03-29T04:18:01.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.836+0000 7efc45351640 1 -- 172.21.15.22:0/1899544766 wait complete. 2024-03-29T04:18:01.835 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:01.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.972+0000 7f34943c7640 1 Processor -- start 2024-03-29T04:18:01.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.972+0000 7f34943c7640 1 -- start start 2024-03-29T04:18:01.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.972+0000 7f34943c7640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f348c147e10 0x7f348c071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:01.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.972+0000 7f34943c7640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f348c162b90 0x7f348c06e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:01.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.972+0000 7f34943c7640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f348c06ec50 0x7f348c06c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:01.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.972+0000 7f34943c7640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f348c06e280 con 0x7f348c162b90 2024-03-29T04:18:01.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.972+0000 7f34943c7640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f348c072070 con 0x7f348c06ec50 2024-03-29T04:18:01.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.972+0000 7f34943c7640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f348c06dc30 con 0x7f348c147e10 2024-03-29T04:18:01.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.972+0000 7f3491962640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f348c162b90 0x7f348c06e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:01.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.972+0000 7f3491962640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f348c162b90 0x7f348c06e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51848/0 (socket says 172.21.15.22:51848) 2024-03-29T04:18:01.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.972+0000 7f3491962640 1 -- 172.21.15.22:0/229425815 learned_addr learned my addr 172.21.15.22:0/229425815 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:01.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.972+0000 7f3492163640 1 --2- 172.21.15.22:0/229425815 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f348c147e10 0x7f348c071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:01.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.972+0000 7f3492964640 1 --2- 172.21.15.22:0/229425815 >> v2:172.21.15.151:3300/0 conn(0x7f348c06ec50 0x7f348c06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:01.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.972+0000 7f3491962640 1 -- 172.21.15.22:0/229425815 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f348c147e10 msgr2=0x7f348c071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:01.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.972+0000 7f3491962640 1 --2- 172.21.15.22:0/229425815 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f348c147e10 0x7f348c071740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.972+0000 7f3491962640 1 -- 172.21.15.22:0/229425815 >> v2:172.21.15.151:3300/0 conn(0x7f348c06ec50 msgr2=0x7f348c06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:01.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.972+0000 7f3491962640 1 --2- 172.21.15.22:0/229425815 >> v2:172.21.15.151:3300/0 conn(0x7f348c06ec50 0x7f348c06c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.972+0000 7f3491962640 1 -- 172.21.15.22:0/229425815 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348c06df10 con 0x7f348c162b90 2024-03-29T04:18:01.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.976+0000 7f3491962640 1 --2- 172.21.15.22:0/229425815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f348c162b90 0x7f348c06e710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f348c06f030 tx=0x7f347c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794f37a01a5eb8ed server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:01.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.976+0000 7f3491161640 1 -- 172.21.15.22:0/229425815 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c0b0070 con 0x7f348c162b90 2024-03-29T04:18:01.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.976+0000 7f3491161640 1 -- 172.21.15.22:0/229425815 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f347c0ab600 con 0x7f348c162b90 2024-03-29T04:18:01.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.976+0000 7f3491161640 1 -- 172.21.15.22:0/229425815 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c0ab9d0 con 0x7f348c162b90 2024-03-29T04:18:01.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.976+0000 7f34943c7640 1 -- 172.21.15.22:0/229425815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f348c162b90 msgr2=0x7f348c06e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:01.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.976+0000 7f34943c7640 1 --2- 172.21.15.22:0/229425815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f348c162b90 0x7f348c06e710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f348c06f030 tx=0x7f347c01c490 comp rx=0 tx=0).stop 2024-03-29T04:18:01.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.976+0000 7f34943c7640 1 -- 172.21.15.22:0/229425815 shutdown_connections 2024-03-29T04:18:01.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.976+0000 7f34943c7640 1 --2- 172.21.15.22:0/229425815 >> v2:172.21.15.151:3300/0 conn(0x7f348c06ec50 0x7f348c06c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.976+0000 7f34943c7640 1 --2- 172.21.15.22:0/229425815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f348c162b90 0x7f348c06e710 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.976+0000 7f34943c7640 1 --2- 172.21.15.22:0/229425815 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f348c147e10 0x7f348c071740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.976+0000 7f34943c7640 1 -- 172.21.15.22:0/229425815 >> 172.21.15.22:0/229425815 conn(0x7f348c0cfee0 msgr2=0x7f348c0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:01.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.976+0000 7f34943c7640 1 -- 172.21.15.22:0/229425815 shutdown_connections 2024-03-29T04:18:01.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.976+0000 7f34943c7640 1 -- 172.21.15.22:0/229425815 wait complete. 2024-03-29T04:18:01.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f34943c7640 1 Processor -- start 2024-03-29T04:18:01.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f34943c7640 1 -- start start 2024-03-29T04:18:01.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f34943c7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f348c06ec50 0x7f348c165420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:01.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f34943c7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f348c147e10 0x7f348c143580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:01.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f34943c7640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f348c162b90 0x7f348c167890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:01.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f34943c7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f348c0adc70 con 0x7f348c147e10 2024-03-29T04:18:01.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f34943c7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f348c0adaf0 con 0x7f348c06ec50 2024-03-29T04:18:01.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f34943c7640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f348c0addf0 con 0x7f348c162b90 2024-03-29T04:18:01.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f3491962640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f348c147e10 0x7f348c143580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:01.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f3491962640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f348c147e10 0x7f348c143580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51864/0 (socket says 172.21.15.22:51864) 2024-03-29T04:18:01.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f3491962640 1 -- 172.21.15.22:0/407687086 learned_addr learned my addr 172.21.15.22:0/407687086 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:01.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f3492163640 1 --2- 172.21.15.22:0/407687086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f348c06ec50 0x7f348c165420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:01.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f3492964640 1 --2- 172.21.15.22:0/407687086 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f348c162b90 0x7f348c167890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f3491962640 1 -- 172.21.15.22:0/407687086 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f348c162b90 msgr2=0x7f348c167890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f3491962640 1 --2- 172.21.15.22:0/407687086 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f348c162b90 0x7f348c167890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f3491962640 1 -- 172.21.15.22:0/407687086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f348c06ec50 msgr2=0x7f348c165420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f3491962640 1 --2- 172.21.15.22:0/407687086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f348c06ec50 0x7f348c165420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f3491962640 1 -- 172.21.15.22:0/407687086 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348c143ef0 con 0x7f348c147e10 2024-03-29T04:18:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f3492163640 1 --2- 172.21.15.22:0/407687086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f348c06ec50 0x7f348c165420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f3491962640 1 --2- 172.21.15.22:0/407687086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f348c147e10 0x7f348c143580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f347c01cd30 tx=0x7f347c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f3482ffd640 1 -- 172.21.15.22:0/407687086 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c0ba020 con 0x7f348c147e10 2024-03-29T04:18:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f3482ffd640 1 -- 172.21.15.22:0/407687086 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f347c0b6420 con 0x7f348c147e10 2024-03-29T04:18:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f34943c7640 1 -- 172.21.15.22:0/407687086 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f348c1441d0 con 0x7f348c147e10 2024-03-29T04:18:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f3482ffd640 1 -- 172.21.15.22:0/407687086 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c0b0040 con 0x7f348c147e10 2024-03-29T04:18:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f34943c7640 1 -- 172.21.15.22:0/407687086 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f348c167370 con 0x7f348c147e10 2024-03-29T04:18:01.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f34943c7640 1 -- 172.21.15.22:0/407687086 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f348c068890 con 0x7f348c147e10 2024-03-29T04:18:01.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.980+0000 7f3482ffd640 1 -- 172.21.15.22:0/407687086 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f347c0bd020 con 0x7f348c147e10 2024-03-29T04:18:01.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.984+0000 7f3482ffd640 1 --2- 172.21.15.22:0/407687086 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3470067fc0 0x7f347006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:01.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.984+0000 7f3482ffd640 1 -- 172.21.15.22:0/407687086 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f347c120a30 con 0x7f348c147e10 2024-03-29T04:18:01.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.984+0000 7f3492163640 1 --2- 172.21.15.22:0/407687086 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3470067fc0 0x7f347006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:01.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.984+0000 7f3492163640 1 --2- 172.21.15.22:0/407687086 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3470067fc0 0x7f347006a480 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3488009840 tx=0x7f3488008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:01.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:01.988+0000 7f3482ffd640 1 -- 172.21.15.22:0/407687086 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f347c0c4030 con 0x7f348c147e10 2024-03-29T04:18:02.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.144+0000 7f34943c7640 1 -- 172.21.15.22:0/407687086 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f348c1617f0 con 0x7f348c147e10 2024-03-29T04:18:02.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.144+0000 7f3482ffd640 1 -- 172.21.15.22:0/407687086 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18514 (secure 0 0 0) 0x7f347c016110 con 0x7f348c147e10 2024-03-29T04:18:02.127 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:02.127 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:17:42.645632+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:02.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.148+0000 7f34943c7640 1 -- 172.21.15.22:0/407687086 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3470067fc0 msgr2=0x7f347006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:02.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.148+0000 7f34943c7640 1 --2- 172.21.15.22:0/407687086 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3470067fc0 0x7f347006a480 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3488009840 tx=0x7f3488008040 comp rx=0 tx=0).stop 2024-03-29T04:18:02.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.148+0000 7f34943c7640 1 -- 172.21.15.22:0/407687086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f348c147e10 msgr2=0x7f348c143580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:02.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.148+0000 7f34943c7640 1 --2- 172.21.15.22:0/407687086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f348c147e10 0x7f348c143580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f347c01cd30 tx=0x7f347c01acd0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.148+0000 7f34943c7640 1 -- 172.21.15.22:0/407687086 shutdown_connections 2024-03-29T04:18:02.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.148+0000 7f34943c7640 1 --2- 172.21.15.22:0/407687086 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f348c162b90 0x7f348c167890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.148+0000 7f34943c7640 1 --2- 172.21.15.22:0/407687086 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3470067fc0 0x7f347006a480 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.148+0000 7f34943c7640 1 --2- 172.21.15.22:0/407687086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f348c147e10 0x7f348c143580 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.148+0000 7f34943c7640 1 --2- 172.21.15.22:0/407687086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f348c06ec50 0x7f348c165420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.148+0000 7f34943c7640 1 -- 172.21.15.22:0/407687086 >> 172.21.15.22:0/407687086 conn(0x7f348c0cfee0 msgr2=0x7f348c0d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:02.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.148+0000 7f34943c7640 1 -- 172.21.15.22:0/407687086 shutdown_connections 2024-03-29T04:18:02.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.148+0000 7f34943c7640 1 -- 172.21.15.22:0/407687086 wait complete. 2024-03-29T04:18:02.145 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:17:34.034142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:17:34.886659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:17:38.628632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:18:02.146 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:18:02.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.276+0000 7fd454411640 1 Processor -- start 2024-03-29T04:18:02.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.276+0000 7fd454411640 1 -- start start 2024-03-29T04:18:02.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.280+0000 7fd454411640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd44c15e4c0 0x7fd44c0b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:02.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.280+0000 7fd454411640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c0cf6c0 0x7fd44c0d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:02.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.280+0000 7fd454411640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd44c06c850 con 0x7fd44c06c070 2024-03-29T04:18:02.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.280+0000 7fd454411640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fd44c15e8a0 con 0x7fd44c15e4c0 2024-03-29T04:18:02.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.280+0000 7fd454411640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd44c06cb30 con 0x7fd44c0cf6c0 2024-03-29T04:18:02.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.280+0000 7fd4521ad640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd44c15e4c0 0x7fd44c0b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:02.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.280+0000 7fd4521ad640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd44c15e4c0 0x7fd44c0b9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51868/0 (socket says 172.21.15.22:51868) 2024-03-29T04:18:02.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.280+0000 7fd4521ad640 1 -- 172.21.15.22:0/1854930912 learned_addr learned my addr 172.21.15.22:0/1854930912 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:02.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.280+0000 7fd4529ae640 1 --2- 172.21.15.22:0/1854930912 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c0cf6c0 0x7fd44c0d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:02.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.280+0000 7fd4521ad640 1 -- 172.21.15.22:0/1854930912 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c0cf6c0 msgr2=0x7fd44c0d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:02.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.280+0000 7fd4521ad640 1 --2- 172.21.15.22:0/1854930912 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c0cf6c0 0x7fd44c0d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.280+0000 7fd4521ad640 1 -- 172.21.15.22:0/1854930912 >> v1:172.21.15.151:6789/0 conn(0x7fd44c06c070 legacy=0x7fd44c0cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:02.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.280+0000 7fd4521ad640 1 -- 172.21.15.22:0/1854930912 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44c06ce10 con 0x7fd44c15e4c0 2024-03-29T04:18:02.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.280+0000 7fd4529ae640 1 --2- 172.21.15.22:0/1854930912 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c0cf6c0 0x7fd44c0d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:02.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.280+0000 7fd4521ad640 1 --2- 172.21.15.22:0/1854930912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c15e4c0 0x7fd44c0b9e30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009870 tx=0x7fd43c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582aa631cf61017d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:02.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.280+0000 7fd4511ab640 1 -- 172.21.15.22:0/1854930912 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0b0070 con 0x7fd44c15e4c0 2024-03-29T04:18:02.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.280+0000 7fd4511ab640 1 -- 172.21.15.22:0/1854930912 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd43c0ab600 con 0x7fd44c15e4c0 2024-03-29T04:18:02.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.280+0000 7fd4511ab640 1 -- 172.21.15.22:0/1854930912 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0ab9d0 con 0x7fd44c15e4c0 2024-03-29T04:18:02.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd454411640 1 -- 172.21.15.22:0/1854930912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c15e4c0 msgr2=0x7fd44c0b9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:02.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd454411640 1 --2- 172.21.15.22:0/1854930912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c15e4c0 0x7fd44c0b9e30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009870 tx=0x7fd43c01c520 comp rx=0 tx=0).stop 2024-03-29T04:18:02.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd454411640 1 -- 172.21.15.22:0/1854930912 shutdown_connections 2024-03-29T04:18:02.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd454411640 1 --2- 172.21.15.22:0/1854930912 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c0cf6c0 0x7fd44c0d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd454411640 1 --2- 172.21.15.22:0/1854930912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c15e4c0 0x7fd44c0b9e30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd454411640 1 -- 172.21.15.22:0/1854930912 >> 172.21.15.22:0/1854930912 conn(0x7fd44c0b6410 msgr2=0x7fd44c0d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:02.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd454411640 1 -- 172.21.15.22:0/1854930912 shutdown_connections 2024-03-29T04:18:02.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd454411640 1 -- 172.21.15.22:0/1854930912 wait complete. 2024-03-29T04:18:02.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd454411640 1 Processor -- start 2024-03-29T04:18:02.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd454411640 1 -- start start 2024-03-29T04:18:02.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd454411640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c06c070 0x7fd44c0ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:02.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd454411640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd44c0cf6c0 0x7fd44c158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:02.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd454411640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c15e4c0 0x7fd44c0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:02.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd454411640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c0d3ed0 con 0x7fd44c06c070 2024-03-29T04:18:02.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd454411640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c0d3d50 con 0x7fd44c0cf6c0 2024-03-29T04:18:02.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd454411640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd44c0d4050 con 0x7fd44c15e4c0 2024-03-29T04:18:02.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd4521ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c06c070 0x7fd44c0ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:02.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd4521ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c06c070 0x7fd44c0ca250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51884/0 (socket says 172.21.15.22:51884) 2024-03-29T04:18:02.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd4521ad640 1 -- 172.21.15.22:0/3984083834 learned_addr learned my addr 172.21.15.22:0/3984083834 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:02.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd4521ad640 1 -- 172.21.15.22:0/3984083834 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c15e4c0 msgr2=0x7fd44c0cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:02.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd4529ae640 1 --2- 172.21.15.22:0/3984083834 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c15e4c0 0x7fd44c0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:02.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd4519ac640 1 --2- 172.21.15.22:0/3984083834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd44c0cf6c0 0x7fd44c158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:02.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd4521ad640 1 --2- 172.21.15.22:0/3984083834 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c15e4c0 0x7fd44c0cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd4521ad640 1 -- 172.21.15.22:0/3984083834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd44c0cf6c0 msgr2=0x7fd44c158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:02.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd4521ad640 1 --2- 172.21.15.22:0/3984083834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd44c0cf6c0 0x7fd44c158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd4521ad640 1 -- 172.21.15.22:0/3984083834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44c0ca790 con 0x7fd44c06c070 2024-03-29T04:18:02.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd4529ae640 1 --2- 172.21.15.22:0/3984083834 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c15e4c0 0x7fd44c0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:02.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd4521ad640 1 --2- 172.21.15.22:0/3984083834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c06c070 0x7fd44c0ca250 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0ae040 tx=0x7fd43c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:02.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd442ffd640 1 -- 172.21.15.22:0/3984083834 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0b0040 con 0x7fd44c06c070 2024-03-29T04:18:02.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd442ffd640 1 -- 172.21.15.22:0/3984083834 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd43c0b9070 con 0x7fd44c06c070 2024-03-29T04:18:02.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd454411640 1 -- 172.21.15.22:0/3984083834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd44c0c9490 con 0x7fd44c06c070 2024-03-29T04:18:02.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd442ffd640 1 -- 172.21.15.22:0/3984083834 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0aa960 con 0x7fd44c06c070 2024-03-29T04:18:02.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd454411640 1 -- 172.21.15.22:0/3984083834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd44c0c9a50 con 0x7fd44c06c070 2024-03-29T04:18:02.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.284+0000 7fd454411640 1 -- 172.21.15.22:0/3984083834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd44c068900 con 0x7fd44c06c070 2024-03-29T04:18:02.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.288+0000 7fd442ffd640 1 -- 172.21.15.22:0/3984083834 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fd43c0aab00 con 0x7fd44c06c070 2024-03-29T04:18:02.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.288+0000 7fd442ffd640 1 --2- 172.21.15.22:0/3984083834 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd430067fc0 0x7fd43006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:02.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.288+0000 7fd442ffd640 1 -- 172.21.15.22:0/3984083834 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd43c1208a0 con 0x7fd44c06c070 2024-03-29T04:18:02.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.288+0000 7fd4519ac640 1 --2- 172.21.15.22:0/3984083834 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd430067fc0 0x7fd43006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:02.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.288+0000 7fd4519ac640 1 --2- 172.21.15.22:0/3984083834 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd430067fc0 0x7fd43006a480 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0b8020 tx=0x7fd448008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:02.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.292+0000 7fd442ffd640 1 -- 172.21.15.22:0/3984083834 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fd43c016110 con 0x7fd44c06c070 2024-03-29T04:18:02.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.448+0000 7fd454411640 1 -- 172.21.15.22:0/3984083834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd44c168df0 con 0x7fd44c06c070 2024-03-29T04:18:02.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.452+0000 7fd442ffd640 1 -- 172.21.15.22:0/3984083834 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fd43c0f0da0 con 0x7fd44c06c070 2024-03-29T04:18:02.432 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:18:02.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.452+0000 7fd454411640 1 -- 172.21.15.22:0/3984083834 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd430067fc0 msgr2=0x7fd43006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:02.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.452+0000 7fd454411640 1 --2- 172.21.15.22:0/3984083834 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd430067fc0 0x7fd43006a480 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0b8020 tx=0x7fd448008040 comp rx=0 tx=0).stop 2024-03-29T04:18:02.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.452+0000 7fd454411640 1 -- 172.21.15.22:0/3984083834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c06c070 msgr2=0x7fd44c0ca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:02.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.452+0000 7fd454411640 1 --2- 172.21.15.22:0/3984083834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c06c070 0x7fd44c0ca250 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0ae040 tx=0x7fd43c0192c0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.452+0000 7fd454411640 1 -- 172.21.15.22:0/3984083834 shutdown_connections 2024-03-29T04:18:02.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.452+0000 7fd454411640 1 --2- 172.21.15.22:0/3984083834 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd44c15e4c0 0x7fd44c0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.452+0000 7fd454411640 1 --2- 172.21.15.22:0/3984083834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd44c0cf6c0 0x7fd44c158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.452+0000 7fd454411640 1 --2- 172.21.15.22:0/3984083834 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd430067fc0 0x7fd43006a480 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.452+0000 7fd454411640 1 --2- 172.21.15.22:0/3984083834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c06c070 0x7fd44c0ca250 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.452+0000 7fd454411640 1 -- 172.21.15.22:0/3984083834 >> 172.21.15.22:0/3984083834 conn(0x7fd44c0b6410 msgr2=0x7fd44c0b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:02.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.452+0000 7fd454411640 1 -- 172.21.15.22:0/3984083834 shutdown_connections 2024-03-29T04:18:02.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.452+0000 7fd454411640 1 -- 172.21.15.22:0/3984083834 wait complete. 2024-03-29T04:18:02.451 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:18:02.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.588+0000 7f7dc6eb5640 1 Processor -- start 2024-03-29T04:18:02.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.588+0000 7f7dc6eb5640 1 -- start start 2024-03-29T04:18:02.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.588+0000 7f7dc6eb5640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f7dc0144930 0x7f7dc00b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:02.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.588+0000 7f7dc6eb5640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7dc006cd80 0x7f7dc00ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:02.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.588+0000 7f7dc6eb5640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dc00b33c0 con 0x7f7dc00b4870 2024-03-29T04:18:02.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.588+0000 7f7dc6eb5640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f7dc00cdb90 con 0x7f7dc006cd80 2024-03-29T04:18:02.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.588+0000 7f7dc6eb5640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f7dc00b3660 con 0x7f7dc0144930 2024-03-29T04:18:02.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.588+0000 7f7dc5452640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7dc006cd80 0x7f7dc00ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:02.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.588+0000 7f7dc5452640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7dc006cd80 0x7f7dc00ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51890/0 (socket says 172.21.15.22:51890) 2024-03-29T04:18:02.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.588+0000 7f7dc5452640 1 -- 172.21.15.22:0/2277244743 learned_addr learned my addr 172.21.15.22:0/2277244743 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:02.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.588+0000 7f7dc4c51640 1 --2- 172.21.15.22:0/2277244743 >> v2:172.21.15.151:3300/0 conn(0x7f7dc0144930 0x7f7dc00b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:02.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.588+0000 7f7dc5452640 1 -- 172.21.15.22:0/2277244743 >> v2:172.21.15.151:3300/0 conn(0x7f7dc0144930 msgr2=0x7f7dc00b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:02.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.588+0000 7f7dc5452640 1 --2- 172.21.15.22:0/2277244743 >> v2:172.21.15.151:3300/0 conn(0x7f7dc0144930 0x7f7dc00b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.588+0000 7f7dc5452640 1 -- 172.21.15.22:0/2277244743 >> v1:172.21.15.151:6789/0 conn(0x7f7dc00b4870 legacy=0x7f7dc006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:02.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.588+0000 7f7dc5452640 1 -- 172.21.15.22:0/2277244743 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc00b3940 con 0x7f7dc006cd80 2024-03-29T04:18:02.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.588+0000 7f7dc5452640 1 --2- 172.21.15.22:0/2277244743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7dc006cd80 0x7f7dc00ce020 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00cdd10 tx=0x7f7db801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56288b6ea713386a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:02.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dbf7fe640 1 -- 172.21.15.22:0/2277244743 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db80a57b0 con 0x7f7dc006cd80 2024-03-29T04:18:02.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dbf7fe640 1 -- 172.21.15.22:0/2277244743 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db80a5950 con 0x7f7dc006cd80 2024-03-29T04:18:02.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dbf7fe640 1 -- 172.21.15.22:0/2277244743 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db80a5d20 con 0x7f7dc006cd80 2024-03-29T04:18:02.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc6eb5640 1 -- 172.21.15.22:0/2277244743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7dc006cd80 msgr2=0x7f7dc00ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:02.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc6eb5640 1 --2- 172.21.15.22:0/2277244743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7dc006cd80 0x7f7dc00ce020 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00cdd10 tx=0x7f7db801c930 comp rx=0 tx=0).stop 2024-03-29T04:18:02.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc6eb5640 1 -- 172.21.15.22:0/2277244743 shutdown_connections 2024-03-29T04:18:02.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc6eb5640 1 --2- 172.21.15.22:0/2277244743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7dc006cd80 0x7f7dc00ce020 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc6eb5640 1 --2- 172.21.15.22:0/2277244743 >> v2:172.21.15.151:3300/0 conn(0x7f7dc0144930 0x7f7dc00b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc6eb5640 1 -- 172.21.15.22:0/2277244743 >> 172.21.15.22:0/2277244743 conn(0x7f7dc00d00a0 msgr2=0x7f7dc00d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:02.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc6eb5640 1 -- 172.21.15.22:0/2277244743 shutdown_connections 2024-03-29T04:18:02.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc6eb5640 1 -- 172.21.15.22:0/2277244743 wait complete. 2024-03-29T04:18:02.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc6eb5640 1 Processor -- start 2024-03-29T04:18:02.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc6eb5640 1 -- start start 2024-03-29T04:18:02.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc6eb5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7dc006cd80 0x7f7dc015ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:02.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc6eb5640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7dc00b4870 0x7f7dc013c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:02.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc6eb5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7dc0144930 0x7f7dc013cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:02.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc6eb5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc00ceef0 con 0x7f7dc006cd80 2024-03-29T04:18:02.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc6eb5640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc00ced70 con 0x7f7dc0144930 2024-03-29T04:18:02.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc6eb5640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7dc00cf070 con 0x7f7dc00b4870 2024-03-29T04:18:02.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc4c51640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7dc006cd80 0x7f7dc015ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:02.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc4c51640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7dc006cd80 0x7f7dc015ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51906/0 (socket says 172.21.15.22:51906) 2024-03-29T04:18:02.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc4c51640 1 -- 172.21.15.22:0/1040766507 learned_addr learned my addr 172.21.15.22:0/1040766507 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:02.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc5452640 1 --2- 172.21.15.22:0/1040766507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7dc0144930 0x7f7dc013cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:02.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dbffff640 1 --2- 172.21.15.22:0/1040766507 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7dc00b4870 0x7f7dc013c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:02.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc4c51640 1 -- 172.21.15.22:0/1040766507 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7dc00b4870 msgr2=0x7f7dc013c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:02.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc4c51640 1 --2- 172.21.15.22:0/1040766507 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7dc00b4870 0x7f7dc013c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc4c51640 1 -- 172.21.15.22:0/1040766507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7dc0144930 msgr2=0x7f7dc013cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:02.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc4c51640 1 --2- 172.21.15.22:0/1040766507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7dc0144930 0x7f7dc013cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc4c51640 1 -- 172.21.15.22:0/1040766507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc00b02f0 con 0x7f7dc006cd80 2024-03-29T04:18:02.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc4c51640 1 --2- 172.21.15.22:0/1040766507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7dc006cd80 0x7f7dc015ba80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7db000d5f0 tx=0x7f7db000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:02.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dbd7fa640 1 -- 172.21.15.22:0/1040766507 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db00125f0 con 0x7f7dc006cd80 2024-03-29T04:18:02.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dbd7fa640 1 -- 172.21.15.22:0/1040766507 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db0019070 con 0x7f7dc006cd80 2024-03-29T04:18:02.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc6eb5640 1 -- 172.21.15.22:0/1040766507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc00b05d0 con 0x7f7dc006cd80 2024-03-29T04:18:02.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dbd7fa640 1 -- 172.21.15.22:0/1040766507 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db00129d0 con 0x7f7dc006cd80 2024-03-29T04:18:02.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.592+0000 7f7dc6eb5640 1 -- 172.21.15.22:0/1040766507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc00b0b90 con 0x7f7dc006cd80 2024-03-29T04:18:02.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.596+0000 7f7dc6eb5640 1 -- 172.21.15.22:0/1040766507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dc0068900 con 0x7f7dc006cd80 2024-03-29T04:18:02.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.596+0000 7f7dbd7fa640 1 -- 172.21.15.22:0/1040766507 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f7db0012b70 con 0x7f7dc006cd80 2024-03-29T04:18:02.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.596+0000 7f7dbd7fa640 1 --2- 172.21.15.22:0/1040766507 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7dac067fc0 0x7f7dac06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:02.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.596+0000 7f7dbd7fa640 1 -- 172.21.15.22:0/1040766507 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7db0086d00 con 0x7f7dc006cd80 2024-03-29T04:18:02.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.596+0000 7f7dbffff640 1 --2- 172.21.15.22:0/1040766507 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7dac067fc0 0x7f7dac06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:02.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.596+0000 7f7dbffff640 1 --2- 172.21.15.22:0/1040766507 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7dac067fc0 0x7f7dac06a480 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00d0580 tx=0x7f7db4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:02.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.600+0000 7f7dbd7fa640 1 -- 172.21.15.22:0/1040766507 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f7db0057200 con 0x7f7dc006cd80 2024-03-29T04:18:02.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.760+0000 7f7dc6eb5640 1 -- 172.21.15.22:0/1040766507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7dc00af090 con 0x7f7dc006cd80 2024-03-29T04:18:02.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.760+0000 7f7dbd7fa640 1 -- 172.21.15.22:0/1040766507 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f7db0056b00 con 0x7f7dc006cd80 2024-03-29T04:18:02.742 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:02.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.764+0000 7f7dc6eb5640 1 -- 172.21.15.22:0/1040766507 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7dac067fc0 msgr2=0x7f7dac06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:02.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.764+0000 7f7dc6eb5640 1 --2- 172.21.15.22:0/1040766507 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7dac067fc0 0x7f7dac06a480 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00d0580 tx=0x7f7db4009040 comp rx=0 tx=0).stop 2024-03-29T04:18:02.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.764+0000 7f7dc6eb5640 1 -- 172.21.15.22:0/1040766507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7dc006cd80 msgr2=0x7f7dc015ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:02.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.764+0000 7f7dc6eb5640 1 --2- 172.21.15.22:0/1040766507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7dc006cd80 0x7f7dc015ba80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7db000d5f0 tx=0x7f7db000dab0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.764+0000 7f7dc6eb5640 1 -- 172.21.15.22:0/1040766507 shutdown_connections 2024-03-29T04:18:02.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.764+0000 7f7dc6eb5640 1 --2- 172.21.15.22:0/1040766507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7dc0144930 0x7f7dc013cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.764+0000 7f7dc6eb5640 1 --2- 172.21.15.22:0/1040766507 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7dc00b4870 0x7f7dc013c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.764+0000 7f7dc6eb5640 1 --2- 172.21.15.22:0/1040766507 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7dac067fc0 0x7f7dac06a480 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.764+0000 7f7dc6eb5640 1 --2- 172.21.15.22:0/1040766507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7dc006cd80 0x7f7dc015ba80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.764+0000 7f7dc6eb5640 1 -- 172.21.15.22:0/1040766507 >> 172.21.15.22:0/1040766507 conn(0x7f7dc00d00a0 msgr2=0x7f7dc0143b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:02.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.764+0000 7f7dc6eb5640 1 -- 172.21.15.22:0/1040766507 shutdown_connections 2024-03-29T04:18:02.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.764+0000 7f7dc6eb5640 1 -- 172.21.15.22:0/1040766507 wait complete. 2024-03-29T04:18:02.762 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:18:02.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.896+0000 7fa669d15640 1 Processor -- start 2024-03-29T04:18:02.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.896+0000 7fa669d15640 1 -- start start 2024-03-29T04:18:02.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa669d15640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa6640cf870 0x7fa66415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:02.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa669d15640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa664071910 con 0x7fa6640726d0 2024-03-29T04:18:02.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa669d15640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa664071bf0 con 0x7fa6640b4090 2024-03-29T04:18:02.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa669d15640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fa664071ed0 con 0x7fa6640cf870 2024-03-29T04:18:02.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa6637fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa6640cf870 0x7fa66415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:02.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa662ffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fa6640726d0 0x7fa6640b3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44372/0 (socket says 172.21.15.22:44372) 2024-03-29T04:18:02.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa662ffd640 1 -- 172.21.15.22:0/630432617 learned_addr learned my addr 172.21.15.22:0/630432617 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:02.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa6637fe640 1 -- 172.21.15.22:0/630432617 >> v1:172.21.15.151:6789/0 conn(0x7fa6640b4090 legacy=0x7fa6640b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:02.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa6637fe640 1 -- 172.21.15.22:0/630432617 >> v1:172.21.15.22:6789/0 conn(0x7fa6640726d0 legacy=0x7fa6640b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:02.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa6627fc640 1 -- 172.21.15.22:0/630432617 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 669528266 0 0) 0x7fa664071910 con 0x7fa6640726d0 2024-03-29T04:18:02.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa6637fe640 1 -- 172.21.15.22:0/630432617 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6640721b0 con 0x7fa6640cf870 2024-03-29T04:18:02.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa6637fe640 1 --2- 172.21.15.22:0/630432617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6640cf870 0x7fa66415baa0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa65c0097c0 tx=0x7fa65c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c222c079e7e79b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:02.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa6627fc640 1 -- 172.21.15.22:0/630432617 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c0ab680 con 0x7fa6640cf870 2024-03-29T04:18:02.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa6627fc640 1 -- 172.21.15.22:0/630432617 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa65c0b0070 con 0x7fa6640cf870 2024-03-29T04:18:02.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa6627fc640 1 -- 172.21.15.22:0/630432617 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c0aba40 con 0x7fa6640cf870 2024-03-29T04:18:02.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa669d15640 1 -- 172.21.15.22:0/630432617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6640cf870 msgr2=0x7fa66415baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:02.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa669d15640 1 --2- 172.21.15.22:0/630432617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6640cf870 0x7fa66415baa0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa65c0097c0 tx=0x7fa65c01c460 comp rx=0 tx=0).stop 2024-03-29T04:18:02.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa669d15640 1 -- 172.21.15.22:0/630432617 shutdown_connections 2024-03-29T04:18:02.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa669d15640 1 --2- 172.21.15.22:0/630432617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6640cf870 0x7fa66415baa0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa669d15640 1 -- 172.21.15.22:0/630432617 >> 172.21.15.22:0/630432617 conn(0x7fa6640d00a0 msgr2=0x7fa6640d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:02.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa669d15640 1 -- 172.21.15.22:0/630432617 shutdown_connections 2024-03-29T04:18:02.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.900+0000 7fa669d15640 1 -- 172.21.15.22:0/630432617 wait complete. 2024-03-29T04:18:02.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa669d15640 1 Processor -- start 2024-03-29T04:18:02.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa669d15640 1 -- start start 2024-03-29T04:18:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa669d15640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6640726d0 0x7fa664140010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa669d15640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6640b4090 0x7fa6640b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa669d15640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa6640cf870 0x7fa6640b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa669d15640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa66406e290 con 0x7fa6640726d0 2024-03-29T04:18:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa669d15640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa66406e110 con 0x7fa6640b4090 2024-03-29T04:18:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa669d15640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa66406c670 con 0x7fa6640cf870 2024-03-29T04:18:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa6637fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6640726d0 0x7fa664140010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa6637fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6640726d0 0x7fa664140010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51920/0 (socket says 172.21.15.22:51920) 2024-03-29T04:18:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa6637fe640 1 -- 172.21.15.22:0/4155246993 learned_addr learned my addr 172.21.15.22:0/4155246993 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa663fff640 1 --2- 172.21.15.22:0/4155246993 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa6640cf870 0x7fa6640b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa662ffd640 1 --2- 172.21.15.22:0/4155246993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6640b4090 0x7fa6640b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa6637fe640 1 -- 172.21.15.22:0/4155246993 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa6640cf870 msgr2=0x7fa6640b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa6637fe640 1 --2- 172.21.15.22:0/4155246993 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa6640cf870 0x7fa6640b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa6637fe640 1 -- 172.21.15.22:0/4155246993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6640b4090 msgr2=0x7fa6640b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:02.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa6637fe640 1 --2- 172.21.15.22:0/4155246993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6640b4090 0x7fa6640b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:02.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa6637fe640 1 -- 172.21.15.22:0/4155246993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa664140550 con 0x7fa6640726d0 2024-03-29T04:18:02.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa662ffd640 1 --2- 172.21.15.22:0/4155246993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6640b4090 0x7fa6640b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:02.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa6637fe640 1 --2- 172.21.15.22:0/4155246993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6640726d0 0x7fa664140010 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa65c019070 tx=0x7fa65c019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:02.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa65bfff640 1 -- 172.21.15.22:0/4155246993 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c004230 con 0x7fa6640726d0 2024-03-29T04:18:02.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa65bfff640 1 -- 172.21.15.22:0/4155246993 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa65c0a7070 con 0x7fa6640726d0 2024-03-29T04:18:02.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa65bfff640 1 -- 172.21.15.22:0/4155246993 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c003620 con 0x7fa6640726d0 2024-03-29T04:18:02.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa669d15640 1 -- 172.21.15.22:0/4155246993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa664148120 con 0x7fa6640726d0 2024-03-29T04:18:02.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa669d15640 1 -- 172.21.15.22:0/4155246993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6641486e0 con 0x7fa6640726d0 2024-03-29T04:18:02.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa669d15640 1 -- 172.21.15.22:0/4155246993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa664068900 con 0x7fa6640726d0 2024-03-29T04:18:02.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.904+0000 7fa65bfff640 1 -- 172.21.15.22:0/4155246993 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fa65c0037c0 con 0x7fa6640726d0 2024-03-29T04:18:02.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.908+0000 7fa65bfff640 1 --2- 172.21.15.22:0/4155246993 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa644067fc0 0x7fa64406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:02.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.908+0000 7fa65bfff640 1 -- 172.21.15.22:0/4155246993 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa65c1219b0 con 0x7fa6640726d0 2024-03-29T04:18:02.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.908+0000 7fa662ffd640 1 --2- 172.21.15.22:0/4155246993 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa644067fc0 0x7fa64406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:02.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.908+0000 7fa662ffd640 1 --2- 172.21.15.22:0/4155246993 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa644067fc0 0x7fa64406a480 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa6641f8000 tx=0x7fa64c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:02.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:02.908+0000 7fa65bfff640 1 -- 172.21.15.22:0/4155246993 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fa65c0cd080 con 0x7fa6640726d0 2024-03-29T04:18:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.064+0000 7fa669d15640 1 -- 172.21.15.22:0/4155246993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa66414c820 con 0x7fa6640726d0 2024-03-29T04:18:03.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.064+0000 7fa65bfff640 1 -- 172.21.15.22:0/4155246993 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fa65c016050 con 0x7fa6640726d0 2024-03-29T04:18:03.046 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-29T04:18:03.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.068+0000 7fa669d15640 1 -- 172.21.15.22:0/4155246993 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa644067fc0 msgr2=0x7fa64406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:03.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.068+0000 7fa669d15640 1 --2- 172.21.15.22:0/4155246993 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa644067fc0 0x7fa64406a480 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa6641f8000 tx=0x7fa64c009040 comp rx=0 tx=0).stop 2024-03-29T04:18:03.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.068+0000 7fa669d15640 1 -- 172.21.15.22:0/4155246993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6640726d0 msgr2=0x7fa664140010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:03.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.068+0000 7fa669d15640 1 --2- 172.21.15.22:0/4155246993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6640726d0 0x7fa664140010 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa65c019070 tx=0x7fa65c019200 comp rx=0 tx=0).stop 2024-03-29T04:18:03.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.068+0000 7fa669d15640 1 -- 172.21.15.22:0/4155246993 shutdown_connections 2024-03-29T04:18:03.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.068+0000 7fa669d15640 1 --2- 172.21.15.22:0/4155246993 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa6640cf870 0x7fa6640b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.068+0000 7fa669d15640 1 --2- 172.21.15.22:0/4155246993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6640b4090 0x7fa6640b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.068+0000 7fa669d15640 1 --2- 172.21.15.22:0/4155246993 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa644067fc0 0x7fa64406a480 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.068+0000 7fa669d15640 1 --2- 172.21.15.22:0/4155246993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6640726d0 0x7fa664140010 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.068+0000 7fa669d15640 1 -- 172.21.15.22:0/4155246993 >> 172.21.15.22:0/4155246993 conn(0x7fa6640d00a0 msgr2=0x7fa6641505e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:03.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.068+0000 7fa669d15640 1 -- 172.21.15.22:0/4155246993 shutdown_connections 2024-03-29T04:18:03.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.068+0000 7fa669d15640 1 -- 172.21.15.22:0/4155246993 wait complete. 2024-03-29T04:18:03.064 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:18:03.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f895f9bb640 1 Processor -- start 2024-03-29T04:18:03.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f895f9bb640 1 -- start start 2024-03-29T04:18:03.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f895f9bb640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f89580b7d40 0x7f89580cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:03.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f895f9bb640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89580b50f0 con 0x7f8958161c50 2024-03-29T04:18:03.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f895f9bb640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89580b53d0 con 0x7f89580cf720 2024-03-29T04:18:03.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f895f9bb640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f89580b78b0 con 0x7f89580b7d40 2024-03-29T04:18:03.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f895d757640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f8958161c50 0x7f89580ce120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44384/0 (socket says 172.21.15.22:44384) 2024-03-29T04:18:03.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f895d757640 1 -- 172.21.15.22:0/847010741 learned_addr learned my addr 172.21.15.22:0/847010741 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:03.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f895cf56640 1 --2- 172.21.15.22:0/847010741 >> v2:172.21.15.151:3300/0 conn(0x7f89580b7d40 0x7f89580cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:03.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f894ffff640 1 -- 172.21.15.22:0/847010741 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 573378758 0 0) 0x7f89580b50f0 con 0x7f8958161c50 2024-03-29T04:18:03.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f894ffff640 1 -- 172.21.15.22:0/847010741 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8944003530 con 0x7f8958161c50 2024-03-29T04:18:03.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f894ffff640 1 -- 172.21.15.22:0/847010741 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3361426415 0 0) 0x7f8944003530 con 0x7f8958161c50 2024-03-29T04:18:03.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f894ffff640 1 -- 172.21.15.22:0/847010741 >> v2:172.21.15.151:3300/0 conn(0x7f89580b7d40 msgr2=0x7f89580cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:03.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f894ffff640 1 --2- 172.21.15.22:0/847010741 >> v2:172.21.15.151:3300/0 conn(0x7f89580b7d40 0x7f89580cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f894ffff640 1 -- 172.21.15.22:0/847010741 >> v1:172.21.15.151:6789/0 conn(0x7f89580cf720 legacy=0x7f89580cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:03.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f894ffff640 1 -- 172.21.15.22:0/847010741 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89580b56b0 con 0x7f8958161c50 2024-03-29T04:18:03.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f894ffff640 1 -- 172.21.15.22:0/847010741 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f8954001680 con 0x7f8958161c50 2024-03-29T04:18:03.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f894ffff640 1 -- 172.21.15.22:0/847010741 >> v1:172.21.15.22:6789/0 conn(0x7f8958161c50 legacy=0x7f89580ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:03.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f894ffff640 1 --2- 172.21.15.22:0/847010741 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89440039c0 0x7f8944003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:03.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f894ffff640 1 --2- 172.21.15.22:0/847010741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8944004810 0x7f8944004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:03.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f894ffff640 1 --2- 172.21.15.22:0/847010741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89440052f0 0x7f894400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:03.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f894ffff640 1 -- 172.21.15.22:0/847010741 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f89580b78b0 con 0x7f8944004810 2024-03-29T04:18:03.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f894ffff640 1 -- 172.21.15.22:0/847010741 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f894400c1c0 con 0x7f89440052f0 2024-03-29T04:18:03.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f894ffff640 1 -- 172.21.15.22:0/847010741 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f894400c3c0 con 0x7f89440039c0 2024-03-29T04:18:03.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f895cf56640 1 --2- 172.21.15.22:0/847010741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8944004810 0x7f8944004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:03.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f895d757640 1 --2- 172.21.15.22:0/847010741 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89440039c0 0x7f8944003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:03.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f895cf56640 1 -- 172.21.15.22:0/847010741 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89440039c0 msgr2=0x7f8944003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:03.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f895cf56640 1 --2- 172.21.15.22:0/847010741 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89440039c0 0x7f8944003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f895cf56640 1 -- 172.21.15.22:0/847010741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89440052f0 msgr2=0x7f894400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:03.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f895cf56640 1 --2- 172.21.15.22:0/847010741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89440052f0 0x7f894400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f895df58640 1 -- 172.21.15.22:0/847010741 reap_dead start 2024-03-29T04:18:03.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.196+0000 7f895cf56640 1 -- 172.21.15.22:0/847010741 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8944003530 con 0x7f8944004810 2024-03-29T04:18:03.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.200+0000 7f895cf56640 1 --2- 172.21.15.22:0/847010741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8944004810 0x7f8944004c90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8948009ab0 tx=0x7f894801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ae8cf4b69999e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:03.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.200+0000 7f894ffff640 1 -- 172.21.15.22:0/847010741 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89480096b0 con 0x7f8944004810 2024-03-29T04:18:03.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.200+0000 7f894ffff640 1 -- 172.21.15.22:0/847010741 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89480a6450 con 0x7f8944004810 2024-03-29T04:18:03.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.200+0000 7f894ffff640 1 -- 172.21.15.22:0/847010741 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89480a6840 con 0x7f8944004810 2024-03-29T04:18:03.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.200+0000 7f895f9bb640 1 -- 172.21.15.22:0/847010741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8944004810 msgr2=0x7f8944004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:03.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.200+0000 7f895f9bb640 1 --2- 172.21.15.22:0/847010741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8944004810 0x7f8944004c90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8948009ab0 tx=0x7f894801ca70 comp rx=0 tx=0).stop 2024-03-29T04:18:03.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.200+0000 7f895f9bb640 1 -- 172.21.15.22:0/847010741 shutdown_connections 2024-03-29T04:18:03.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.200+0000 7f895f9bb640 1 --2- 172.21.15.22:0/847010741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8944004810 0x7f8944004c90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.200+0000 7f895f9bb640 1 -- 172.21.15.22:0/847010741 >> 172.21.15.22:0/847010741 conn(0x7f89580ceae0 msgr2=0x7f89580ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:03.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.200+0000 7f895f9bb640 1 -- 172.21.15.22:0/847010741 shutdown_connections 2024-03-29T04:18:03.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.200+0000 7f895f9bb640 1 -- 172.21.15.22:0/847010741 wait complete. 2024-03-29T04:18:03.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.200+0000 7f895f9bb640 1 Processor -- start 2024-03-29T04:18:03.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.200+0000 7f895f9bb640 1 -- start start 2024-03-29T04:18:03.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f895f9bb640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8944004810 0x7f89580cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:03.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f895f9bb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f895814b750 0x7f895814bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:03.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f895f9bb640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89581ea290 0x7f89581ea710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:03.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f895f9bb640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f89581625e0 con 0x7f895814b750 2024-03-29T04:18:03.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f895f9bb640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8958162460 con 0x7f89581ea290 2024-03-29T04:18:03.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f895f9bb640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8958162760 con 0x7f8944004810 2024-03-29T04:18:03.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f895cf56640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f895814b750 0x7f895814bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:03.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f895cf56640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f895814b750 0x7f895814bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51938/0 (socket says 172.21.15.22:51938) 2024-03-29T04:18:03.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f895cf56640 1 -- 172.21.15.22:0/1349290820 learned_addr learned my addr 172.21.15.22:0/1349290820 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:03.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f895d757640 1 --2- 172.21.15.22:0/1349290820 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8944004810 0x7f89580cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:03.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f895cf56640 1 -- 172.21.15.22:0/1349290820 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8944004810 msgr2=0x7f89580cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:03.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f895cf56640 1 --2- 172.21.15.22:0/1349290820 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8944004810 0x7f89580cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f895cf56640 1 -- 172.21.15.22:0/1349290820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89581ea290 msgr2=0x7f89581ea710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:03.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f895df58640 1 --2- 172.21.15.22:0/1349290820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89581ea290 0x7f89581ea710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:03.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f895cf56640 1 --2- 172.21.15.22:0/1349290820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89581ea290 0x7f89581ea710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f895cf56640 1 -- 172.21.15.22:0/1349290820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895815e8d0 con 0x7f895814b750 2024-03-29T04:18:03.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f895cf56640 1 --2- 172.21.15.22:0/1349290820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f895814b750 0x7f895814bbb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8948009ab0 tx=0x7f8948019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:03.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f894dffb640 1 -- 172.21.15.22:0/1349290820 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89480096b0 con 0x7f895814b750 2024-03-29T04:18:03.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f894dffb640 1 -- 172.21.15.22:0/1349290820 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89480a7630 con 0x7f895814b750 2024-03-29T04:18:03.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f895f9bb640 1 -- 172.21.15.22:0/1349290820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895815db10 con 0x7f895814b750 2024-03-29T04:18:03.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f894dffb640 1 -- 172.21.15.22:0/1349290820 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89480a7a00 con 0x7f895814b750 2024-03-29T04:18:03.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f895f9bb640 1 -- 172.21.15.22:0/1349290820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f895815e0d0 con 0x7f895814b750 2024-03-29T04:18:03.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f895f9bb640 1 -- 172.21.15.22:0/1349290820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8958068900 con 0x7f895814b750 2024-03-29T04:18:03.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f894dffb640 1 -- 172.21.15.22:0/1349290820 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f89480c1060 con 0x7f895814b750 2024-03-29T04:18:03.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f894dffb640 1 --2- 172.21.15.22:0/1349290820 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f893c067fc0 0x7f893c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:03.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f894dffb640 1 -- 172.21.15.22:0/1349290820 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8948120ad0 con 0x7f895814b750 2024-03-29T04:18:03.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.204+0000 7f895d757640 1 --2- 172.21.15.22:0/1349290820 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f893c067fc0 0x7f893c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:03.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.208+0000 7f895d757640 1 --2- 172.21.15.22:0/1349290820 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f893c067fc0 0x7f893c06a480 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8958074a70 tx=0x7f8954009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:03.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.208+0000 7f894dffb640 1 -- 172.21.15.22:0/1349290820 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f89480b5070 con 0x7f895814b750 2024-03-29T04:18:03.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.368+0000 7f895f9bb640 1 -- 172.21.15.22:0/1349290820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f89580c9b70 con 0x7f895814b750 2024-03-29T04:18:03.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.372+0000 7f894dffb640 1 -- 172.21.15.22:0/1349290820 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f89480ac020 con 0x7f895814b750 2024-03-29T04:18:03.352 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:03.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.372+0000 7f895f9bb640 1 -- 172.21.15.22:0/1349290820 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f893c067fc0 msgr2=0x7f893c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:03.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.372+0000 7f895f9bb640 1 --2- 172.21.15.22:0/1349290820 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f893c067fc0 0x7f893c06a480 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8958074a70 tx=0x7f8954009040 comp rx=0 tx=0).stop 2024-03-29T04:18:03.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.372+0000 7f895f9bb640 1 -- 172.21.15.22:0/1349290820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f895814b750 msgr2=0x7f895814bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:03.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.372+0000 7f895f9bb640 1 --2- 172.21.15.22:0/1349290820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f895814b750 0x7f895814bbb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8948009ab0 tx=0x7f8948019810 comp rx=0 tx=0).stop 2024-03-29T04:18:03.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.372+0000 7f895f9bb640 1 -- 172.21.15.22:0/1349290820 shutdown_connections 2024-03-29T04:18:03.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.372+0000 7f895f9bb640 1 --2- 172.21.15.22:0/1349290820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89581ea290 0x7f89581ea710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.372+0000 7f895f9bb640 1 --2- 172.21.15.22:0/1349290820 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f893c067fc0 0x7f893c06a480 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.372+0000 7f895f9bb640 1 --2- 172.21.15.22:0/1349290820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f895814b750 0x7f895814bbb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.372+0000 7f895f9bb640 1 --2- 172.21.15.22:0/1349290820 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8944004810 0x7f89580cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.372+0000 7f895f9bb640 1 -- 172.21.15.22:0/1349290820 >> 172.21.15.22:0/1349290820 conn(0x7f89580ceae0 msgr2=0x7f89580b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:03.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.372+0000 7f895f9bb640 1 -- 172.21.15.22:0/1349290820 shutdown_connections 2024-03-29T04:18:03.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.372+0000 7f895f9bb640 1 -- 172.21.15.22:0/1349290820 wait complete. 2024-03-29T04:18:03.373 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:18:03.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.508+0000 7fedab7b2640 1 Processor -- start 2024-03-29T04:18:03.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.508+0000 7fedab7b2640 1 -- start start 2024-03-29T04:18:03.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.512+0000 7fedab7b2640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7feda406f390 0x7feda41604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:03.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.512+0000 7fedab7b2640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feda40cfd20 0x7feda40d0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:03.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.512+0000 7fedab7b2640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feda415c610 con 0x7feda415c230 2024-03-29T04:18:03.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.512+0000 7fedab7b2640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7feda40b4fa0 con 0x7feda406f390 2024-03-29T04:18:03.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.512+0000 7fedab7b2640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7feda40cf4f0 con 0x7feda40cfd20 2024-03-29T04:18:03.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.512+0000 7feda8d4d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7feda406f390 0x7feda41604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:03.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.512+0000 7feda8d4d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7feda406f390 0x7feda41604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51948/0 (socket says 172.21.15.22:51948) 2024-03-29T04:18:03.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.512+0000 7feda8d4d640 1 -- 172.21.15.22:0/2549024337 learned_addr learned my addr 172.21.15.22:0/2549024337 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:03.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.512+0000 7feda9d4f640 1 --2- 172.21.15.22:0/2549024337 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feda40cfd20 0x7feda40d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:03.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.512+0000 7feda8d4d640 1 -- 172.21.15.22:0/2549024337 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feda40cfd20 msgr2=0x7feda40d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:03.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.512+0000 7feda8d4d640 1 --2- 172.21.15.22:0/2549024337 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feda40cfd20 0x7feda40d0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.512+0000 7feda8d4d640 1 -- 172.21.15.22:0/2549024337 >> v1:172.21.15.22:6789/0 conn(0x7feda415c230 legacy=0x7feda415fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:03.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.512+0000 7fed9bfff640 1 -- 172.21.15.22:0/2549024337 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2007985036 0 0) 0x7feda415c610 con 0x7feda415c230 2024-03-29T04:18:03.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.512+0000 7feda8d4d640 1 -- 172.21.15.22:0/2549024337 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda40cc9f0 con 0x7feda406f390 2024-03-29T04:18:03.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.512+0000 7feda9d4f640 1 --2- 172.21.15.22:0/2549024337 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feda40cfd20 0x7feda40d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:03.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7feda8d4d640 1 --2- 172.21.15.22:0/2549024337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feda406f390 0x7feda41604b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fed94009870 tx=0x7fed9401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8659616592af6d31 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:03.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7fed9bfff640 1 -- 172.21.15.22:0/2549024337 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed940b1070 con 0x7feda406f390 2024-03-29T04:18:03.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7fed9bfff640 1 -- 172.21.15.22:0/2549024337 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed940a8050 con 0x7feda406f390 2024-03-29T04:18:03.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7fed9bfff640 1 -- 172.21.15.22:0/2549024337 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed940ac860 con 0x7feda406f390 2024-03-29T04:18:03.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7fedab7b2640 1 -- 172.21.15.22:0/2549024337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feda406f390 msgr2=0x7feda41604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:03.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7fedab7b2640 1 --2- 172.21.15.22:0/2549024337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feda406f390 0x7feda41604b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fed94009870 tx=0x7fed9401c4d0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7fedab7b2640 1 -- 172.21.15.22:0/2549024337 shutdown_connections 2024-03-29T04:18:03.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7fedab7b2640 1 --2- 172.21.15.22:0/2549024337 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feda40cfd20 0x7feda40d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7fedab7b2640 1 --2- 172.21.15.22:0/2549024337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feda406f390 0x7feda41604b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7fedab7b2640 1 -- 172.21.15.22:0/2549024337 >> 172.21.15.22:0/2549024337 conn(0x7feda40b3c80 msgr2=0x7feda40b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:03.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7fedab7b2640 1 -- 172.21.15.22:0/2549024337 shutdown_connections 2024-03-29T04:18:03.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7fedab7b2640 1 -- 172.21.15.22:0/2549024337 wait complete. 2024-03-29T04:18:03.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7fedab7b2640 1 Processor -- start 2024-03-29T04:18:03.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7fedab7b2640 1 -- start start 2024-03-29T04:18:03.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7fedab7b2640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feda406f390 0x7feda415a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:03.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7fedab7b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feda40cfd20 0x7feda41652e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:03.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7fedab7b2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feda415c230 0x7feda4165820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:03.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7fedab7b2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7feda406f820 con 0x7feda40cfd20 2024-03-29T04:18:03.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7fedab7b2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feda40cc7a0 con 0x7feda415c230 2024-03-29T04:18:03.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7fedab7b2640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7feda41609f0 con 0x7feda406f390 2024-03-29T04:18:03.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7feda8d4d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feda40cfd20 0x7feda41652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:03.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7feda8d4d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feda40cfd20 0x7feda41652e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51950/0 (socket says 172.21.15.22:51950) 2024-03-29T04:18:03.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.516+0000 7feda8d4d640 1 -- 172.21.15.22:0/3267556426 learned_addr learned my addr 172.21.15.22:0/3267556426 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:03.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.520+0000 7feda954e640 1 --2- 172.21.15.22:0/3267556426 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feda406f390 0x7feda415a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:03.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.520+0000 7feda9d4f640 1 --2- 172.21.15.22:0/3267556426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feda415c230 0x7feda4165820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:03.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.520+0000 7feda8d4d640 1 -- 172.21.15.22:0/3267556426 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feda406f390 msgr2=0x7feda415a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:03.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.520+0000 7feda8d4d640 1 --2- 172.21.15.22:0/3267556426 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feda406f390 0x7feda415a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.520+0000 7feda8d4d640 1 -- 172.21.15.22:0/3267556426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feda415c230 msgr2=0x7feda4165820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.520+0000 7feda8d4d640 1 --2- 172.21.15.22:0/3267556426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feda415c230 0x7feda4165820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.520+0000 7feda8d4d640 1 -- 172.21.15.22:0/3267556426 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda406c8f0 con 0x7feda40cfd20 2024-03-29T04:18:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.520+0000 7feda9d4f640 1 --2- 172.21.15.22:0/3267556426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feda415c230 0x7feda4165820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.520+0000 7feda8d4d640 1 --2- 172.21.15.22:0/3267556426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feda40cfd20 0x7feda41652e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fed9401ab80 tx=0x7fed9401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.520+0000 7fed99ffb640 1 -- 172.21.15.22:0/3267556426 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed940bc070 con 0x7feda40cfd20 2024-03-29T04:18:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.520+0000 7fed99ffb640 1 -- 172.21.15.22:0/3267556426 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed940b7630 con 0x7feda40cfd20 2024-03-29T04:18:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.520+0000 7fed99ffb640 1 -- 172.21.15.22:0/3267556426 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed940b1040 con 0x7feda40cfd20 2024-03-29T04:18:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.520+0000 7fedab7b2640 1 -- 172.21.15.22:0/3267556426 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda406cbd0 con 0x7feda40cfd20 2024-03-29T04:18:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.520+0000 7fedab7b2640 1 -- 172.21.15.22:0/3267556426 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda406d190 con 0x7feda40cfd20 2024-03-29T04:18:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.520+0000 7fedab7b2640 1 -- 172.21.15.22:0/3267556426 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda4068890 con 0x7feda40cfd20 2024-03-29T04:18:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.520+0000 7fed99ffb640 1 -- 172.21.15.22:0/3267556426 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fed940bf070 con 0x7feda40cfd20 2024-03-29T04:18:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.520+0000 7fed99ffb640 1 --2- 172.21.15.22:0/3267556426 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fed88067fc0 0x7fed8806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.520+0000 7fed99ffb640 1 -- 172.21.15.22:0/3267556426 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fed94120220 con 0x7feda40cfd20 2024-03-29T04:18:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.520+0000 7feda954e640 1 --2- 172.21.15.22:0/3267556426 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fed88067fc0 0x7fed8806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.520+0000 7feda954e640 1 --2- 172.21.15.22:0/3267556426 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fed88067fc0 0x7fed8806a480 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7feda00012f0 tx=0x7feda0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.524+0000 7fed99ffb640 1 -- 172.21.15.22:0/3267556426 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fed94016150 con 0x7feda40cfd20 2024-03-29T04:18:03.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.724+0000 7fedab7b2640 1 -- 172.21.15.22:0/3267556426 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feda4070230 con 0x7feda40cfd20 2024-03-29T04:18:03.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.728+0000 7fed99ffb640 1 -- 172.21.15.22:0/3267556426 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7fed940f01f0 con 0x7feda40cfd20 2024-03-29T04:18:03.708 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:03.708 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3094991863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3094991863},{"type":"v1","addr":"172.21.15.22:6834","nonce":3094991863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2464050026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2464050026},{"type":"v1","addr":"172.21.15.151:6835","nonce":2464050026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2999052384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2999052384},{"type":"v1","addr":"172.21.15.22:6835","nonce":2999052384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:17:35.918069+0000","modified":"2024-03-29T04:17:53.156365+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6837/1466916671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":1466916671},{"type":"v1","addr":"172.21.15.151:6837","nonce":1466916671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-03-29T04:18:03.708 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2024-03-29T04:18:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.728+0000 7fedab7b2640 1 -- 172.21.15.22:0/3267556426 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fed88067fc0 msgr2=0x7fed8806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.728+0000 7fedab7b2640 1 --2- 172.21.15.22:0/3267556426 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fed88067fc0 0x7fed8806a480 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7feda00012f0 tx=0x7feda0009040 comp rx=0 tx=0).stop 2024-03-29T04:18:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.728+0000 7fedab7b2640 1 -- 172.21.15.22:0/3267556426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feda40cfd20 msgr2=0x7feda41652e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.728+0000 7fedab7b2640 1 --2- 172.21.15.22:0/3267556426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feda40cfd20 0x7feda41652e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fed9401ab80 tx=0x7fed9401ad10 comp rx=0 tx=0).stop 2024-03-29T04:18:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.728+0000 7fedab7b2640 1 -- 172.21.15.22:0/3267556426 shutdown_connections 2024-03-29T04:18:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.728+0000 7fedab7b2640 1 --2- 172.21.15.22:0/3267556426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feda415c230 0x7feda4165820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.728+0000 7fedab7b2640 1 --2- 172.21.15.22:0/3267556426 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fed88067fc0 0x7fed8806a480 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.728+0000 7fedab7b2640 1 --2- 172.21.15.22:0/3267556426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feda40cfd20 0x7feda41652e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.728+0000 7fedab7b2640 1 --2- 172.21.15.22:0/3267556426 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7feda406f390 0x7feda415a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.728+0000 7fedab7b2640 1 -- 172.21.15.22:0/3267556426 >> 172.21.15.22:0/3267556426 conn(0x7feda40b3c80 msgr2=0x7feda40cb230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.728+0000 7fedab7b2640 1 -- 172.21.15.22:0/3267556426 shutdown_connections 2024-03-29T04:18:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.728+0000 7fedab7b2640 1 -- 172.21.15.22:0/3267556426 wait complete. 2024-03-29T04:18:03.728 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:03.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.864+0000 7ff998668640 1 Processor -- start 2024-03-29T04:18:03.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.864+0000 7ff998668640 1 -- start start 2024-03-29T04:18:03.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff998668640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff99015a840 0x7ff99015e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:03.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff998668640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff99015e940 0x7ff99006da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:03.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff998668640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff99006df90 0x7ff99006e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:03.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff998668640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7ff99015eda0 con 0x7ff99015e940 2024-03-29T04:18:03.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff998668640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7ff990072cb0 con 0x7ff99006df90 2024-03-29T04:18:03.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff998668640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff990072f90 con 0x7ff99015a840 2024-03-29T04:18:03.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff995c03640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff99015e940 0x7ff99006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:03.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff995c03640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff99015e940 0x7ff99006da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51962/0 (socket says 172.21.15.22:51962) 2024-03-29T04:18:03.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff995c03640 1 -- 172.21.15.22:0/3829089104 learned_addr learned my addr 172.21.15.22:0/3829089104 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:03.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff996404640 1 --2- 172.21.15.22:0/3829089104 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff99015a840 0x7ff99015e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:03.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff996c05640 1 --2- 172.21.15.22:0/3829089104 >> v2:172.21.15.151:3300/0 conn(0x7ff99006df90 0x7ff99006e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:03.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff995c03640 1 -- 172.21.15.22:0/3829089104 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff99015a840 msgr2=0x7ff99015e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:03.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff995c03640 1 --2- 172.21.15.22:0/3829089104 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff99015a840 0x7ff99015e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff995c03640 1 -- 172.21.15.22:0/3829089104 >> v2:172.21.15.151:3300/0 conn(0x7ff99006df90 msgr2=0x7ff99006e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:03.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff995c03640 1 --2- 172.21.15.22:0/3829089104 >> v2:172.21.15.151:3300/0 conn(0x7ff99006df90 0x7ff99006e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff995c03640 1 -- 172.21.15.22:0/3829089104 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff990073270 con 0x7ff99015e940 2024-03-29T04:18:03.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff995c03640 1 --2- 172.21.15.22:0/3829089104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff99015e940 0x7ff99006da50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff9900cc0a0 tx=0x7ff98c01eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d62d77dd2d94bb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:03.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff995402640 1 -- 172.21.15.22:0/3829089104 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c0b4070 con 0x7ff99015e940 2024-03-29T04:18:03.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff995402640 1 -- 172.21.15.22:0/3829089104 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff98c0a7a50 con 0x7ff99015e940 2024-03-29T04:18:03.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff995402640 1 -- 172.21.15.22:0/3829089104 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c0a7e20 con 0x7ff99015e940 2024-03-29T04:18:03.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff998668640 1 -- 172.21.15.22:0/3829089104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff99015e940 msgr2=0x7ff99006da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:03.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff998668640 1 --2- 172.21.15.22:0/3829089104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff99015e940 0x7ff99006da50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff9900cc0a0 tx=0x7ff98c01eb30 comp rx=0 tx=0).stop 2024-03-29T04:18:03.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff998668640 1 -- 172.21.15.22:0/3829089104 shutdown_connections 2024-03-29T04:18:03.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff998668640 1 --2- 172.21.15.22:0/3829089104 >> v2:172.21.15.151:3300/0 conn(0x7ff99006df90 0x7ff99006e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff998668640 1 --2- 172.21.15.22:0/3829089104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff99015e940 0x7ff99006da50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff998668640 1 --2- 172.21.15.22:0/3829089104 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff99015a840 0x7ff99015e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff998668640 1 -- 172.21.15.22:0/3829089104 >> 172.21.15.22:0/3829089104 conn(0x7ff9900cfee0 msgr2=0x7ff9900d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:03.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff998668640 1 -- 172.21.15.22:0/3829089104 shutdown_connections 2024-03-29T04:18:03.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.868+0000 7ff998668640 1 -- 172.21.15.22:0/3829089104 wait complete. 2024-03-29T04:18:03.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff998668640 1 Processor -- start 2024-03-29T04:18:03.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff998668640 1 -- start start 2024-03-29T04:18:03.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff998668640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff99006df90 0x7ff99014c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:03.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff998668640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff99015a840 0x7ff99014c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:03.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff998668640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff99015e940 0x7ff99015ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:03.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff998668640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff990072a50 con 0x7ff99006df90 2024-03-29T04:18:03.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff998668640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff9900728d0 con 0x7ff99015a840 2024-03-29T04:18:03.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff998668640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff9900b2ef0 con 0x7ff99015e940 2024-03-29T04:18:03.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff996404640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff99006df90 0x7ff99014c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:03.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff996404640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff99006df90 0x7ff99014c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51978/0 (socket says 172.21.15.22:51978) 2024-03-29T04:18:03.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff996404640 1 -- 172.21.15.22:0/1733730681 learned_addr learned my addr 172.21.15.22:0/1733730681 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:03.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff995c03640 1 --2- 172.21.15.22:0/1733730681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff99015a840 0x7ff99014c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:03.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff996c05640 1 --2- 172.21.15.22:0/1733730681 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff99015e940 0x7ff99015ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:03.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff996404640 1 -- 172.21.15.22:0/1733730681 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff99015e940 msgr2=0x7ff99015ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:03.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff996404640 1 --2- 172.21.15.22:0/1733730681 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff99015e940 0x7ff99015ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff996404640 1 -- 172.21.15.22:0/1733730681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff99015a840 msgr2=0x7ff99014c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:03.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff996404640 1 --2- 172.21.15.22:0/1733730681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff99015a840 0x7ff99014c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:03.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff996404640 1 -- 172.21.15.22:0/1733730681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99014e4a0 con 0x7ff99006df90 2024-03-29T04:18:03.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff996404640 1 --2- 172.21.15.22:0/1733730681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff99006df90 0x7ff99014c210 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff98000d800 tx=0x7ff98000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:03.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff986ffd640 1 -- 172.21.15.22:0/1733730681 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98000b6c0 con 0x7ff99006df90 2024-03-29T04:18:03.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff986ffd640 1 -- 172.21.15.22:0/1733730681 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff980019070 con 0x7ff99006df90 2024-03-29T04:18:03.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff998668640 1 -- 172.21.15.22:0/1733730681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99014e780 con 0x7ff99006df90 2024-03-29T04:18:03.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff986ffd640 1 -- 172.21.15.22:0/1733730681 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98000bae0 con 0x7ff99006df90 2024-03-29T04:18:03.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff998668640 1 -- 172.21.15.22:0/1733730681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff99014ed40 con 0x7ff99006df90 2024-03-29T04:18:03.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.872+0000 7ff998668640 1 -- 172.21.15.22:0/1733730681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9900d0570 con 0x7ff99006df90 2024-03-29T04:18:03.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.876+0000 7ff986ffd640 1 -- 172.21.15.22:0/1733730681 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff98000bc80 con 0x7ff99006df90 2024-03-29T04:18:03.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.876+0000 7ff986ffd640 1 --2- 172.21.15.22:0/1733730681 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff974067fc0 0x7ff97406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:03.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.876+0000 7ff986ffd640 1 -- 172.21.15.22:0/1733730681 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff980086c60 con 0x7ff99006df90 2024-03-29T04:18:03.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.876+0000 7ff995c03640 1 --2- 172.21.15.22:0/1733730681 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff974067fc0 0x7ff97406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:03.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.876+0000 7ff995c03640 1 --2- 172.21.15.22:0/1733730681 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff974067fc0 0x7ff97406a480 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0b1000 tx=0x7ff98c0a9070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:03.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:03.880+0000 7ff986ffd640 1 -- 172.21.15.22:0/1733730681 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff980057160 con 0x7ff99006df90 2024-03-29T04:18:04.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.036+0000 7ff998668640 1 -- 172.21.15.22:0/1733730681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff99006dd70 con 0x7ff99006df90 2024-03-29T04:18:04.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.036+0000 7ff986ffd640 1 -- 172.21.15.22:0/1733730681 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18514 (secure 0 0 0) 0x7ff980056a60 con 0x7ff99006df90 2024-03-29T04:18:04.019 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:04.019 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:17:42.645632+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:04.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.040+0000 7ff998668640 1 -- 172.21.15.22:0/1733730681 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff974067fc0 msgr2=0x7ff97406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:04.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.040+0000 7ff998668640 1 --2- 172.21.15.22:0/1733730681 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff974067fc0 0x7ff97406a480 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0b1000 tx=0x7ff98c0a9070 comp rx=0 tx=0).stop 2024-03-29T04:18:04.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.040+0000 7ff998668640 1 -- 172.21.15.22:0/1733730681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff99006df90 msgr2=0x7ff99014c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:04.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.040+0000 7ff998668640 1 --2- 172.21.15.22:0/1733730681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff99006df90 0x7ff99014c210 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff98000d800 tx=0x7ff98000dcc0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.040+0000 7ff998668640 1 -- 172.21.15.22:0/1733730681 shutdown_connections 2024-03-29T04:18:04.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.040+0000 7ff998668640 1 --2- 172.21.15.22:0/1733730681 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff99015e940 0x7ff99015ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.040+0000 7ff998668640 1 --2- 172.21.15.22:0/1733730681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff99015a840 0x7ff99014c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.040+0000 7ff998668640 1 --2- 172.21.15.22:0/1733730681 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff974067fc0 0x7ff97406a480 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.040+0000 7ff998668640 1 --2- 172.21.15.22:0/1733730681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff99006df90 0x7ff99014c210 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.040+0000 7ff998668640 1 -- 172.21.15.22:0/1733730681 >> 172.21.15.22:0/1733730681 conn(0x7ff9900cfee0 msgr2=0x7ff9900aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:04.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.040+0000 7ff998668640 1 -- 172.21.15.22:0/1733730681 shutdown_connections 2024-03-29T04:18:04.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.040+0000 7ff998668640 1 -- 172.21.15.22:0/1733730681 wait complete. 2024-03-29T04:18:04.036 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:18:04.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.172+0000 7f8952934640 1 Processor -- start 2024-03-29T04:18:04.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.172+0000 7f8952934640 1 -- start start 2024-03-29T04:18:04.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f8952934640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f894c0700f0 0x7f894c167170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f8952934640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f894c146100 con 0x7f894c145d20 2024-03-29T04:18:04.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f8952934640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f894c0cfe80 con 0x7f894c070a00 2024-03-29T04:18:04.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f8952934640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f894c06e110 con 0x7f894c0700f0 2024-03-29T04:18:04.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894bfff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f894c145d20 0x7f894c0cf6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:44402/0 (socket says 172.21.15.22:44402) 2024-03-29T04:18:04.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894bfff640 1 -- 172.21.15.22:0/1031072471 learned_addr learned my addr 172.21.15.22:0/1031072471 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:04.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f8950ed1640 1 --2- 172.21.15.22:0/1031072471 >> v2:172.21.15.151:3300/0 conn(0x7f894c0700f0 0x7f894c167170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894affd640 1 -- 172.21.15.22:0/1031072471 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1538640562 0 0) 0x7f894c146100 con 0x7f894c145d20 2024-03-29T04:18:04.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894affd640 1 -- 172.21.15.22:0/1031072471 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8934003530 con 0x7f894c145d20 2024-03-29T04:18:04.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894affd640 1 -- 172.21.15.22:0/1031072471 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1576609915 0 0) 0x7f8934003530 con 0x7f894c145d20 2024-03-29T04:18:04.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894affd640 1 -- 172.21.15.22:0/1031072471 >> v2:172.21.15.151:3300/0 conn(0x7f894c0700f0 msgr2=0x7f894c167170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:04.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894affd640 1 --2- 172.21.15.22:0/1031072471 >> v2:172.21.15.151:3300/0 conn(0x7f894c0700f0 0x7f894c167170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894affd640 1 -- 172.21.15.22:0/1031072471 >> v1:172.21.15.151:6789/0 conn(0x7f894c070a00 legacy=0x7f894c06f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894affd640 1 -- 172.21.15.22:0/1031072471 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894c0d0160 con 0x7f894c145d20 2024-03-29T04:18:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894affd640 1 -- 172.21.15.22:0/1031072471 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f893c001a00 con 0x7f894c145d20 2024-03-29T04:18:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894affd640 1 -- 172.21.15.22:0/1031072471 >> v1:172.21.15.22:6789/0 conn(0x7f894c145d20 legacy=0x7f894c0cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894affd640 1 --2- 172.21.15.22:0/1031072471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89340039c0 0x7f8934003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894affd640 1 --2- 172.21.15.22:0/1031072471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8934004810 0x7f8934004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894affd640 1 --2- 172.21.15.22:0/1031072471 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89340051d0 0x7f893400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894affd640 1 -- 172.21.15.22:0/1031072471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f894c06e110 con 0x7f8934004810 2024-03-29T04:18:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894affd640 1 -- 172.21.15.22:0/1031072471 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f893400c170 con 0x7f89340039c0 2024-03-29T04:18:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894affd640 1 -- 172.21.15.22:0/1031072471 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f893400c370 con 0x7f89340051d0 2024-03-29T04:18:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894b7fe640 1 --2- 172.21.15.22:0/1031072471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8934004810 0x7f8934004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894bfff640 1 --2- 172.21.15.22:0/1031072471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89340039c0 0x7f8934003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f8950ed1640 1 --2- 172.21.15.22:0/1031072471 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89340051d0 0x7f893400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894b7fe640 1 -- 172.21.15.22:0/1031072471 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89340051d0 msgr2=0x7f893400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:04.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894b7fe640 1 --2- 172.21.15.22:0/1031072471 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89340051d0 0x7f893400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894b7fe640 1 -- 172.21.15.22:0/1031072471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89340039c0 msgr2=0x7f8934003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:04.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894b7fe640 1 --2- 172.21.15.22:0/1031072471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89340039c0 0x7f8934003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894b7fe640 1 -- 172.21.15.22:0/1031072471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8934003530 con 0x7f8934004810 2024-03-29T04:18:04.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f8950ed1640 1 --2- 172.21.15.22:0/1031072471 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89340051d0 0x7f893400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:04.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f8950ed1640 1 -- 172.21.15.22:0/1031072471 reap_dead start 2024-03-29T04:18:04.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894b7fe640 1 --2- 172.21.15.22:0/1031072471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8934004810 0x7f8934004c90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8940001780 tx=0x7f894001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2768ddf60e1776b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:04.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894affd640 1 -- 172.21.15.22:0/1031072471 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89400013a0 con 0x7f8934004810 2024-03-29T04:18:04.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894affd640 1 -- 172.21.15.22:0/1031072471 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89400052f0 con 0x7f8934004810 2024-03-29T04:18:04.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.176+0000 7f894affd640 1 -- 172.21.15.22:0/1031072471 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940003840 con 0x7f8934004810 2024-03-29T04:18:04.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f8952934640 1 -- 172.21.15.22:0/1031072471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8934004810 msgr2=0x7f8934004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:04.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f8952934640 1 --2- 172.21.15.22:0/1031072471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8934004810 0x7f8934004c90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8940001780 tx=0x7f894001c400 comp rx=0 tx=0).stop 2024-03-29T04:18:04.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f8952934640 1 -- 172.21.15.22:0/1031072471 shutdown_connections 2024-03-29T04:18:04.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f8952934640 1 --2- 172.21.15.22:0/1031072471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8934004810 0x7f8934004c90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f8952934640 1 -- 172.21.15.22:0/1031072471 >> 172.21.15.22:0/1031072471 conn(0x7f894c06e410 msgr2=0x7f894c06e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:04.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f8952934640 1 -- 172.21.15.22:0/1031072471 shutdown_connections 2024-03-29T04:18:04.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f8952934640 1 -- 172.21.15.22:0/1031072471 wait complete. 2024-03-29T04:18:04.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f8952934640 1 Processor -- start 2024-03-29T04:18:04.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f8952934640 1 -- start start 2024-03-29T04:18:04.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f8952934640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8934004810 0x7f894c14dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f8952934640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f894c14e230 0x7f894c13f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f8952934640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f894c13fb30 0x7f894c1e8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f8952934640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f894c0cfc10 con 0x7f894c14e230 2024-03-29T04:18:04.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f8952934640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f894c0cfa90 con 0x7f8934004810 2024-03-29T04:18:04.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f8952934640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f894c1605f0 con 0x7f894c13fb30 2024-03-29T04:18:04.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f894b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f894c14e230 0x7f894c13f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f8950ed1640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f894c13fb30 0x7f894c1e8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f894b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f894c14e230 0x7f894c13f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51998/0 (socket says 172.21.15.22:51998) 2024-03-29T04:18:04.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f894b7fe640 1 -- 172.21.15.22:0/2191367226 learned_addr learned my addr 172.21.15.22:0/2191367226 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:04.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f894bfff640 1 --2- 172.21.15.22:0/2191367226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8934004810 0x7f894c14dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f894b7fe640 1 -- 172.21.15.22:0/2191367226 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f894c13fb30 msgr2=0x7f894c1e8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:04.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f894b7fe640 1 --2- 172.21.15.22:0/2191367226 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f894c13fb30 0x7f894c1e8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f894b7fe640 1 -- 172.21.15.22:0/2191367226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8934004810 msgr2=0x7f894c14dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:04.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f894b7fe640 1 --2- 172.21.15.22:0/2191367226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8934004810 0x7f894c14dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f894b7fe640 1 -- 172.21.15.22:0/2191367226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894c13ffb0 con 0x7f894c14e230 2024-03-29T04:18:04.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f894b7fe640 1 --2- 172.21.15.22:0/2191367226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f894c14e230 0x7f894c13f5f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8940005040 tx=0x7f8934005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:04.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f8948ff9640 1 -- 172.21.15.22:0/2191367226 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940007a40 con 0x7f894c14e230 2024-03-29T04:18:04.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f8948ff9640 1 -- 172.21.15.22:0/2191367226 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89400ba070 con 0x7f894c14e230 2024-03-29T04:18:04.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f8952934640 1 -- 172.21.15.22:0/2191367226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894c140260 con 0x7f894c14e230 2024-03-29T04:18:04.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f8948ff9640 1 -- 172.21.15.22:0/2191367226 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89400054a0 con 0x7f894c14e230 2024-03-29T04:18:04.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.180+0000 7f8952934640 1 -- 172.21.15.22:0/2191367226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f894c140820 con 0x7f894c14e230 2024-03-29T04:18:04.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.184+0000 7f8952934640 1 -- 172.21.15.22:0/2191367226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f894c06f410 con 0x7f894c14e230 2024-03-29T04:18:04.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.184+0000 7f8948ff9640 1 -- 172.21.15.22:0/2191367226 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f8940007030 con 0x7f894c14e230 2024-03-29T04:18:04.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.184+0000 7f8948ff9640 1 --2- 172.21.15.22:0/2191367226 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8938067fc0 0x7f893806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.184+0000 7f8948ff9640 1 -- 172.21.15.22:0/2191367226 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8940120790 con 0x7f894c14e230 2024-03-29T04:18:04.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.184+0000 7f894bfff640 1 --2- 172.21.15.22:0/2191367226 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8938067fc0 0x7f893806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.184+0000 7f894bfff640 1 --2- 172.21.15.22:0/2191367226 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8938067fc0 0x7f893806a480 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f893c0013e0 tx=0x7f893c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:04.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.188+0000 7f8948ff9640 1 -- 172.21.15.22:0/2191367226 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f8940016110 con 0x7f894c14e230 2024-03-29T04:18:04.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.388+0000 7f8952934640 1 -- 172.21.15.22:0/2191367226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f894c068870 con 0x7f894c14e230 2024-03-29T04:18:04.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.388+0000 7f8948ff9640 1 -- 172.21.15.22:0/2191367226 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f89400f0b60 con 0x7f894c14e230 2024-03-29T04:18:04.370 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:04.370 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3094991863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3094991863},{"type":"v1","addr":"172.21.15.22:6834","nonce":3094991863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2464050026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2464050026},{"type":"v1","addr":"172.21.15.151:6835","nonce":2464050026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2999052384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2999052384},{"type":"v1","addr":"172.21.15.22:6835","nonce":2999052384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:17:35.918069+0000","modified":"2024-03-29T04:17:53.156365+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6837/1466916671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":1466916671},{"type":"v1","addr":"172.21.15.151:6837","nonce":1466916671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-03-29T04:18:04.370 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2024-03-29T04:18:04.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.392+0000 7f8952934640 1 -- 172.21.15.22:0/2191367226 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8938067fc0 msgr2=0x7f893806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:04.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.392+0000 7f8952934640 1 --2- 172.21.15.22:0/2191367226 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8938067fc0 0x7f893806a480 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f893c0013e0 tx=0x7f893c009040 comp rx=0 tx=0).stop 2024-03-29T04:18:04.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.392+0000 7f8952934640 1 -- 172.21.15.22:0/2191367226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f894c14e230 msgr2=0x7f894c13f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:04.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.392+0000 7f8952934640 1 --2- 172.21.15.22:0/2191367226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f894c14e230 0x7f894c13f5f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8940005040 tx=0x7f8934005840 comp rx=0 tx=0).stop 2024-03-29T04:18:04.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.392+0000 7f8952934640 1 -- 172.21.15.22:0/2191367226 shutdown_connections 2024-03-29T04:18:04.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.392+0000 7f8952934640 1 --2- 172.21.15.22:0/2191367226 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f894c13fb30 0x7f894c1e8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.392+0000 7f8952934640 1 --2- 172.21.15.22:0/2191367226 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8938067fc0 0x7f893806a480 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.392+0000 7f8952934640 1 --2- 172.21.15.22:0/2191367226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f894c14e230 0x7f894c13f5f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.392+0000 7f8952934640 1 --2- 172.21.15.22:0/2191367226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8934004810 0x7f894c14dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.392+0000 7f8952934640 1 -- 172.21.15.22:0/2191367226 >> 172.21.15.22:0/2191367226 conn(0x7f894c06e410 msgr2=0x7f894c06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:04.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.392+0000 7f8952934640 1 -- 172.21.15.22:0/2191367226 shutdown_connections 2024-03-29T04:18:04.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.392+0000 7f8952934640 1 -- 172.21.15.22:0/2191367226 wait complete. 2024-03-29T04:18:04.792 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T04:17:35.918069+0000', 'modified': '2024-03-29T04:17:53.156365+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4294}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4294': {'gid': 4294, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.151:6837/1466916671', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 1466916671}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 1466916671}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4294, 'qdb_cluster': [4294]} 2024-03-29T04:18:04.792 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T04:18:04.793 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-29T04:18:04.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.948+0000 7f0c1929a640 1 Processor -- start 2024-03-29T04:18:04.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.948+0000 7f0c1929a640 1 -- start start 2024-03-29T04:18:04.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.948+0000 7f0c1929a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0c140b41f0 0x7f0c140cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.948+0000 7f0c1929a640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c140ccef0 con 0x7f0c1406ca50 2024-03-29T04:18:04.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.948+0000 7f0c1929a640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c140b3710 con 0x7f0c1406c670 2024-03-29T04:18:04.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.948+0000 7f0c1929a640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f0c140b34c0 con 0x7f0c140b41f0 2024-03-29T04:18:04.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.948+0000 7f0c0bfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0c140b41f0 0x7f0c140cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.948+0000 7f0c137fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f0c1406ca50 0x7f0c140ce5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36272/0 (socket says 172.21.15.22:36272) 2024-03-29T04:18:04.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.948+0000 7f0c137fe640 1 -- 172.21.15.22:0/3768685231 learned_addr learned my addr 172.21.15.22:0/3768685231 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:04.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.948+0000 7f0c0bfff640 1 -- 172.21.15.22:0/3768685231 >> v1:172.21.15.151:6789/0 conn(0x7f0c1406c670 legacy=0x7f0c1406e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:04.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.948+0000 7f0c0bfff640 1 -- 172.21.15.22:0/3768685231 >> v1:172.21.15.22:6789/0 conn(0x7f0c1406ca50 legacy=0x7f0c140ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:04.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.948+0000 7f0c0bfff640 1 -- 172.21.15.22:0/3768685231 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c140b39f0 con 0x7f0c140b41f0 2024-03-29T04:18:04.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.948+0000 7f0c127fc640 1 -- 172.21.15.22:0/3768685231 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1387464145 0 0) 0x7f0c140ccef0 con 0x7f0c1406ca50 2024-03-29T04:18:04.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.952+0000 7f0c0bfff640 1 --2- 172.21.15.22:0/3768685231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c140b41f0 0x7f0c140cc910 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0097c0 tx=0x7f0bfc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ce956eda306d16 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:04.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.952+0000 7f0c127fc640 1 -- 172.21.15.22:0/3768685231 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc0b1070 con 0x7f0c140b41f0 2024-03-29T04:18:04.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.952+0000 7f0c127fc640 1 -- 172.21.15.22:0/3768685231 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bfc0ac410 con 0x7f0c140b41f0 2024-03-29T04:18:04.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.952+0000 7f0c127fc640 1 -- 172.21.15.22:0/3768685231 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc0ac7e0 con 0x7f0c140b41f0 2024-03-29T04:18:04.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.952+0000 7f0c1929a640 1 -- 172.21.15.22:0/3768685231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c140b41f0 msgr2=0x7f0c140cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:04.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.952+0000 7f0c1929a640 1 --2- 172.21.15.22:0/3768685231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c140b41f0 0x7f0c140cc910 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0097c0 tx=0x7f0bfc01c4a0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.952+0000 7f0c1929a640 1 -- 172.21.15.22:0/3768685231 shutdown_connections 2024-03-29T04:18:04.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.952+0000 7f0c1929a640 1 --2- 172.21.15.22:0/3768685231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c140b41f0 0x7f0c140cc910 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.952+0000 7f0c1929a640 1 -- 172.21.15.22:0/3768685231 >> 172.21.15.22:0/3768685231 conn(0x7f0c140cfee0 msgr2=0x7f0c140d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:04.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.952+0000 7f0c1929a640 1 -- 172.21.15.22:0/3768685231 shutdown_connections 2024-03-29T04:18:04.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.952+0000 7f0c1929a640 1 -- 172.21.15.22:0/3768685231 wait complete. 2024-03-29T04:18:04.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c1929a640 1 Processor -- start 2024-03-29T04:18:04.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c1929a640 1 -- start start 2024-03-29T04:18:04.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c1929a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c1406c670 0x7f0c1415e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c1929a640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0c1406ca50 0x7f0c14071db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c1929a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c140b41f0 0x7f0c140722f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c1929a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1416a080 con 0x7f0c140b41f0 2024-03-29T04:18:04.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c1929a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0c14169f00 con 0x7f0c1406c670 2024-03-29T04:18:04.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c1929a640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0c1416a200 con 0x7f0c1406ca50 2024-03-29T04:18:04.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c137fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c140b41f0 0x7f0c140722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c137fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c140b41f0 0x7f0c140722f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49664/0 (socket says 172.21.15.22:49664) 2024-03-29T04:18:04.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c137fe640 1 -- 172.21.15.22:0/2526676972 learned_addr learned my addr 172.21.15.22:0/2526676972 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:04.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c12ffd640 1 --2- 172.21.15.22:0/2526676972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c1406c670 0x7f0c1415e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c0bfff640 1 --2- 172.21.15.22:0/2526676972 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0c1406ca50 0x7f0c14071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c137fe640 1 -- 172.21.15.22:0/2526676972 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0c1406ca50 msgr2=0x7f0c14071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:04.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c137fe640 1 --2- 172.21.15.22:0/2526676972 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0c1406ca50 0x7f0c14071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c137fe640 1 -- 172.21.15.22:0/2526676972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c1406c670 msgr2=0x7f0c1415e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:04.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c137fe640 1 --2- 172.21.15.22:0/2526676972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c1406c670 0x7f0c1415e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c137fe640 1 -- 172.21.15.22:0/2526676972 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c14071750 con 0x7f0c140b41f0 2024-03-29T04:18:04.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c0bfff640 1 --2- 172.21.15.22:0/2526676972 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0c1406ca50 0x7f0c14071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:04.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c137fe640 1 --2- 172.21.15.22:0/2526676972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c140b41f0 0x7f0c140722f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00c730 tx=0x7f0c0c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:04.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c0b7fe640 1 -- 172.21.15.22:0/2526676972 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c018070 con 0x7f0c140b41f0 2024-03-29T04:18:04.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c0b7fe640 1 -- 172.21.15.22:0/2526676972 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c0c00ce40 con 0x7f0c140b41f0 2024-03-29T04:18:04.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c1929a640 1 -- 172.21.15.22:0/2526676972 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c1415cdb0 con 0x7f0c140b41f0 2024-03-29T04:18:04.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c0b7fe640 1 -- 172.21.15.22:0/2526676972 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c00a860 con 0x7f0c140b41f0 2024-03-29T04:18:04.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c1929a640 1 -- 172.21.15.22:0/2526676972 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c1415d370 con 0x7f0c140b41f0 2024-03-29T04:18:04.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c0b7fe640 1 -- 172.21.15.22:0/2526676972 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f0c0c01b070 con 0x7f0c140b41f0 2024-03-29T04:18:04.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.956+0000 7f0c0b7fe640 1 --2- 172.21.15.22:0/2526676972 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0bf4067fc0 0x7f0bf406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c0b7fe640 1 -- 172.21.15.22:0/2526676972 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0c0c085d00 con 0x7f0c140b41f0 2024-03-29T04:18:04.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c12ffd640 1 --2- 172.21.15.22:0/2526676972 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0bf4067fc0 0x7f0bf406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c1c41c000 1 Processor -- start 2024-03-29T04:18:04.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c1c41c000 1 -- start start 2024-03-29T04:18:04.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c12ffd640 1 --2- 172.21.15.22:0/2526676972 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0bf4067fc0 0x7f0bf406a480 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0c1413e740 tx=0x7f0c04009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:04.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c1c41c000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec844a0 0x55d8eec84880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c1c41c000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8eec84dc0 0x55d8eec37b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c1c41c000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec380b0 0x55d8eec38490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c1c41c000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d8eea8a950 con 0x55d8eec844a0 2024-03-29T04:18:04.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c1c41c000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d8eea27990 con 0x55d8eec84dc0 2024-03-29T04:18:04.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c1c41c000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x55d8eea86900 con 0x55d8eec380b0 2024-03-29T04:18:04.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c12ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec844a0 0x55d8eec84880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c12ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec844a0 0x55d8eec84880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49666/0 (socket says 172.21.15.22:49666) 2024-03-29T04:18:04.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c12ffd640 1 -- 172.21.15.22:0/179111333 learned_addr learned my addr 172.21.15.22:0/179111333 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:04.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c0bfff640 1 --2- 172.21.15.22:0/179111333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8eec84dc0 0x55d8eec37b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c137fe640 1 --2- 172.21.15.22:0/179111333 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec380b0 0x55d8eec38490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c12ffd640 1 -- 172.21.15.22:0/179111333 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec380b0 msgr2=0x55d8eec38490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:04.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c12ffd640 1 --2- 172.21.15.22:0/179111333 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec380b0 0x55d8eec38490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c12ffd640 1 -- 172.21.15.22:0/179111333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8eec84dc0 msgr2=0x55d8eec37b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:04.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c12ffd640 1 --2- 172.21.15.22:0/179111333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8eec84dc0 0x55d8eec37b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c12ffd640 1 -- 172.21.15.22:0/179111333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8eeacf100 con 0x55d8eec844a0 2024-03-29T04:18:04.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c137fe640 1 --2- 172.21.15.22:0/179111333 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec380b0 0x55d8eec38490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:04.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c12ffd640 1 --2- 172.21.15.22:0/179111333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec844a0 0x55d8eec84880 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0c040052f0 tx=0x7f0c04011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd399135c7ffbd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:04.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c08ff9640 1 -- 172.21.15.22:0/179111333 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c04005460 con 0x55d8eec844a0 2024-03-29T04:18:04.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c08ff9640 1 -- 172.21.15.22:0/179111333 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c0400e6b0 con 0x55d8eec844a0 2024-03-29T04:18:04.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c08ff9640 1 -- 172.21.15.22:0/179111333 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0400ea60 con 0x55d8eec844a0 2024-03-29T04:18:04.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c1c41c000 1 -- 172.21.15.22:0/179111333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec844a0 msgr2=0x55d8eec84880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:04.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c1c41c000 1 --2- 172.21.15.22:0/179111333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec844a0 0x55d8eec84880 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0c040052f0 tx=0x7f0c04011000 comp rx=0 tx=0).stop 2024-03-29T04:18:04.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c1c41c000 1 -- 172.21.15.22:0/179111333 shutdown_connections 2024-03-29T04:18:04.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c1c41c000 1 --2- 172.21.15.22:0/179111333 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec380b0 0x55d8eec38490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c1c41c000 1 --2- 172.21.15.22:0/179111333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8eec84dc0 0x55d8eec37b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c1c41c000 1 --2- 172.21.15.22:0/179111333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec844a0 0x55d8eec84880 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c1c41c000 1 -- 172.21.15.22:0/179111333 >> 172.21.15.22:0/179111333 conn(0x55d8eeb22b80 msgr2=0x55d8eec67460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:04.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c1c41c000 1 -- 172.21.15.22:0/179111333 shutdown_connections 2024-03-29T04:18:04.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c1c41c000 1 -- 172.21.15.22:0/179111333 wait complete. 2024-03-29T04:18:04.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.960+0000 7f0c1c41c000 1 -- start start 2024-03-29T04:18:04.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c1c41c000 1 Processor -- start 2024-03-29T04:18:04.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c1c41c000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8eec380b0 0x55d8eec3fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c1c41c000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec844a0 0x55d8eec68690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c1c41c000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec84dc0 0x55d8eec68bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c1c41c000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d8eea227b0 con 0x55d8eec844a0 2024-03-29T04:18:04.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c1c41c000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d8eea483e0 con 0x55d8eec380b0 2024-03-29T04:18:04.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c1c41c000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x55d8eea8ef80 con 0x55d8eec84dc0 2024-03-29T04:18:04.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c1c41c000 10 client.0 _ll_register_callbacks cb 0x55d8eeaeb0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:18:04.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c0bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec844a0 0x55d8eec68690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c0bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec844a0 0x55d8eec68690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49670/0 (socket says 172.21.15.22:49670) 2024-03-29T04:18:04.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c0bfff640 1 -- 172.21.15.22:0/1556155913 learned_addr learned my addr 172.21.15.22:0/1556155913 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:04.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c12ffd640 1 --2- 172.21.15.22:0/1556155913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8eec380b0 0x55d8eec3fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c137fe640 1 --2- 172.21.15.22:0/1556155913 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec84dc0 0x55d8eec68bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c0bfff640 1 -- 172.21.15.22:0/1556155913 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec84dc0 msgr2=0x55d8eec68bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:04.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c0bfff640 1 --2- 172.21.15.22:0/1556155913 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec84dc0 0x55d8eec68bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c0bfff640 1 -- 172.21.15.22:0/1556155913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8eec380b0 msgr2=0x55d8eec3fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:04.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c0bfff640 1 --2- 172.21.15.22:0/1556155913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8eec380b0 0x55d8eec3fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c0bfff640 1 -- 172.21.15.22:0/1556155913 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8eea548c0 con 0x55d8eec844a0 2024-03-29T04:18:04.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c12ffd640 1 --2- 172.21.15.22:0/1556155913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8eec380b0 0x55d8eec3fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:04.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c0bfff640 1 --2- 172.21.15.22:0/1556155913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec844a0 0x55d8eec68690 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0af040 tx=0x7f0bfc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae15f1fa0d2da006 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:04.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0bf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-29T04:18:04.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0bf1ffb640 1 -- 172.21.15.22:0/1556155913 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc0b1050 con 0x55d8eec844a0 2024-03-29T04:18:04.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0bf1ffb640 1 -- 172.21.15.22:0/1556155913 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0bfc0a9040 con 0x55d8eec844a0 2024-03-29T04:18:04.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c1c41c000 1 -- 172.21.15.22:0/1556155913 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c0400e6b0 con 0x55d8eec844a0 2024-03-29T04:18:04.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0bf1ffb640 1 -- 172.21.15.22:0/1556155913 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f0bfc0c3050 con 0x55d8eec844a0 2024-03-29T04:18:04.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c1c41c000 10 client.4851 fetch_fsmap learned FSMap version 9 2024-03-29T04:18:04.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c1c41c000 1 -- 172.21.15.22:0/1556155913 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55d8eea12e80 con 0x55d8eec844a0 2024-03-29T04:18:04.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0bf1ffb640 1 -- 172.21.15.22:0/1556155913 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c0400e6b0 con 0x55d8eec844a0 2024-03-29T04:18:04.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0bf1ffb640 1 -- 172.21.15.22:0/1556155913 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f0bfc0c6020 con 0x55d8eec844a0 2024-03-29T04:18:04.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c1c41c000 10 client.4851 fetch_fsmap finished waiting for FSMap version 9 2024-03-29T04:18:04.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c1c41c000 10 client.4851 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-29T04:18:04.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c1c41c000 20 client.4851 populate_metadata read hostname 'smithi022' 2024-03-29T04:18:04.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c1c41c000 1 --2- 172.21.15.22:0/1556155913 >> [v2:172.21.15.151:6836/1466916671,v1:172.21.15.151:6837/1466916671] conn(0x55d8eec94540 0x55d8eec96930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c1c41c000 4 client.4851 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:18:04.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c1c41c000 1 -- 172.21.15.22:0/1556155913 --> [v2:172.21.15.151:6836/1466916671,v1:172.21.15.151:6837/1466916671] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8eea25bc0 con 0x55d8eec94540 2024-03-29T04:18:04.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c12ffd640 1 --2- 172.21.15.22:0/1556155913 >> [v2:172.21.15.151:6836/1466916671,v1:172.21.15.151:6837/1466916671] conn(0x55d8eec94540 0x55d8eec96930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0c12ffd640 1 --2- 172.21.15.22:0/1556155913 >> [v2:172.21.15.151:6836/1466916671,v1:172.21.15.151:6837/1466916671] conn(0x55d8eec94540 0x55d8eec96930 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8535ed6c9a177b0c server_cookie=abb5c293a51e7675 in_seq=0 out_seq=0 2024-03-29T04:18:04.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.964+0000 7f0bf1ffb640 10 client.4851 ms_handle_connect on v2:172.21.15.151:6836/1466916671 2024-03-29T04:18:04.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.968+0000 7f0bf1ffb640 1 -- 172.21.15.22:0/1556155913 <== mds.0 v2:172.21.15.151:6836/1466916671 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55d8eea25bc0 con 0x55d8eec94540 2024-03-29T04:18:04.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.968+0000 7f0bf1ffb640 10 client.4851 handle_command_reply: tid=0 2024-03-29T04:18:04.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.968+0000 7f0c1c41c000 1 client.4851 shutdown 2024-03-29T04:18:04.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.968+0000 7f0c1c41c000 1 -- 172.21.15.22:0/1556155913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec844a0 msgr2=0x55d8eec68690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:04.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.968+0000 7f0c1c41c000 1 --2- 172.21.15.22:0/1556155913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec844a0 0x55d8eec68690 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0af040 tx=0x7f0bfc0046d0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.968+0000 7f0c1c41c000 1 -- 172.21.15.22:0/1556155913 shutdown_connections 2024-03-29T04:18:04.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.968+0000 7f0c1c41c000 1 --2- 172.21.15.22:0/1556155913 >> [v2:172.21.15.151:6836/1466916671,v1:172.21.15.151:6837/1466916671] conn(0x55d8eec94540 0x55d8eec96930 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.968+0000 7f0c1c41c000 1 --2- 172.21.15.22:0/1556155913 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec84dc0 0x55d8eec68bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.968+0000 7f0c1c41c000 1 --2- 172.21.15.22:0/1556155913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec844a0 0x55d8eec68690 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.968+0000 7f0c1c41c000 1 --2- 172.21.15.22:0/1556155913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8eec380b0 0x55d8eec3fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.968+0000 7f0c1c41c000 1 -- 172.21.15.22:0/1556155913 >> 172.21.15.22:0/1556155913 conn(0x55d8eeb22b80 msgr2=0x55d8eec37270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:04.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.968+0000 7f0bf1ffb640 0 client.4851 ms_handle_reset on v2:172.21.15.151:6836/1466916671 2024-03-29T04:18:04.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.968+0000 7f0c1c41c000 1 -- 172.21.15.22:0/1556155913 shutdown_connections 2024-03-29T04:18:04.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.968+0000 7f0c1c41c000 1 -- 172.21.15.22:0/1556155913 wait complete. 2024-03-29T04:18:04.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.968+0000 7f0c1c41c000 20 client.4851 trim_cache size 0 max 16384 2024-03-29T04:18:04.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.984+0000 7f0c1c41c000 1 Processor -- start 2024-03-29T04:18:04.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.984+0000 7f0c1c41c000 1 -- start start 2024-03-29T04:18:04.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.984+0000 7f0c1c41c000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec380b0 0x55d8eec3df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.984+0000 7f0c1c41c000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec844a0 0x55d8eec3e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.984+0000 7f0c1c41c000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8eec84dc0 0x55d8eec362d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.984+0000 7f0c1c41c000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0c14204ec0 con 0x55d8eec844a0 2024-03-29T04:18:04.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.984+0000 7f0c1c41c000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0c14203ef0 con 0x55d8eec84dc0 2024-03-29T04:18:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.984+0000 7f0c1c41c000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0c14203370 con 0x55d8eec380b0 2024-03-29T04:18:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c0bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec844a0 0x55d8eec3e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c0bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec844a0 0x55d8eec3e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49678/0 (socket says 172.21.15.22:49678) 2024-03-29T04:18:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c0bfff640 1 -- 172.21.15.22:0/1473361641 learned_addr learned my addr 172.21.15.22:0/1473361641 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c137fe640 1 --2- 172.21.15.22:0/1473361641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8eec84dc0 0x55d8eec362d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c12ffd640 1 --2- 172.21.15.22:0/1473361641 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec380b0 0x55d8eec3df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c0bfff640 1 -- 172.21.15.22:0/1473361641 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec380b0 msgr2=0x55d8eec3df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c0bfff640 1 --2- 172.21.15.22:0/1473361641 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec380b0 0x55d8eec3df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c0bfff640 1 -- 172.21.15.22:0/1473361641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8eec84dc0 msgr2=0x55d8eec362d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c0bfff640 1 --2- 172.21.15.22:0/1473361641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8eec84dc0 0x55d8eec362d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c0bfff640 1 -- 172.21.15.22:0/1473361641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8eea45490 con 0x55d8eec844a0 2024-03-29T04:18:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c12ffd640 1 --2- 172.21.15.22:0/1473361641 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec380b0 0x55d8eec3df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c0bfff640 1 --2- 172.21.15.22:0/1473361641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec844a0 0x55d8eec3e490 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc01c980 tx=0x7f0bfc0a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac4775245dca30 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0bf3fff640 1 -- 172.21.15.22:0/1473361641 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc0b1070 con 0x55d8eec844a0 2024-03-29T04:18:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0bf3fff640 1 -- 172.21.15.22:0/1473361641 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bfc000e50 con 0x55d8eec844a0 2024-03-29T04:18:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0bf3fff640 1 -- 172.21.15.22:0/1473361641 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc004230 con 0x55d8eec844a0 2024-03-29T04:18:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c1c41c000 1 -- 172.21.15.22:0/1473361641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec844a0 msgr2=0x55d8eec3e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c1c41c000 1 --2- 172.21.15.22:0/1473361641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec844a0 0x55d8eec3e490 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc01c980 tx=0x7f0bfc0a7000 comp rx=0 tx=0).stop 2024-03-29T04:18:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c1c41c000 1 -- 172.21.15.22:0/1473361641 shutdown_connections 2024-03-29T04:18:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c1c41c000 1 --2- 172.21.15.22:0/1473361641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8eec84dc0 0x55d8eec362d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c1c41c000 1 --2- 172.21.15.22:0/1473361641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec844a0 0x55d8eec3e490 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c1c41c000 1 --2- 172.21.15.22:0/1473361641 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec380b0 0x55d8eec3df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c1c41c000 1 -- 172.21.15.22:0/1473361641 >> 172.21.15.22:0/1473361641 conn(0x55d8eeb22b80 msgr2=0x55d8eec91a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c1c41c000 1 -- 172.21.15.22:0/1473361641 shutdown_connections 2024-03-29T04:18:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c1c41c000 1 -- 172.21.15.22:0/1473361641 wait complete. 2024-03-29T04:18:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c1c41c000 1 -- start start 2024-03-29T04:18:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c1c41c000 1 Processor -- start 2024-03-29T04:18:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c1c41c000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8eec380b0 0x55d8eecda050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c1c41c000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec844a0 0x55d8eec97a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c1c41c000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec84dc0 0x55d8eec97f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c1c41c000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d8eea30e80 con 0x55d8eec84dc0 2024-03-29T04:18:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c1c41c000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0c142038a0 con 0x55d8eec380b0 2024-03-29T04:18:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c1c41c000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x55d8eea2cf10 con 0x55d8eec844a0 2024-03-29T04:18:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c1c41c000 10 client.0 _ll_register_callbacks cb 0x7f0bfc0af070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:18:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c137fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec84dc0 0x55d8eec97f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c137fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec84dc0 0x55d8eec97f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49686/0 (socket says 172.21.15.22:49686) 2024-03-29T04:18:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c137fe640 1 -- 172.21.15.22:0/1984223237 learned_addr learned my addr 172.21.15.22:0/1984223237 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c12ffd640 1 --2- 172.21.15.22:0/1984223237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8eec380b0 0x55d8eecda050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c0bfff640 1 --2- 172.21.15.22:0/1984223237 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec844a0 0x55d8eec97a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c137fe640 1 -- 172.21.15.22:0/1984223237 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec844a0 msgr2=0x55d8eec97a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:04.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c137fe640 1 --2- 172.21.15.22:0/1984223237 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec844a0 0x55d8eec97a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c137fe640 1 -- 172.21.15.22:0/1984223237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8eec380b0 msgr2=0x55d8eecda050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:04.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c137fe640 1 --2- 172.21.15.22:0/1984223237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8eec380b0 0x55d8eecda050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c137fe640 1 -- 172.21.15.22:0/1984223237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8eea43e80 con 0x55d8eec84dc0 2024-03-29T04:18:04.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c137fe640 1 --2- 172.21.15.22:0/1984223237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec84dc0 0x55d8eec97f40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00c700 tx=0x7f0c0c05c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a589433a1716d3c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:04.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0bf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-29T04:18:04.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0bf1ffb640 1 -- 172.21.15.22:0/1984223237 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c055150 con 0x55d8eec84dc0 2024-03-29T04:18:04.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0bf1ffb640 1 -- 172.21.15.22:0/1984223237 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0c0c05c0b0 con 0x55d8eec84dc0 2024-03-29T04:18:04.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c1c41c000 1 -- 172.21.15.22:0/1984223237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bfc000e50 con 0x55d8eec84dc0 2024-03-29T04:18:04.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0bf1ffb640 1 -- 172.21.15.22:0/1984223237 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f0c0c05c2b0 con 0x55d8eec84dc0 2024-03-29T04:18:04.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c1c41c000 10 client.4857 fetch_fsmap learned FSMap version 9 2024-03-29T04:18:04.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.988+0000 7f0c1c41c000 1 -- 172.21.15.22:0/1984223237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55d8eea31270 con 0x55d8eec84dc0 2024-03-29T04:18:04.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0bf1ffb640 1 -- 172.21.15.22:0/1984223237 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c0c070020 con 0x55d8eec84dc0 2024-03-29T04:18:04.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0bf1ffb640 1 -- 172.21.15.22:0/1984223237 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f0c0c0741a0 con 0x55d8eec84dc0 2024-03-29T04:18:04.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1c41c000 10 client.4857 fetch_fsmap finished waiting for FSMap version 9 2024-03-29T04:18:04.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1c41c000 10 client.4857 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-29T04:18:04.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1c41c000 20 client.4857 populate_metadata read hostname 'smithi022' 2024-03-29T04:18:04.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1c41c000 1 --2- 172.21.15.22:0/1984223237 >> [v2:172.21.15.151:6836/1466916671,v1:172.21.15.151:6837/1466916671] conn(0x55d8eec3cb20 0x55d8eec98480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:04.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1c41c000 4 client.4857 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T04:18:04.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1c41c000 1 -- 172.21.15.22:0/1984223237 --> [v2:172.21.15.151:6836/1466916671,v1:172.21.15.151:6837/1466916671] -- command(tid 0: {"prefix": "status"}) -- 0x55d8eea97e90 con 0x55d8eec3cb20 2024-03-29T04:18:04.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c12ffd640 1 --2- 172.21.15.22:0/1984223237 >> [v2:172.21.15.151:6836/1466916671,v1:172.21.15.151:6837/1466916671] conn(0x55d8eec3cb20 0x55d8eec98480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:04.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0bf1ffb640 10 client.4857 ms_handle_connect on v2:172.21.15.151:6836/1466916671 2024-03-29T04:18:04.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c12ffd640 1 --2- 172.21.15.22:0/1984223237 >> [v2:172.21.15.151:6836/1466916671,v1:172.21.15.151:6837/1466916671] conn(0x55d8eec3cb20 0x55d8eec98480 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5f51cfa26b1a9cf server_cookie=57daf416f033b45d in_seq=0 out_seq=0 2024-03-29T04:18:04.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0bf1ffb640 1 -- 172.21.15.22:0/1984223237 <== mds.0 v2:172.21.15.151:6836/1466916671 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55d8eea97e90 con 0x55d8eec3cb20 2024-03-29T04:18:04.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0bf1ffb640 10 client.4857 handle_command_reply: tid=0 2024-03-29T04:18:04.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1c41c000 1 client.4857 shutdown 2024-03-29T04:18:04.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1c41c000 1 -- 172.21.15.22:0/1984223237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec84dc0 msgr2=0x55d8eec97f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:04.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1c41c000 1 --2- 172.21.15.22:0/1984223237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec84dc0 0x55d8eec97f40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00c700 tx=0x7f0c0c05c040 comp rx=0 tx=0).stop 2024-03-29T04:18:04.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1c41c000 1 -- 172.21.15.22:0/1984223237 shutdown_connections 2024-03-29T04:18:04.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1c41c000 1 --2- 172.21.15.22:0/1984223237 >> [v2:172.21.15.151:6836/1466916671,v1:172.21.15.151:6837/1466916671] conn(0x55d8eec3cb20 0x55d8eec98480 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1c41c000 1 --2- 172.21.15.22:0/1984223237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8eec84dc0 0x55d8eec97f40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1c41c000 1 --2- 172.21.15.22:0/1984223237 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x55d8eec844a0 0x55d8eec97a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1c41c000 1 --2- 172.21.15.22:0/1984223237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8eec380b0 0x55d8eecda050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0bf1ffb640 0 client.4857 ms_handle_reset on v2:172.21.15.151:6836/1466916671 2024-03-29T04:18:04.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1c41c000 1 -- 172.21.15.22:0/1984223237 >> 172.21.15.22:0/1984223237 conn(0x55d8eeb22b80 msgr2=0x55d8eec66b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:04.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1c41c000 1 -- 172.21.15.22:0/1984223237 shutdown_connections 2024-03-29T04:18:04.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1c41c000 1 -- 172.21.15.22:0/1984223237 wait complete. 2024-03-29T04:18:04.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1c41c000 20 client.4857 trim_cache size 0 max 16384 2024-03-29T04:18:04.981 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-29T04:18:04.981 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "9f2f9779-fe00-4763-9aa1-fd3768e2f135", 2024-03-29T04:18:04.981 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-03-29T04:18:04.981 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 4294, 2024-03-29T04:18:04.981 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-03-29T04:18:04.981 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-03-29T04:18:04.981 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-03-29T04:18:04.982 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 28.420708345000001, 2024-03-29T04:18:04.982 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 9, 2024-03-29T04:18:04.982 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 26, 2024-03-29T04:18:04.982 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 26, 2024-03-29T04:18:04.982 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 35.495066768999997 2024-03-29T04:18:04.982 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-29T04:18:04.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1929a640 1 -- 172.21.15.22:0/2526676972 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0bf4067fc0 msgr2=0x7f0bf406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:04.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1929a640 1 --2- 172.21.15.22:0/2526676972 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0bf4067fc0 0x7f0bf406a480 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0c1413e740 tx=0x7f0c04009040 comp rx=0 tx=0).stop 2024-03-29T04:18:04.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1929a640 1 -- 172.21.15.22:0/2526676972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c140b41f0 msgr2=0x7f0c140722f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:04.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1929a640 1 --2- 172.21.15.22:0/2526676972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c140b41f0 0x7f0c140722f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00c730 tx=0x7f0c0c00cbf0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1929a640 1 -- 172.21.15.22:0/2526676972 shutdown_connections 2024-03-29T04:18:04.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1929a640 1 --2- 172.21.15.22:0/2526676972 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0bf4067fc0 0x7f0bf406a480 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1929a640 1 --2- 172.21.15.22:0/2526676972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c140b41f0 0x7f0c140722f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1929a640 1 --2- 172.21.15.22:0/2526676972 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0c1406ca50 0x7f0c14071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1929a640 1 --2- 172.21.15.22:0/2526676972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c1406c670 0x7f0c1415e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:04.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1929a640 1 -- 172.21.15.22:0/2526676972 >> 172.21.15.22:0/2526676972 conn(0x7f0c140cfee0 msgr2=0x7f0c140ce5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:04.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1929a640 1 -- 172.21.15.22:0/2526676972 shutdown_connections 2024-03-29T04:18:04.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:04.992+0000 7f0c1929a640 1 -- 172.21.15.22:0/2526676972 wait complete. 2024-03-29T04:18:04.990 INFO:teuthology.run_tasks:Running task sequential... 2024-03-29T04:18:04.999 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-03-29T04:18:04.999 INFO:teuthology.task.sequential:In sequential, running task print... 2024-03-29T04:18:05.000 INFO:teuthology.task.print:Enabling mgr modules 2024-03-29T04:18:05.000 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-29T04:18:05.015 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-29T04:18:05.015 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-29T04:18:05.016 INFO:tasks.ceph_fuse:client.0 config is {} 2024-03-29T04:18:05.016 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T04:18:05.016 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T04:18:05.016 INFO:tasks.ceph_fuse:client.2 config is {} 2024-03-29T04:18:05.016 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T04:18:05.017 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-29T04:18:05.017 INFO:tasks.ceph_fuse:client.1 config is {} 2024-03-29T04:18:05.017 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T04:18:05.017 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-29T04:18:05.017 INFO:tasks.ceph_fuse:client.3 config is {} 2024-03-29T04:18:05.018 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T04:18:05.018 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-29T04:18:05.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:05.018 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-03-29T04:18:05.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:05.026 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link delete ceph-brx 2024-03-29T04:18:05.086 INFO:teuthology.orchestra.run.smithi151.stderr:Cannot find device "ceph-brx" 2024-03-29T04:18:05.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:18:05.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:05.087 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-03-29T04:18:05.094 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-29T04:18:05.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:05.094 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T04:18:05.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:05.150 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2024-03-29T04:18:05.338 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-29T04:18:05.338 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:18:05.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.484+0000 7fc0c478e640 1 Processor -- start 2024-03-29T04:18:05.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.488+0000 7fc0c478e640 1 -- start start 2024-03-29T04:18:05.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.488+0000 7fc0c478e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc0bc15f790 0x7fc0bc15ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:05.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.488+0000 7fc0c478e640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0bc0725d0 0x7fc0bc072a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:05.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.488+0000 7fc0c478e640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0bc0b5ba0 con 0x7fc0bc161cb0 2024-03-29T04:18:05.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.488+0000 7fc0c478e640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fc0bc0cee60 con 0x7fc0bc15f790 2024-03-29T04:18:05.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.488+0000 7fc0c478e640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc0bc0b5e80 con 0x7fc0bc0725d0 2024-03-29T04:18:05.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.488+0000 7fc0c1d29640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc0bc15f790 0x7fc0bc15ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:05.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.488+0000 7fc0c252a640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fc0bc161cb0 0x7fc0bc0b3f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36286/0 (socket says 172.21.15.22:36286) 2024-03-29T04:18:05.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.488+0000 7fc0c252a640 1 -- 172.21.15.22:0/1580945016 learned_addr learned my addr 172.21.15.22:0/1580945016 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:05.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.488+0000 7fc0c1d29640 1 -- 172.21.15.22:0/1580945016 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0bc0725d0 msgr2=0x7fc0bc072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:05.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.488+0000 7fc0c1d29640 1 --2- 172.21.15.22:0/1580945016 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0bc0725d0 0x7fc0bc072a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:05.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.488+0000 7fc0c1d29640 1 -- 172.21.15.22:0/1580945016 >> v1:172.21.15.22:6789/0 conn(0x7fc0bc161cb0 legacy=0x7fc0bc0b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:05.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.488+0000 7fc0c1d29640 1 -- 172.21.15.22:0/1580945016 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0bc0b6160 con 0x7fc0bc15f790 2024-03-29T04:18:05.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.488+0000 7fc0c1d29640 1 --2- 172.21.15.22:0/1580945016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0bc15f790 0x7fc0bc15ff90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80097c0 tx=0x7fc0b801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ae99f547422eeb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:05.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.488+0000 7fc0c1528640 1 -- 172.21.15.22:0/1580945016 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b80ab430 con 0x7fc0bc15f790 2024-03-29T04:18:05.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.488+0000 7fc0c1528640 1 -- 172.21.15.22:0/1580945016 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b80b0070 con 0x7fc0bc15f790 2024-03-29T04:18:05.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.488+0000 7fc0c1528640 1 -- 172.21.15.22:0/1580945016 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b80ab7f0 con 0x7fc0bc15f790 2024-03-29T04:18:05.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c478e640 1 -- 172.21.15.22:0/1580945016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0bc15f790 msgr2=0x7fc0bc15ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:05.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c478e640 1 --2- 172.21.15.22:0/1580945016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0bc15f790 0x7fc0bc15ff90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80097c0 tx=0x7fc0b801c410 comp rx=0 tx=0).stop 2024-03-29T04:18:05.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c478e640 1 -- 172.21.15.22:0/1580945016 shutdown_connections 2024-03-29T04:18:05.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c478e640 1 --2- 172.21.15.22:0/1580945016 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0bc0725d0 0x7fc0bc072a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:05.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c478e640 1 --2- 172.21.15.22:0/1580945016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0bc15f790 0x7fc0bc15ff90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:05.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c478e640 1 -- 172.21.15.22:0/1580945016 >> 172.21.15.22:0/1580945016 conn(0x7fc0bc0d0f60 msgr2=0x7fc0bc0d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:05.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c478e640 1 -- 172.21.15.22:0/1580945016 shutdown_connections 2024-03-29T04:18:05.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c478e640 1 -- 172.21.15.22:0/1580945016 wait complete. 2024-03-29T04:18:05.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c478e640 1 Processor -- start 2024-03-29T04:18:05.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c478e640 1 -- start start 2024-03-29T04:18:05.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c478e640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0bc0725d0 0x7fc0bc14ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:05.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c478e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0bc15f790 0x7fc0bc152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:05.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c478e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0bc161cb0 0x7fc0bc1529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:05.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c478e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc0cf5d0 con 0x7fc0bc161cb0 2024-03-29T04:18:05.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c478e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc0cf750 con 0x7fc0bc15f790 2024-03-29T04:18:05.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c478e640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc0bc0cf8d0 con 0x7fc0bc0725d0 2024-03-29T04:18:05.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c2d2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0bc161cb0 0x7fc0bc1529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:05.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c2d2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0bc161cb0 0x7fc0bc1529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49712/0 (socket says 172.21.15.22:49712) 2024-03-29T04:18:05.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c2d2b640 1 -- 172.21.15.22:0/2978267906 learned_addr learned my addr 172.21.15.22:0/2978267906 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:05.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c1d29640 1 --2- 172.21.15.22:0/2978267906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0bc15f790 0x7fc0bc152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:05.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c252a640 1 --2- 172.21.15.22:0/2978267906 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0bc0725d0 0x7fc0bc14ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:05.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c2d2b640 1 -- 172.21.15.22:0/2978267906 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0bc0725d0 msgr2=0x7fc0bc14ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:05.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c2d2b640 1 --2- 172.21.15.22:0/2978267906 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0bc0725d0 0x7fc0bc14ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:05.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c2d2b640 1 -- 172.21.15.22:0/2978267906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0bc15f790 msgr2=0x7fc0bc152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:05.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c2d2b640 1 --2- 172.21.15.22:0/2978267906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0bc15f790 0x7fc0bc152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:05.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c2d2b640 1 -- 172.21.15.22:0/2978267906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0bc14f880 con 0x7fc0bc161cb0 2024-03-29T04:18:05.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c1d29640 1 --2- 172.21.15.22:0/2978267906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0bc15f790 0x7fc0bc152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:05.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c2d2b640 1 --2- 172.21.15.22:0/2978267906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0bc161cb0 0x7fc0bc1529d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400b9a0 tx=0x7fc0b400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:05.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0b2ffd640 1 -- 172.21.15.22:0/2978267906 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4019070 con 0x7fc0bc161cb0 2024-03-29T04:18:05.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0b2ffd640 1 -- 172.21.15.22:0/2978267906 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b400cbc0 con 0x7fc0bc161cb0 2024-03-29T04:18:05.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c478e640 1 -- 172.21.15.22:0/2978267906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0bc14fb60 con 0x7fc0bc161cb0 2024-03-29T04:18:05.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0b2ffd640 1 -- 172.21.15.22:0/2978267906 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4014440 con 0x7fc0bc161cb0 2024-03-29T04:18:05.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.492+0000 7fc0c478e640 1 -- 172.21.15.22:0/2978267906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0bc150120 con 0x7fc0bc161cb0 2024-03-29T04:18:05.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.496+0000 7fc0c478e640 1 -- 172.21.15.22:0/2978267906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0bc068890 con 0x7fc0bc161cb0 2024-03-29T04:18:05.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.496+0000 7fc0b2ffd640 1 -- 172.21.15.22:0/2978267906 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fc0b401c070 con 0x7fc0bc161cb0 2024-03-29T04:18:05.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.496+0000 7fc0b2ffd640 1 --2- 172.21.15.22:0/2978267906 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc0a0067fc0 0x7fc0a006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:05.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.496+0000 7fc0b2ffd640 1 -- 172.21.15.22:0/2978267906 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc0b40865c0 con 0x7fc0bc161cb0 2024-03-29T04:18:05.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.496+0000 7fc0c252a640 1 --2- 172.21.15.22:0/2978267906 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc0a0067fc0 0x7fc0a006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:05.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.496+0000 7fc0c252a640 1 --2- 172.21.15.22:0/2978267906 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc0a0067fc0 0x7fc0a006a480 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac000f70 tx=0x7fc0ac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:05.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.500+0000 7fc0b2ffd640 1 -- 172.21.15.22:0/2978267906 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fc0b4056a30 con 0x7fc0bc161cb0 2024-03-29T04:18:05.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.704+0000 7fc0c478e640 1 -- 172.21.15.22:0/2978267906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc0bc0ceb40 con 0x7fc0bc161cb0 2024-03-29T04:18:05.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.704+0000 7fc0b2ffd640 1 -- 172.21.15.22:0/2978267906 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fc0b4056330 con 0x7fc0bc161cb0 2024-03-29T04:18:05.684 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-29T04:18:05.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.704+0000 7fc0c478e640 1 -- 172.21.15.22:0/2978267906 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc0a0067fc0 msgr2=0x7fc0a006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:05.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.704+0000 7fc0c478e640 1 --2- 172.21.15.22:0/2978267906 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc0a0067fc0 0x7fc0a006a480 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac000f70 tx=0x7fc0ac009040 comp rx=0 tx=0).stop 2024-03-29T04:18:05.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.704+0000 7fc0c478e640 1 -- 172.21.15.22:0/2978267906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0bc161cb0 msgr2=0x7fc0bc1529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:05.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.704+0000 7fc0c478e640 1 --2- 172.21.15.22:0/2978267906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0bc161cb0 0x7fc0bc1529d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400b9a0 tx=0x7fc0b400be60 comp rx=0 tx=0).stop 2024-03-29T04:18:05.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.704+0000 7fc0c478e640 1 -- 172.21.15.22:0/2978267906 shutdown_connections 2024-03-29T04:18:05.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.704+0000 7fc0c478e640 1 --2- 172.21.15.22:0/2978267906 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc0a0067fc0 0x7fc0a006a480 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:05.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.704+0000 7fc0c478e640 1 --2- 172.21.15.22:0/2978267906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0bc161cb0 0x7fc0bc1529d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:05.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.704+0000 7fc0c478e640 1 --2- 172.21.15.22:0/2978267906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0bc15f790 0x7fc0bc152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:05.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.704+0000 7fc0c478e640 1 --2- 172.21.15.22:0/2978267906 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc0bc0725d0 0x7fc0bc14ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:05.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.704+0000 7fc0c478e640 1 -- 172.21.15.22:0/2978267906 >> 172.21.15.22:0/2978267906 conn(0x7fc0bc0d0f60 msgr2=0x7fc0bc0b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:05.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.704+0000 7fc0c478e640 1 -- 172.21.15.22:0/2978267906 shutdown_connections 2024-03-29T04:18:05.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05.704+0000 7fc0c478e640 1 -- 172.21.15.22:0/2978267906 wait complete. 2024-03-29T04:18:05.703 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:18:05.703 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:18:05.703 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-03-29T04:18:05.703 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T04:18:05.703 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T04:18:05.704 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:18:05.704 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:18:05.704 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:18:05.704 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:18:05.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:05.704 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-03-29T04:18:05.709 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:18:05.709 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:18:05.709 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:18:05.709 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:05.710 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-03-29T04:18:05.710 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:05.710 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T04:18:05.710 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:05.710 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-03-29T04:18:05.710 INFO:teuthology.orchestra.run.smithi022.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:18:05.710 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:05.710 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-03-29T04:18:05.710 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-29T04:18:05.710 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42458sec preferred_lft 42458sec 2024-03-29T04:18:05.711 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-03-29T04:18:05.711 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:05.711 INFO:teuthology.orchestra.run.smithi022.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T04:18:05.711 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:05.711 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-03-29T04:18:05.711 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T04:18:05.711 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:05.711 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-03-29T04:18:05.711 INFO:teuthology.orchestra.run.smithi022.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T04:18:05.711 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 02:42:df:ac:77:fb brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:05.711 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T04:18:05.711 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:05.711 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-29T04:18:05.712 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:18:05.712 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-29T04:18:05.712 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add name ceph-brx type bridge 2024-03-29T04:18:05.712 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr flush dev ceph-brx 2024-03-29T04:18:05.712 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set ceph-brx up 2024-03-29T04:18:05.712 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-29T04:18:05.712 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-29T04:18:05.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:18:05.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:05.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:05.837 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:18:05.845 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-03-29T04:18:05.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:05.847 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-03-29T04:18:05.896 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-03-29T04:18:05.897 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:18:05.897 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-29T04:18:05.897 INFO:teuthology.orchestra.run.smithi022.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T04:18:05.897 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-29T04:18:05.897 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:18:05.898 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:18:05.898 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-29T04:18:05.898 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T04:18:05.898 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T04:18:05.898 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T04:18:05.898 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-29T04:18:05.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:18:06.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:06.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:06.020 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-03-29T04:18:06.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:06.026 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list-id 2024-03-29T04:18:06.073 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:18:06.074 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-29T04:18:06.074 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:18:06.074 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-29T04:18:06.074 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-29T04:18:06.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:18:06.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:06.189 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-29T04:18:06.189 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:18:06.189 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-29T04:18:06.189 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-29T04:18:06.189 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-29T04:18:06.190 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-29T04:18:06.190 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-29T04:18:06.190 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-29T04:18:06.190 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-29T04:18:06.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:18:06.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:06.328 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:18:06.329 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-29T04:18:06.329 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-03-29T04:18:06.329 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set dev brx.0 master ceph-brx 2024-03-29T04:18:06.329 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-29T04:18:06.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:18:06.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:06.386 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T04:18:06.386 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:18:06.386 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T04:18:06.431 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:18:06.432 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:18:06.432 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T04:18:06.479 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-03-29T04:18:06.537 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:18:06.585 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-03-29T04:18:06.585 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-03-29T04:18:06.585 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-03-29T04:18:06.585 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-03-29T04:18:06.585 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-03-29T04:18:06.585 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-03-29T04:18:06.586 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-03-29T04:18:06.586 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-03-29T04:18:06.586 INFO:teuthology.orchestra.run.smithi022.stdout:/run/lock 2024-03-29T04:18:06.586 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-03-29T04:18:06.586 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-03-29T04:18:06.586 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-03-29T04:18:06.586 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:18:06.586 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-03-29T04:18:06.586 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-03-29T04:18:06.586 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-03-29T04:18:06.586 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-03-29T04:18:06.586 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/fs/nfsd 2024-03-29T04:18:06.586 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-03-29T04:18:06.586 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-03-29T04:18:06.586 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysusers.service 2024-03-29T04:18:06.587 INFO:teuthology.orchestra.run.smithi022.stdout:/run/rpc_pipefs 2024-03-29T04:18:06.587 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-03-29T04:18:06.587 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-03-29T04:18:06.587 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:18:06.587 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:18:06.587 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:18:06.587 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:18:06.587 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-03-29T04:18:06.587 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:18:06.587 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:18:06.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:06.588 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-03-29T04:18:06.636 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T04:18:06.636 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T04:18:06.681 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-03-29T04:18:06.695 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:18:06.748 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-03-29T04:18:06.748 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-03-29T04:18:06.748 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-03-29T04:18:06.748 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-03-29T04:18:06.748 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-03-29T04:18:06.748 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-03-29T04:18:06.748 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-03-29T04:18:06.748 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-03-29T04:18:06.748 INFO:teuthology.orchestra.run.smithi022.stdout:/run/lock 2024-03-29T04:18:06.748 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-03-29T04:18:06.748 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-03-29T04:18:06.748 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-03-29T04:18:06.749 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:18:06.749 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-03-29T04:18:06.749 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-03-29T04:18:06.749 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-03-29T04:18:06.749 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-03-29T04:18:06.749 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/fs/nfsd 2024-03-29T04:18:06.749 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-03-29T04:18:06.749 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-03-29T04:18:06.749 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysusers.service 2024-03-29T04:18:06.749 INFO:teuthology.orchestra.run.smithi022.stdout:/run/rpc_pipefs 2024-03-29T04:18:06.749 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-03-29T04:18:06.749 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-03-29T04:18:06.749 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:18:06.749 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:18:06.750 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:18:06.750 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:18:06.750 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-03-29T04:18:06.750 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:18:06.750 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:18:06.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:06.751 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-03-29T04:18:06.764 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:2024-03-29T04:18:06.784+0000 7ff8829ab580 -1 init, newargv = 0x55dc5482fcd0 newargc=15 2024-03-29T04:18:06.764 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[35151]: starting ceph client 2024-03-29T04:18:06.771 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[35151]: starting fuse 2024-03-29T04:18:07.870 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-03-29T04:18:07.886 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:18:07.933 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-03-29T04:18:07.934 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-03-29T04:18:07.934 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-03-29T04:18:07.934 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-03-29T04:18:07.934 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-03-29T04:18:07.934 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-03-29T04:18:07.934 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-03-29T04:18:07.934 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-03-29T04:18:07.934 INFO:teuthology.orchestra.run.smithi022.stdout:/run/lock 2024-03-29T04:18:07.934 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-03-29T04:18:07.934 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-03-29T04:18:07.934 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-03-29T04:18:07.934 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:18:07.934 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-03-29T04:18:07.935 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-03-29T04:18:07.935 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-03-29T04:18:07.935 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-03-29T04:18:07.935 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/fs/nfsd 2024-03-29T04:18:07.935 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-03-29T04:18:07.935 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-03-29T04:18:07.935 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysusers.service 2024-03-29T04:18:07.935 INFO:teuthology.orchestra.run.smithi022.stdout:/run/rpc_pipefs 2024-03-29T04:18:07.935 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-03-29T04:18:07.935 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-03-29T04:18:07.935 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:18:07.935 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:18:07.935 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:18:07.935 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:18:07.936 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-03-29T04:18:07.936 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:18:07.936 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:18:07.936 INFO:teuthology.orchestra.run.smithi022.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-29T04:18:07.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:07.937 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-03-29T04:18:07.988 INFO:teuthology.orchestra.run.smithi022.stdout:45 2024-03-29T04:18:07.989 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-03-29T04:18:07.989 DEBUG:teuthology.orchestra.run.smithi022:> sudo stdin-killer -- python3 -c ' 2024-03-29T04:18:07.989 DEBUG:teuthology.orchestra.run.smithi022:> import glob 2024-03-29T04:18:07.989 DEBUG:teuthology.orchestra.run.smithi022:> import re 2024-03-29T04:18:07.989 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-03-29T04:18:07.989 DEBUG:teuthology.orchestra.run.smithi022:> import subprocess 2024-03-29T04:18:07.989 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-29T04:18:07.989 DEBUG:teuthology.orchestra.run.smithi022:> def _find_admin_socket(client_name): 2024-03-29T04:18:07.989 DEBUG:teuthology.orchestra.run.smithi022:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T04:18:07.989 DEBUG:teuthology.orchestra.run.smithi022:> files = glob.glob(asok_path) 2024-03-29T04:18:07.989 DEBUG:teuthology.orchestra.run.smithi022:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T04:18:07.989 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-29T04:18:07.990 DEBUG:teuthology.orchestra.run.smithi022:> # Given a non-glob path, it better be there 2024-03-29T04:18:07.990 DEBUG:teuthology.orchestra.run.smithi022:> if "*" not in asok_path: 2024-03-29T04:18:07.990 DEBUG:teuthology.orchestra.run.smithi022:> assert(len(files) == 1) 2024-03-29T04:18:07.990 DEBUG:teuthology.orchestra.run.smithi022:> return files[0] 2024-03-29T04:18:07.990 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-29T04:18:07.990 DEBUG:teuthology.orchestra.run.smithi022:> for f in files: 2024-03-29T04:18:07.990 DEBUG:teuthology.orchestra.run.smithi022:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T04:18:07.990 DEBUG:teuthology.orchestra.run.smithi022:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T04:18:07.990 DEBUG:teuthology.orchestra.run.smithi022:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T04:18:07.990 DEBUG:teuthology.orchestra.run.smithi022:> contents = proc_f.read() 2024-03-29T04:18:07.990 DEBUG:teuthology.orchestra.run.smithi022:> if mountpoint in contents: 2024-03-29T04:18:07.990 DEBUG:teuthology.orchestra.run.smithi022:> return f 2024-03-29T04:18:07.990 DEBUG:teuthology.orchestra.run.smithi022:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T04:18:07.990 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-29T04:18:07.990 DEBUG:teuthology.orchestra.run.smithi022:> print(_find_admin_socket("client.0")) 2024-03-29T04:18:07.990 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-03-29T04:18:08.114 INFO:teuthology.orchestra.run.smithi022.stdout:/var/run/ceph/ceph-client.0.35151.asok 2024-03-29T04:18:08.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:08.127 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35151.asok 2024-03-29T04:18:08.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:08.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.35151.asok status 2024-03-29T04:18:08.284 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-29T04:18:08.284 INFO:teuthology.orchestra.run.smithi022.stdout: "metadata": { 2024-03-29T04:18:08.284 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_sha1": "69e2cd73c52284dfd5096c66b60c6e6b26a3c447", 2024-03-29T04:18:08.284 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 19.0.0-2520-g69e2cd73 (69e2cd73c52284dfd5096c66b60c6e6b26a3c447) squid (dev)", 2024-03-29T04:18:08.284 INFO:teuthology.orchestra.run.smithi022.stdout: "entity_id": "0", 2024-03-29T04:18:08.284 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": "smithi022", 2024-03-29T04:18:08.285 INFO:teuthology.orchestra.run.smithi022.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T04:18:08.285 INFO:teuthology.orchestra.run.smithi022.stdout: "pid": "35151", 2024-03-29T04:18:08.285 INFO:teuthology.orchestra.run.smithi022.stdout: "root": "/" 2024-03-29T04:18:08.285 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:08.285 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_count": 0, 2024-03-29T04:18:08.285 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_pinned_count": 0, 2024-03-29T04:18:08.285 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 4872, 2024-03-29T04:18:08.285 INFO:teuthology.orchestra.run.smithi022.stdout: "inst": { 2024-03-29T04:18:08.285 INFO:teuthology.orchestra.run.smithi022.stdout: "name": { 2024-03-29T04:18:08.285 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "client", 2024-03-29T04:18:08.285 INFO:teuthology.orchestra.run.smithi022.stdout: "num": 4872 2024-03-29T04:18:08.285 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2024-03-29T04:18:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-03-29T04:18:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:18:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4132012671 2024-03-29T04:18:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2024-03-29T04:18:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-03-29T04:18:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:18:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4132012671 2024-03-29T04:18:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: "inst_str": "client.4872 192.168.0.1:0/4132012671", 2024-03-29T04:18:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: "addr_str": "192.168.0.1:0/4132012671", 2024-03-29T04:18:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: "inode_count": 1, 2024-03-29T04:18:08.287 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_epoch": 9, 2024-03-29T04:18:08.287 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch": 26, 2024-03-29T04:18:08.287 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch_barrier": 0, 2024-03-29T04:18:08.287 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklisted": false, 2024-03-29T04:18:08.287 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs" 2024-03-29T04:18:08.287 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-29T04:18:08.298 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:18:08.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.444+0000 7efe8f780640 1 Processor -- start 2024-03-29T04:18:08.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.444+0000 7efe8f780640 1 -- start start 2024-03-29T04:18:08.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.444+0000 7efe8f780640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efe8806c670 0x7efe880cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:08.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.444+0000 7efe8f780640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efe8806ca50 0x7efe880ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:08.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.444+0000 7efe8f780640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe880ccef0 con 0x7efe880b41f0 2024-03-29T04:18:08.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.444+0000 7efe8f780640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7efe880b34c0 con 0x7efe8806ca50 2024-03-29T04:18:08.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.444+0000 7efe8f780640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7efe880b3710 con 0x7efe8806c670 2024-03-29T04:18:08.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.444+0000 7efe8dd1d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efe8806ca50 0x7efe880ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:08.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.444+0000 7efe8dd1d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efe8806ca50 0x7efe880ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49728/0 (socket says 172.21.15.22:49728) 2024-03-29T04:18:08.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.444+0000 7efe8dd1d640 1 -- 172.21.15.22:0/2601804845 learned_addr learned my addr 172.21.15.22:0/2601804845 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:08.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.444+0000 7efe8d51c640 1 --2- 172.21.15.22:0/2601804845 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efe8806c670 0x7efe880cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:08.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.444+0000 7efe8dd1d640 1 -- 172.21.15.22:0/2601804845 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efe8806c670 msgr2=0x7efe880cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:08.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.444+0000 7efe8dd1d640 1 --2- 172.21.15.22:0/2601804845 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efe8806c670 0x7efe880cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:08.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.444+0000 7efe8dd1d640 1 -- 172.21.15.22:0/2601804845 >> v1:172.21.15.151:6789/0 conn(0x7efe880b41f0 legacy=0x7efe8806e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:08.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.444+0000 7efe8dd1d640 1 -- 172.21.15.22:0/2601804845 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe880b39f0 con 0x7efe8806ca50 2024-03-29T04:18:08.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8dd1d640 1 --2- 172.21.15.22:0/2601804845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe8806ca50 0x7efe880ce5b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efe80009e50 tx=0x7efe8001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8bbfe70198da70e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:08.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe7ffff640 1 -- 172.21.15.22:0/2601804845 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe800a5810 con 0x7efe8806ca50 2024-03-29T04:18:08.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe7ffff640 1 -- 172.21.15.22:0/2601804845 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe800a59b0 con 0x7efe8806ca50 2024-03-29T04:18:08.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe7ffff640 1 -- 172.21.15.22:0/2601804845 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe800a5d80 con 0x7efe8806ca50 2024-03-29T04:18:08.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8f780640 1 -- 172.21.15.22:0/2601804845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe8806ca50 msgr2=0x7efe880ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:08.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8f780640 1 --2- 172.21.15.22:0/2601804845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe8806ca50 0x7efe880ce5b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efe80009e50 tx=0x7efe8001ca30 comp rx=0 tx=0).stop 2024-03-29T04:18:08.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8f780640 1 -- 172.21.15.22:0/2601804845 shutdown_connections 2024-03-29T04:18:08.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8f780640 1 --2- 172.21.15.22:0/2601804845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe8806ca50 0x7efe880ce5b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:08.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8f780640 1 --2- 172.21.15.22:0/2601804845 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efe8806c670 0x7efe880cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:08.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8f780640 1 -- 172.21.15.22:0/2601804845 >> 172.21.15.22:0/2601804845 conn(0x7efe880cfee0 msgr2=0x7efe880d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:08.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8f780640 1 -- 172.21.15.22:0/2601804845 shutdown_connections 2024-03-29T04:18:08.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8f780640 1 -- 172.21.15.22:0/2601804845 wait complete. 2024-03-29T04:18:08.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8f780640 1 Processor -- start 2024-03-29T04:18:08.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8f780640 1 -- start start 2024-03-29T04:18:08.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8f780640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe8806c670 0x7efe8813fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:08.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8f780640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efe8806ca50 0x7efe8815e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:08.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8f780640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe880b41f0 0x7efe88144a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:08.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8f780640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efe88169070 con 0x7efe880b41f0 2024-03-29T04:18:08.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8f780640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efe88168ef0 con 0x7efe8806c670 2024-03-29T04:18:08.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8f780640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7efe881691f0 con 0x7efe8806ca50 2024-03-29T04:18:08.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8dd1d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe880b41f0 0x7efe88144a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:08.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8dd1d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe880b41f0 0x7efe88144a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49740/0 (socket says 172.21.15.22:49740) 2024-03-29T04:18:08.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8dd1d640 1 -- 172.21.15.22:0/2373266495 learned_addr learned my addr 172.21.15.22:0/2373266495 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:08.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8cd1b640 1 --2- 172.21.15.22:0/2373266495 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efe8806ca50 0x7efe8815e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:08.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8d51c640 1 --2- 172.21.15.22:0/2373266495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe8806c670 0x7efe8813fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:08.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8dd1d640 1 -- 172.21.15.22:0/2373266495 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efe8806ca50 msgr2=0x7efe8815e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:08.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8dd1d640 1 --2- 172.21.15.22:0/2373266495 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efe8806ca50 0x7efe8815e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:08.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8dd1d640 1 -- 172.21.15.22:0/2373266495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe8806c670 msgr2=0x7efe8813fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:08.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8dd1d640 1 --2- 172.21.15.22:0/2373266495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe8806c670 0x7efe8813fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:08.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8dd1d640 1 -- 172.21.15.22:0/2373266495 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe88140130 con 0x7efe880b41f0 2024-03-29T04:18:08.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.448+0000 7efe8dd1d640 1 --2- 172.21.15.22:0/2373266495 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe880b41f0 0x7efe88144a30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efe800af040 tx=0x7efe800197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:08.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.452+0000 7efe7dffb640 1 -- 172.21.15.22:0/2373266495 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe800bc020 con 0x7efe880b41f0 2024-03-29T04:18:08.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.452+0000 7efe7dffb640 1 -- 172.21.15.22:0/2373266495 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe800b8670 con 0x7efe880b41f0 2024-03-29T04:18:08.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.452+0000 7efe8f780640 1 -- 172.21.15.22:0/2373266495 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe88140410 con 0x7efe880b41f0 2024-03-29T04:18:08.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.452+0000 7efe7dffb640 1 -- 172.21.15.22:0/2373266495 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe800b8a60 con 0x7efe880b41f0 2024-03-29T04:18:08.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.452+0000 7efe8f780640 1 -- 172.21.15.22:0/2373266495 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe881409d0 con 0x7efe880b41f0 2024-03-29T04:18:08.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.452+0000 7efe8f780640 1 -- 172.21.15.22:0/2373266495 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe88068890 con 0x7efe880b41f0 2024-03-29T04:18:08.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.452+0000 7efe7dffb640 1 -- 172.21.15.22:0/2373266495 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7efe800bf020 con 0x7efe880b41f0 2024-03-29T04:18:08.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.452+0000 7efe7dffb640 1 --2- 172.21.15.22:0/2373266495 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efe6c067fc0 0x7efe6c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:08.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.452+0000 7efe7dffb640 1 -- 172.21.15.22:0/2373266495 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7efe80121810 con 0x7efe880b41f0 2024-03-29T04:18:08.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.452+0000 7efe8d51c640 1 --2- 172.21.15.22:0/2373266495 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efe6c067fc0 0x7efe6c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:08.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.452+0000 7efe8d51c640 1 --2- 172.21.15.22:0/2373266495 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efe6c067fc0 0x7efe6c06a480 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efe88069040 tx=0x7efe84008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:08.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.456+0000 7efe7dffb640 1 -- 172.21.15.22:0/2373266495 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7efe800c6030 con 0x7efe880b41f0 2024-03-29T04:18:08.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.660+0000 7efe8f780640 1 -- 172.21.15.22:0/2373266495 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efe880b32a0 con 0x7efe880b41f0 2024-03-29T04:18:08.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.660+0000 7efe7dffb640 1 -- 172.21.15.22:0/2373266495 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7efe8011fe20 con 0x7efe880b41f0 2024-03-29T04:18:08.642 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-29T04:18:08.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.664+0000 7efe8f780640 1 -- 172.21.15.22:0/2373266495 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efe6c067fc0 msgr2=0x7efe6c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:08.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.664+0000 7efe8f780640 1 --2- 172.21.15.22:0/2373266495 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efe6c067fc0 0x7efe6c06a480 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efe88069040 tx=0x7efe84008040 comp rx=0 tx=0).stop 2024-03-29T04:18:08.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.664+0000 7efe8f780640 1 -- 172.21.15.22:0/2373266495 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe880b41f0 msgr2=0x7efe88144a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:08.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.664+0000 7efe8f780640 1 --2- 172.21.15.22:0/2373266495 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe880b41f0 0x7efe88144a30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efe800af040 tx=0x7efe800197d0 comp rx=0 tx=0).stop 2024-03-29T04:18:08.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.664+0000 7efe8f780640 1 -- 172.21.15.22:0/2373266495 shutdown_connections 2024-03-29T04:18:08.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.664+0000 7efe8f780640 1 --2- 172.21.15.22:0/2373266495 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efe6c067fc0 0x7efe6c06a480 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:08.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.664+0000 7efe8f780640 1 --2- 172.21.15.22:0/2373266495 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe880b41f0 0x7efe88144a30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:08.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.664+0000 7efe8f780640 1 --2- 172.21.15.22:0/2373266495 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efe8806ca50 0x7efe8815e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:08.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.664+0000 7efe8f780640 1 --2- 172.21.15.22:0/2373266495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe8806c670 0x7efe8813fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:08.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.664+0000 7efe8f780640 1 -- 172.21.15.22:0/2373266495 >> 172.21.15.22:0/2373266495 conn(0x7efe880cfee0 msgr2=0x7efe881437d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:08.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.664+0000 7efe8f780640 1 -- 172.21.15.22:0/2373266495 shutdown_connections 2024-03-29T04:18:08.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08.664+0000 7efe8f780640 1 -- 172.21.15.22:0/2373266495 wait complete. 2024-03-29T04:18:08.663 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:18:08.663 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:18:08.663 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-03-29T04:18:08.663 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-29T04:18:08.663 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-29T04:18:08.663 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:18:08.663 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:18:08.663 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:18:08.663 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-29T04:18:08.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:08.664 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-03-29T04:18:08.670 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:18:08.670 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:18:08.671 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:18:08.671 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:08.671 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-03-29T04:18:08.671 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:08.671 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T04:18:08.671 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:08.671 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-03-29T04:18:08.671 INFO:teuthology.orchestra.run.smithi022.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:18:08.671 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:08.671 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-03-29T04:18:08.671 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-29T04:18:08.671 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42455sec preferred_lft 42455sec 2024-03-29T04:18:08.671 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-03-29T04:18:08.671 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:08.672 INFO:teuthology.orchestra.run.smithi022.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T04:18:08.672 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:08.672 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-03-29T04:18:08.672 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T04:18:08.672 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:08.672 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-03-29T04:18:08.672 INFO:teuthology.orchestra.run.smithi022.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T04:18:08.672 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 02:42:df:ac:77:fb brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:08.672 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T04:18:08.672 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:08.672 INFO:teuthology.orchestra.run.smithi022.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:18:08.672 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:08.672 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T04:18:08.672 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:08.672 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-29T04:18:08.673 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:08.673 INFO:teuthology.orchestra.run.smithi022.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:18:08.673 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:18:08.673 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-29T04:18:08.673 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:08.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:08.673 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:18:08.727 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-03-29T04:18:08.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:08.728 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-03-29T04:18:08.778 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-03-29T04:18:08.778 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:18:08.778 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-29T04:18:08.778 INFO:teuthology.orchestra.run.smithi022.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T04:18:08.778 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-29T04:18:08.778 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:18:08.779 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:18:08.779 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-29T04:18:08.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T04:18:08.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T04:18:08.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T04:18:08.779 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-29T04:18:08.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:18:08.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:08.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:08.905 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-03-29T04:18:08.911 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T04:18:08.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:08.912 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list-id 2024-03-29T04:18:08.960 INFO:teuthology.orchestra.run.smithi022.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-29T04:18:08.960 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:18:08.961 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-29T04:18:08.961 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:18:08.961 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-29T04:18:08.961 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-29T04:18:09.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:18:09.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:09.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:09.081 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-29T04:18:09.092 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:18:09.092 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:18:09.092 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:18:09.093 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:09.093 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-03-29T04:18:09.093 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:09.093 INFO:teuthology.orchestra.run.smithi022.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:18:09.093 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 26:85:7b:66:89:e7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T04:18:09.093 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T04:18:09.093 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:09.093 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::2485:7bff:fe66:89e7/64 scope link 2024-03-29T04:18:09.093 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:09.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:09.094 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-29T04:18:09.150 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:18:09.150 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:18:09.150 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:18:09.150 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:09.150 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-03-29T04:18:09.150 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:09.150 INFO:teuthology.orchestra.run.smithi022.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:18:09.150 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 26:85:7b:66:89:e7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T04:18:09.150 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T04:18:09.150 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:09.150 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::2485:7bff:fe66:89e7/64 scope link 2024-03-29T04:18:09.151 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:09.152 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-29T04:18:09.152 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:18:09.152 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-29T04:18:09.153 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-29T04:18:09.153 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-29T04:18:09.153 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-29T04:18:09.153 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-29T04:18:09.153 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-29T04:18:09.153 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-29T04:18:09.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:18:09.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:09.297 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:18:09.297 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-29T04:18:09.297 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.1 up 2024-03-29T04:18:09.297 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set dev brx.1 master ceph-brx 2024-03-29T04:18:09.297 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-29T04:18:09.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:18:09.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:09.360 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-03-29T04:18:09.360 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:18:09.360 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-29T04:18:09.407 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-29T04:18:09.408 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:18:09.408 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-29T04:18:09.455 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-03-29T04:18:09.513 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:18:09.562 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-03-29T04:18:09.562 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-03-29T04:18:09.563 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-03-29T04:18:09.563 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-03-29T04:18:09.563 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-03-29T04:18:09.563 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-03-29T04:18:09.563 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-03-29T04:18:09.563 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-03-29T04:18:09.563 INFO:teuthology.orchestra.run.smithi022.stdout:/run/lock 2024-03-29T04:18:09.563 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-03-29T04:18:09.563 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-03-29T04:18:09.563 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-03-29T04:18:09.563 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:18:09.563 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-03-29T04:18:09.563 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-03-29T04:18:09.564 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-03-29T04:18:09.564 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-03-29T04:18:09.564 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/fs/nfsd 2024-03-29T04:18:09.564 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-03-29T04:18:09.564 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-03-29T04:18:09.564 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysusers.service 2024-03-29T04:18:09.564 INFO:teuthology.orchestra.run.smithi022.stdout:/run/rpc_pipefs 2024-03-29T04:18:09.564 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-03-29T04:18:09.564 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-03-29T04:18:09.564 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:18:09.564 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:18:09.564 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:18:09.564 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:18:09.564 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-03-29T04:18:09.565 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:18:09.565 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:18:09.565 INFO:teuthology.orchestra.run.smithi022.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-29T04:18:09.565 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:18:09.565 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:18:09.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:09.566 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-03-29T04:18:09.612 INFO:teuthology.orchestra.run.smithi022.stdout:45 2024-03-29T04:18:09.612 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-03-29T04:18:09.612 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-03-29T04:18:09.657 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-03-29T04:18:09.672 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:18:09.724 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-03-29T04:18:09.725 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-03-29T04:18:09.725 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-03-29T04:18:09.725 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-03-29T04:18:09.725 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-03-29T04:18:09.725 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-03-29T04:18:09.725 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-03-29T04:18:09.725 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-03-29T04:18:09.725 INFO:teuthology.orchestra.run.smithi022.stdout:/run/lock 2024-03-29T04:18:09.725 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-03-29T04:18:09.725 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-03-29T04:18:09.725 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-03-29T04:18:09.725 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:18:09.725 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-03-29T04:18:09.725 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-03-29T04:18:09.726 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-03-29T04:18:09.726 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-03-29T04:18:09.726 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/fs/nfsd 2024-03-29T04:18:09.726 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-03-29T04:18:09.726 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-03-29T04:18:09.726 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysusers.service 2024-03-29T04:18:09.726 INFO:teuthology.orchestra.run.smithi022.stdout:/run/rpc_pipefs 2024-03-29T04:18:09.726 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-03-29T04:18:09.726 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-03-29T04:18:09.726 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:18:09.726 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:18:09.726 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:18:09.726 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:18:09.726 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-03-29T04:18:09.727 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:18:09.727 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:18:09.727 INFO:teuthology.orchestra.run.smithi022.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-29T04:18:09.727 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:18:09.727 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:18:09.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:09.727 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-03-29T04:18:09.737 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi022.stderr:2024-03-29T04:18:09.756+0000 7f9b91ba8580 -1 init, newargv = 0x562a26f14820 newargc=15 2024-03-29T04:18:09.737 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi022.stderr:ceph-fuse[35296]: starting ceph client 2024-03-29T04:18:09.741 INFO:teuthology.orchestra.run.smithi022.stdout:45 2024-03-29T04:18:09.744 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi022.stderr:ceph-fuse[35296]: starting fuse 2024-03-29T04:18:10.843 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-03-29T04:18:10.861 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:18:10.913 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-03-29T04:18:10.913 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-03-29T04:18:10.913 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-03-29T04:18:10.913 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-03-29T04:18:10.913 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-03-29T04:18:10.913 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-03-29T04:18:10.914 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-03-29T04:18:10.914 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-03-29T04:18:10.914 INFO:teuthology.orchestra.run.smithi022.stdout:/run/lock 2024-03-29T04:18:10.914 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-03-29T04:18:10.914 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-03-29T04:18:10.914 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-03-29T04:18:10.914 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:18:10.914 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-03-29T04:18:10.914 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-03-29T04:18:10.914 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-03-29T04:18:10.914 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-03-29T04:18:10.914 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/fs/nfsd 2024-03-29T04:18:10.914 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-03-29T04:18:10.915 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-03-29T04:18:10.915 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysusers.service 2024-03-29T04:18:10.915 INFO:teuthology.orchestra.run.smithi022.stdout:/run/rpc_pipefs 2024-03-29T04:18:10.915 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-03-29T04:18:10.915 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-03-29T04:18:10.915 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:18:10.915 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:18:10.915 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:18:10.915 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:18:10.915 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-03-29T04:18:10.915 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:18:10.915 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:18:10.915 INFO:teuthology.orchestra.run.smithi022.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-29T04:18:10.915 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:18:10.916 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:18:10.916 INFO:teuthology.orchestra.run.smithi022.stdout:/home/ubuntu/cephtest/mnt.2 2024-03-29T04:18:10.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:10.917 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-03-29T04:18:10.964 INFO:teuthology.orchestra.run.smithi022.stdout:45 2024-03-29T04:18:10.964 INFO:teuthology.orchestra.run.smithi022.stdout:46 2024-03-29T04:18:10.965 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-03-29T04:18:10.965 DEBUG:teuthology.orchestra.run.smithi022:> sudo stdin-killer -- python3 -c ' 2024-03-29T04:18:10.965 DEBUG:teuthology.orchestra.run.smithi022:> import glob 2024-03-29T04:18:10.965 DEBUG:teuthology.orchestra.run.smithi022:> import re 2024-03-29T04:18:10.965 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-03-29T04:18:10.965 DEBUG:teuthology.orchestra.run.smithi022:> import subprocess 2024-03-29T04:18:10.965 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-29T04:18:10.965 DEBUG:teuthology.orchestra.run.smithi022:> def _find_admin_socket(client_name): 2024-03-29T04:18:10.965 DEBUG:teuthology.orchestra.run.smithi022:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-03-29T04:18:10.965 DEBUG:teuthology.orchestra.run.smithi022:> files = glob.glob(asok_path) 2024-03-29T04:18:10.965 DEBUG:teuthology.orchestra.run.smithi022:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-03-29T04:18:10.965 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-29T04:18:10.966 DEBUG:teuthology.orchestra.run.smithi022:> # Given a non-glob path, it better be there 2024-03-29T04:18:10.966 DEBUG:teuthology.orchestra.run.smithi022:> if "*" not in asok_path: 2024-03-29T04:18:10.966 DEBUG:teuthology.orchestra.run.smithi022:> assert(len(files) == 1) 2024-03-29T04:18:10.966 DEBUG:teuthology.orchestra.run.smithi022:> return files[0] 2024-03-29T04:18:10.966 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-29T04:18:10.966 DEBUG:teuthology.orchestra.run.smithi022:> for f in files: 2024-03-29T04:18:10.966 DEBUG:teuthology.orchestra.run.smithi022:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T04:18:10.966 DEBUG:teuthology.orchestra.run.smithi022:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T04:18:10.966 DEBUG:teuthology.orchestra.run.smithi022:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T04:18:10.966 DEBUG:teuthology.orchestra.run.smithi022:> contents = proc_f.read() 2024-03-29T04:18:10.966 DEBUG:teuthology.orchestra.run.smithi022:> if mountpoint in contents: 2024-03-29T04:18:10.966 DEBUG:teuthology.orchestra.run.smithi022:> return f 2024-03-29T04:18:10.966 DEBUG:teuthology.orchestra.run.smithi022:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T04:18:10.966 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-29T04:18:10.966 DEBUG:teuthology.orchestra.run.smithi022:> print(_find_admin_socket("client.2")) 2024-03-29T04:18:10.967 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-03-29T04:18:11.092 INFO:teuthology.orchestra.run.smithi022.stdout:/var/run/ceph/ceph-client.2.35296.asok 2024-03-29T04:18:11.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:11.105 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.35296.asok 2024-03-29T04:18:11.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:11.105 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.35296.asok status 2024-03-29T04:18:11.262 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-29T04:18:11.262 INFO:teuthology.orchestra.run.smithi022.stdout: "metadata": { 2024-03-29T04:18:11.262 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_sha1": "69e2cd73c52284dfd5096c66b60c6e6b26a3c447", 2024-03-29T04:18:11.262 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 19.0.0-2520-g69e2cd73 (69e2cd73c52284dfd5096c66b60c6e6b26a3c447) squid (dev)", 2024-03-29T04:18:11.262 INFO:teuthology.orchestra.run.smithi022.stdout: "entity_id": "2", 2024-03-29T04:18:11.262 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": "smithi022", 2024-03-29T04:18:11.262 INFO:teuthology.orchestra.run.smithi022.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-03-29T04:18:11.262 INFO:teuthology.orchestra.run.smithi022.stdout: "pid": "35296", 2024-03-29T04:18:11.263 INFO:teuthology.orchestra.run.smithi022.stdout: "root": "/" 2024-03-29T04:18:11.263 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:11.263 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_count": 0, 2024-03-29T04:18:11.263 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_pinned_count": 0, 2024-03-29T04:18:11.263 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 4884, 2024-03-29T04:18:11.263 INFO:teuthology.orchestra.run.smithi022.stdout: "inst": { 2024-03-29T04:18:11.263 INFO:teuthology.orchestra.run.smithi022.stdout: "name": { 2024-03-29T04:18:11.263 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "client", 2024-03-29T04:18:11.263 INFO:teuthology.orchestra.run.smithi022.stdout: "num": 4884 2024-03-29T04:18:11.263 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:11.263 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2024-03-29T04:18:11.263 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-03-29T04:18:11.263 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.2:0", 2024-03-29T04:18:11.264 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1268566224 2024-03-29T04:18:11.264 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:11.264 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:11.264 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2024-03-29T04:18:11.264 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-03-29T04:18:11.264 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.2:0", 2024-03-29T04:18:11.264 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1268566224 2024-03-29T04:18:11.264 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:11.264 INFO:teuthology.orchestra.run.smithi022.stdout: "inst_str": "client.4884 192.168.0.2:0/1268566224", 2024-03-29T04:18:11.264 INFO:teuthology.orchestra.run.smithi022.stdout: "addr_str": "192.168.0.2:0/1268566224", 2024-03-29T04:18:11.264 INFO:teuthology.orchestra.run.smithi022.stdout: "inode_count": 1, 2024-03-29T04:18:11.264 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_epoch": 9, 2024-03-29T04:18:11.264 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch": 26, 2024-03-29T04:18:11.264 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch_barrier": 0, 2024-03-29T04:18:11.264 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklisted": false, 2024-03-29T04:18:11.264 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs" 2024-03-29T04:18:11.265 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-29T04:18:11.276 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:18:11.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.424+0000 7fa66c5f7640 1 Processor -- start 2024-03-29T04:18:11.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.424+0000 7fa66c5f7640 1 -- start start 2024-03-29T04:18:11.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.424+0000 7fa66c5f7640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa66415f790 0x7fa66415ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:11.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.424+0000 7fa66c5f7640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fa6640725d0 0x7fa664072a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:11.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.424+0000 7fa66c5f7640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6640b5ba0 con 0x7fa664161cb0 2024-03-29T04:18:11.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.424+0000 7fa66c5f7640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fa6640cee60 con 0x7fa6640725d0 2024-03-29T04:18:11.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.424+0000 7fa66c5f7640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa6640b5e80 con 0x7fa66415f790 2024-03-29T04:18:11.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.424+0000 7fa669b92640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa66415f790 0x7fa66415ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:11.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.424+0000 7fa66ab94640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fa6640725d0 0x7fa664072a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:11.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.424+0000 7fa669b92640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa66415f790 0x7fa66415ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.22:44670/0 (socket says 172.21.15.22:44670) 2024-03-29T04:18:11.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.424+0000 7fa669b92640 1 -- 172.21.15.22:0/4241274425 learned_addr learned my addr 172.21.15.22:0/4241274425 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:11.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.428+0000 7fa669b92640 1 -- 172.21.15.22:0/4241274425 >> v2:172.21.15.151:3300/0 conn(0x7fa6640725d0 msgr2=0x7fa664072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:11.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.428+0000 7fa669b92640 1 --2- 172.21.15.22:0/4241274425 >> v2:172.21.15.151:3300/0 conn(0x7fa6640725d0 0x7fa664072a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:11.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.428+0000 7fa669b92640 1 -- 172.21.15.22:0/4241274425 >> v1:172.21.15.151:6789/0 conn(0x7fa664161cb0 legacy=0x7fa6640b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:11.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.428+0000 7fa669b92640 1 -- 172.21.15.22:0/4241274425 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6640b6160 con 0x7fa66415f790 2024-03-29T04:18:11.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.428+0000 7fa669b92640 1 --2- 172.21.15.22:0/4241274425 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa66415f790 0x7fa66415ff90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa654009870 tx=0x7fa65401c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31d12f0471879b33 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:11.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.428+0000 7fa669391640 1 -- 172.21.15.22:0/4241274425 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6540b1070 con 0x7fa66415f790 2024-03-29T04:18:11.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa669391640 1 -- 172.21.15.22:0/4241274425 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6540a8050 con 0x7fa66415f790 2024-03-29T04:18:11.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa669391640 1 -- 172.21.15.22:0/4241274425 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6540ac860 con 0x7fa66415f790 2024-03-29T04:18:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa66c5f7640 1 -- 172.21.15.22:0/4241274425 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa66415f790 msgr2=0x7fa66415ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa66c5f7640 1 --2- 172.21.15.22:0/4241274425 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa66415f790 0x7fa66415ff90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa654009870 tx=0x7fa65401c4d0 comp rx=0 tx=0).stop 2024-03-29T04:18:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa66c5f7640 1 -- 172.21.15.22:0/4241274425 shutdown_connections 2024-03-29T04:18:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa66c5f7640 1 --2- 172.21.15.22:0/4241274425 >> v2:172.21.15.151:3300/0 conn(0x7fa6640725d0 0x7fa664072a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa66c5f7640 1 --2- 172.21.15.22:0/4241274425 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa66415f790 0x7fa66415ff90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa66c5f7640 1 -- 172.21.15.22:0/4241274425 >> 172.21.15.22:0/4241274425 conn(0x7fa6640d0f60 msgr2=0x7fa6640d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa66c5f7640 1 -- 172.21.15.22:0/4241274425 shutdown_connections 2024-03-29T04:18:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa66c5f7640 1 -- 172.21.15.22:0/4241274425 wait complete. 2024-03-29T04:18:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa66c5f7640 1 Processor -- start 2024-03-29T04:18:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa66c5f7640 1 -- start start 2024-03-29T04:18:11.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa66c5f7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6640725d0 0x7fa664147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:11.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa66c5f7640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa66415f790 0x7fa6641481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:11.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa66c5f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa664161cb0 0x7fa6641486e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:11.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa66c5f7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa6640cf5d0 con 0x7fa664161cb0 2024-03-29T04:18:11.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa66c5f7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa66406e0a0 con 0x7fa6640725d0 2024-03-29T04:18:11.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa66c5f7640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa6640cf750 con 0x7fa66415f790 2024-03-29T04:18:11.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa66ab94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa664161cb0 0x7fa6641486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:11.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa66ab94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa664161cb0 0x7fa6641486e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49748/0 (socket says 172.21.15.22:49748) 2024-03-29T04:18:11.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa66ab94640 1 -- 172.21.15.22:0/2904290018 learned_addr learned my addr 172.21.15.22:0/2904290018 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:11.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa669b92640 1 --2- 172.21.15.22:0/2904290018 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa66415f790 0x7fa6641481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:11.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.432+0000 7fa66a393640 1 --2- 172.21.15.22:0/2904290018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6640725d0 0x7fa664147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:11.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.436+0000 7fa66ab94640 1 -- 172.21.15.22:0/2904290018 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa66415f790 msgr2=0x7fa6641481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:11.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.436+0000 7fa66ab94640 1 --2- 172.21.15.22:0/2904290018 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa66415f790 0x7fa6641481a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:11.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.436+0000 7fa66ab94640 1 -- 172.21.15.22:0/2904290018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6640725d0 msgr2=0x7fa664147810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:11.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.436+0000 7fa66ab94640 1 --2- 172.21.15.22:0/2904290018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6640725d0 0x7fa664147810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:11.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.436+0000 7fa66ab94640 1 -- 172.21.15.22:0/2904290018 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa664147d50 con 0x7fa664161cb0 2024-03-29T04:18:11.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.436+0000 7fa66ab94640 1 --2- 172.21.15.22:0/2904290018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa664161cb0 0x7fa6641486e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00e930 tx=0x7fa65c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:11.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.436+0000 7fa65affd640 1 -- 172.21.15.22:0/2904290018 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c017070 con 0x7fa664161cb0 2024-03-29T04:18:11.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.436+0000 7fa65affd640 1 -- 172.21.15.22:0/2904290018 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa65c00c9a0 con 0x7fa664161cb0 2024-03-29T04:18:11.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.436+0000 7fa66c5f7640 1 -- 172.21.15.22:0/2904290018 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa664148d10 con 0x7fa664161cb0 2024-03-29T04:18:11.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.436+0000 7fa65affd640 1 -- 172.21.15.22:0/2904290018 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c00cd70 con 0x7fa664161cb0 2024-03-29T04:18:11.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.436+0000 7fa66c5f7640 1 -- 172.21.15.22:0/2904290018 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6641492d0 con 0x7fa664161cb0 2024-03-29T04:18:11.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.436+0000 7fa66c5f7640 1 -- 172.21.15.22:0/2904290018 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa664068890 con 0x7fa664161cb0 2024-03-29T04:18:11.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.436+0000 7fa65affd640 1 -- 172.21.15.22:0/2904290018 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fa65c01a070 con 0x7fa664161cb0 2024-03-29T04:18:11.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.436+0000 7fa65affd640 1 --2- 172.21.15.22:0/2904290018 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa648067fc0 0x7fa64806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:11.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.436+0000 7fa65affd640 1 -- 172.21.15.22:0/2904290018 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa65c086c60 con 0x7fa664161cb0 2024-03-29T04:18:11.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.436+0000 7fa66a393640 1 --2- 172.21.15.22:0/2904290018 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa648067fc0 0x7fa64806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:11.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.436+0000 7fa66a393640 1 --2- 172.21.15.22:0/2904290018 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa648067fc0 0x7fa64806a480 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa66406ca70 tx=0x7fa660009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:11.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.440+0000 7fa65affd640 1 -- 172.21.15.22:0/2904290018 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fa65c056a20 con 0x7fa664161cb0 2024-03-29T04:18:11.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.640+0000 7fa66c5f7640 1 -- 172.21.15.22:0/2904290018 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa6640738f0 con 0x7fa664161cb0 2024-03-29T04:18:11.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.640+0000 7fa65affd640 1 -- 172.21.15.22:0/2904290018 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fa65c05ad20 con 0x7fa664161cb0 2024-03-29T04:18:11.621 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-29T04:18:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.644+0000 7fa66c5f7640 1 -- 172.21.15.22:0/2904290018 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa648067fc0 msgr2=0x7fa64806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.644+0000 7fa66c5f7640 1 --2- 172.21.15.22:0/2904290018 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa648067fc0 0x7fa64806a480 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa66406ca70 tx=0x7fa660009040 comp rx=0 tx=0).stop 2024-03-29T04:18:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.644+0000 7fa66c5f7640 1 -- 172.21.15.22:0/2904290018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa664161cb0 msgr2=0x7fa6641486e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.644+0000 7fa66c5f7640 1 --2- 172.21.15.22:0/2904290018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa664161cb0 0x7fa6641486e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00e930 tx=0x7fa65c00edf0 comp rx=0 tx=0).stop 2024-03-29T04:18:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.644+0000 7fa66c5f7640 1 -- 172.21.15.22:0/2904290018 shutdown_connections 2024-03-29T04:18:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.644+0000 7fa66c5f7640 1 --2- 172.21.15.22:0/2904290018 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa648067fc0 0x7fa64806a480 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.644+0000 7fa66c5f7640 1 --2- 172.21.15.22:0/2904290018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa664161cb0 0x7fa6641486e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.644+0000 7fa66c5f7640 1 --2- 172.21.15.22:0/2904290018 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa66415f790 0x7fa6641481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.644+0000 7fa66c5f7640 1 --2- 172.21.15.22:0/2904290018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6640725d0 0x7fa664147810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.644+0000 7fa66c5f7640 1 -- 172.21.15.22:0/2904290018 >> 172.21.15.22:0/2904290018 conn(0x7fa6640d0f60 msgr2=0x7fa6641545e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:11.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.644+0000 7fa66c5f7640 1 -- 172.21.15.22:0/2904290018 shutdown_connections 2024-03-29T04:18:11.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:11.644+0000 7fa66c5f7640 1 -- 172.21.15.22:0/2904290018 wait complete. 2024-03-29T04:18:11.639 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:18:11.639 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:18:11.639 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi151.front.sepia.ceph.com 2024-03-29T04:18:11.639 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-29T04:18:11.639 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-29T04:18:11.639 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:18:11.639 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:18:11.640 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:18:11.640 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:18:11.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:11.640 DEBUG:teuthology.orchestra.run.smithi151:> ip addr 2024-03-29T04:18:11.646 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:18:11.646 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:18:11.646 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:18:11.646 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:11.647 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-03-29T04:18:11.647 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:11.647 INFO:teuthology.orchestra.run.smithi151.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T04:18:11.647 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0c brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:11.647 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f0 2024-03-29T04:18:11.647 INFO:teuthology.orchestra.run.smithi151.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T04:18:11.647 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0d brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:11.647 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f1 2024-03-29T04:18:11.647 INFO:teuthology.orchestra.run.smithi151.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T04:18:11.647 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:62 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:11.647 INFO:teuthology.orchestra.run.smithi151.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:18:11.647 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:63 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:11.647 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.21.15.151/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-29T04:18:11.648 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft 42468sec preferred_lft 42468sec 2024-03-29T04:18:11.648 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::ec4:7aff:fe88:6f63/64 scope link 2024-03-29T04:18:11.648 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:11.648 INFO:teuthology.orchestra.run.smithi151.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T04:18:11.648 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 02:42:63:20:ce:96 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:11.648 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T04:18:11.648 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:11.648 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-29T04:18:11.648 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:18:11.648 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-03-29T04:18:11.648 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link add name ceph-brx type bridge 2024-03-29T04:18:11.649 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip addr flush dev ceph-brx 2024-03-29T04:18:11.649 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set ceph-brx up 2024-03-29T04:18:11.649 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-29T04:18:11.649 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-03-29T04:18:11.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:18:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:18:11.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:18:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:11.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:11.772 DEBUG:teuthology.orchestra.run.smithi151:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:18:11.781 INFO:teuthology.orchestra.run.smithi151.stdout:1 2024-03-29T04:18:11.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:11.782 DEBUG:teuthology.orchestra.run.smithi151:> route 2024-03-29T04:18:11.850 INFO:teuthology.orchestra.run.smithi151.stdout:Kernel IP routing table 2024-03-29T04:18:11.850 INFO:teuthology.orchestra.run.smithi151.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:18:11.850 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-29T04:18:11.850 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:18:11.850 INFO:teuthology.orchestra.run.smithi151.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-29T04:18:11.850 INFO:teuthology.orchestra.run.smithi151.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T04:18:11.851 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:18:11.851 INFO:teuthology.orchestra.run.smithi151.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-29T04:18:11.851 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-29T04:18:11.851 INFO:teuthology.orchestra.run.smithi151.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-29T04:18:11.851 INFO:teuthology.orchestra.run.smithi151.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:18:11.851 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:18:11.851 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-03-29T04:18:11.851 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:18:11.851 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:18:11.851 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:18:11.852 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-03-29T04:18:11.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:18:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:18:11.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:18:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:11.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:11.965 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-03-29T04:18:12.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:12.014 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list-id 2024-03-29T04:18:12.062 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:18:12.062 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-03-29T04:18:12.062 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:18:12.062 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-29T04:18:12.062 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-03-29T04:18:12.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:18:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:18:12.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:18:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:12.190 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-29T04:18:12.190 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:18:12.190 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-03-29T04:18:12.190 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-29T04:18:12.190 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-29T04:18:12.190 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-29T04:18:12.190 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-29T04:18:12.190 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-29T04:18:12.190 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-03-29T04:18:12.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:18:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:18:12.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:18:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:12.289 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:18:12.289 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-03-29T04:18:12.289 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set brx.0 up 2024-03-29T04:18:12.289 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set dev brx.0 master ceph-brx 2024-03-29T04:18:12.289 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-03-29T04:18:12.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:18:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:18:12.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:18:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:12.396 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-03-29T04:18:12.396 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:18:12.396 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-29T04:18:12.445 INFO:teuthology.orchestra.run.smithi151.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:18:12.445 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:18:12.446 DEBUG:teuthology.orchestra.run.smithi151:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-29T04:18:12.496 DEBUG:teuthology.orchestra.run.smithi151:> sudo modprobe fuse 2024-03-29T04:18:12.546 DEBUG:teuthology.orchestra.run.smithi151:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:18:12.594 INFO:teuthology.orchestra.run.smithi151.stdout:/sys 2024-03-29T04:18:12.594 INFO:teuthology.orchestra.run.smithi151.stdout:/proc 2024-03-29T04:18:12.594 INFO:teuthology.orchestra.run.smithi151.stdout:/dev 2024-03-29T04:18:12.594 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/pts 2024-03-29T04:18:12.594 INFO:teuthology.orchestra.run.smithi151.stdout:/run 2024-03-29T04:18:12.594 INFO:teuthology.orchestra.run.smithi151.stdout:/ 2024-03-29T04:18:12.594 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/security 2024-03-29T04:18:12.594 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/shm 2024-03-29T04:18:12.594 INFO:teuthology.orchestra.run.smithi151.stdout:/run/lock 2024-03-29T04:18:12.595 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/cgroup 2024-03-29T04:18:12.595 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/pstore 2024-03-29T04:18:12.595 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/bpf 2024-03-29T04:18:12.595 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:18:12.595 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/hugepages 2024-03-29T04:18:12.595 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/mqueue 2024-03-29T04:18:12.595 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/debug 2024-03-29T04:18:12.595 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/tracing 2024-03-29T04:18:12.595 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/fs/nfsd 2024-03-29T04:18:12.595 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/fuse/connections 2024-03-29T04:18:12.595 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/config 2024-03-29T04:18:12.595 INFO:teuthology.orchestra.run.smithi151.stdout:/run/credentials/systemd-sysusers.service 2024-03-29T04:18:12.595 INFO:teuthology.orchestra.run.smithi151.stdout:/run/rpc_pipefs 2024-03-29T04:18:12.595 INFO:teuthology.orchestra.run.smithi151.stdout:/run/user/1000 2024-03-29T04:18:12.595 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph 2024-03-29T04:18:12.596 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-4 2024-03-29T04:18:12.596 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-5 2024-03-29T04:18:12.596 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-6 2024-03-29T04:18:12.596 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-7 2024-03-29T04:18:12.596 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns 2024-03-29T04:18:12.596 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:18:12.596 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:18:12.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:12.597 DEBUG:teuthology.orchestra.run.smithi151:> ls /sys/fs/fuse/connections 2024-03-29T04:18:12.642 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T04:18:12.642 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-03-29T04:18:12.687 DEBUG:teuthology.orchestra.run.smithi151:> sudo modprobe fuse 2024-03-29T04:18:12.697 DEBUG:teuthology.orchestra.run.smithi151:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:18:12.747 INFO:teuthology.orchestra.run.smithi151.stdout:/sys 2024-03-29T04:18:12.747 INFO:teuthology.orchestra.run.smithi151.stdout:/proc 2024-03-29T04:18:12.747 INFO:teuthology.orchestra.run.smithi151.stdout:/dev 2024-03-29T04:18:12.747 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/pts 2024-03-29T04:18:12.747 INFO:teuthology.orchestra.run.smithi151.stdout:/run 2024-03-29T04:18:12.747 INFO:teuthology.orchestra.run.smithi151.stdout:/ 2024-03-29T04:18:12.747 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/security 2024-03-29T04:18:12.747 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/shm 2024-03-29T04:18:12.747 INFO:teuthology.orchestra.run.smithi151.stdout:/run/lock 2024-03-29T04:18:12.747 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/cgroup 2024-03-29T04:18:12.747 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/pstore 2024-03-29T04:18:12.747 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/bpf 2024-03-29T04:18:12.748 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:18:12.748 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/hugepages 2024-03-29T04:18:12.748 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/mqueue 2024-03-29T04:18:12.748 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/debug 2024-03-29T04:18:12.748 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/tracing 2024-03-29T04:18:12.748 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/fs/nfsd 2024-03-29T04:18:12.748 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/fuse/connections 2024-03-29T04:18:12.748 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/config 2024-03-29T04:18:12.748 INFO:teuthology.orchestra.run.smithi151.stdout:/run/credentials/systemd-sysusers.service 2024-03-29T04:18:12.748 INFO:teuthology.orchestra.run.smithi151.stdout:/run/rpc_pipefs 2024-03-29T04:18:12.748 INFO:teuthology.orchestra.run.smithi151.stdout:/run/user/1000 2024-03-29T04:18:12.748 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph 2024-03-29T04:18:12.748 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-4 2024-03-29T04:18:12.749 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-5 2024-03-29T04:18:12.749 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-6 2024-03-29T04:18:12.749 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-7 2024-03-29T04:18:12.749 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns 2024-03-29T04:18:12.749 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:18:12.749 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:18:12.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:12.749 DEBUG:teuthology.orchestra.run.smithi151:> ls /sys/fs/fuse/connections 2024-03-29T04:18:12.757 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi151.stderr:2024-03-29T04:18:12.775+0000 7f451c97f580 -1 init, newargv = 0x55fb3fc1b6f0 newargc=15 2024-03-29T04:18:12.757 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi151.stderr:ceph-fuse[33833]: starting ceph client 2024-03-29T04:18:12.763 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi151.stderr:ceph-fuse[33833]: starting fuse 2024-03-29T04:18:13.863 DEBUG:teuthology.orchestra.run.smithi151:> sudo modprobe fuse 2024-03-29T04:18:13.878 DEBUG:teuthology.orchestra.run.smithi151:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:18:13.927 INFO:teuthology.orchestra.run.smithi151.stdout:/sys 2024-03-29T04:18:13.927 INFO:teuthology.orchestra.run.smithi151.stdout:/proc 2024-03-29T04:18:13.927 INFO:teuthology.orchestra.run.smithi151.stdout:/dev 2024-03-29T04:18:13.927 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/pts 2024-03-29T04:18:13.927 INFO:teuthology.orchestra.run.smithi151.stdout:/run 2024-03-29T04:18:13.927 INFO:teuthology.orchestra.run.smithi151.stdout:/ 2024-03-29T04:18:13.927 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/security 2024-03-29T04:18:13.928 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/shm 2024-03-29T04:18:13.928 INFO:teuthology.orchestra.run.smithi151.stdout:/run/lock 2024-03-29T04:18:13.928 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/cgroup 2024-03-29T04:18:13.928 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/pstore 2024-03-29T04:18:13.928 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/bpf 2024-03-29T04:18:13.928 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:18:13.928 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/hugepages 2024-03-29T04:18:13.928 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/mqueue 2024-03-29T04:18:13.928 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/debug 2024-03-29T04:18:13.928 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/tracing 2024-03-29T04:18:13.928 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/fs/nfsd 2024-03-29T04:18:13.928 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/fuse/connections 2024-03-29T04:18:13.928 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/config 2024-03-29T04:18:13.929 INFO:teuthology.orchestra.run.smithi151.stdout:/run/credentials/systemd-sysusers.service 2024-03-29T04:18:13.929 INFO:teuthology.orchestra.run.smithi151.stdout:/run/rpc_pipefs 2024-03-29T04:18:13.929 INFO:teuthology.orchestra.run.smithi151.stdout:/run/user/1000 2024-03-29T04:18:13.929 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph 2024-03-29T04:18:13.929 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-4 2024-03-29T04:18:13.929 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-5 2024-03-29T04:18:13.929 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-6 2024-03-29T04:18:13.929 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-7 2024-03-29T04:18:13.929 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns 2024-03-29T04:18:13.929 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:18:13.929 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:18:13.929 INFO:teuthology.orchestra.run.smithi151.stdout:/home/ubuntu/cephtest/mnt.1 2024-03-29T04:18:13.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:13.931 DEBUG:teuthology.orchestra.run.smithi151:> ls /sys/fs/fuse/connections 2024-03-29T04:18:13.978 INFO:teuthology.orchestra.run.smithi151.stdout:45 2024-03-29T04:18:13.978 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-03-29T04:18:13.978 DEBUG:teuthology.orchestra.run.smithi151:> sudo stdin-killer -- python3 -c ' 2024-03-29T04:18:13.978 DEBUG:teuthology.orchestra.run.smithi151:> import glob 2024-03-29T04:18:13.979 DEBUG:teuthology.orchestra.run.smithi151:> import re 2024-03-29T04:18:13.979 DEBUG:teuthology.orchestra.run.smithi151:> import os 2024-03-29T04:18:13.979 DEBUG:teuthology.orchestra.run.smithi151:> import subprocess 2024-03-29T04:18:13.979 DEBUG:teuthology.orchestra.run.smithi151:> 2024-03-29T04:18:13.979 DEBUG:teuthology.orchestra.run.smithi151:> def _find_admin_socket(client_name): 2024-03-29T04:18:13.979 DEBUG:teuthology.orchestra.run.smithi151:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-03-29T04:18:13.979 DEBUG:teuthology.orchestra.run.smithi151:> files = glob.glob(asok_path) 2024-03-29T04:18:13.979 DEBUG:teuthology.orchestra.run.smithi151:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-03-29T04:18:13.979 DEBUG:teuthology.orchestra.run.smithi151:> 2024-03-29T04:18:13.979 DEBUG:teuthology.orchestra.run.smithi151:> # Given a non-glob path, it better be there 2024-03-29T04:18:13.979 DEBUG:teuthology.orchestra.run.smithi151:> if "*" not in asok_path: 2024-03-29T04:18:13.980 DEBUG:teuthology.orchestra.run.smithi151:> assert(len(files) == 1) 2024-03-29T04:18:13.980 DEBUG:teuthology.orchestra.run.smithi151:> return files[0] 2024-03-29T04:18:13.980 DEBUG:teuthology.orchestra.run.smithi151:> 2024-03-29T04:18:13.980 DEBUG:teuthology.orchestra.run.smithi151:> for f in files: 2024-03-29T04:18:13.980 DEBUG:teuthology.orchestra.run.smithi151:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T04:18:13.980 DEBUG:teuthology.orchestra.run.smithi151:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T04:18:13.980 DEBUG:teuthology.orchestra.run.smithi151:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T04:18:13.980 DEBUG:teuthology.orchestra.run.smithi151:> contents = proc_f.read() 2024-03-29T04:18:13.980 DEBUG:teuthology.orchestra.run.smithi151:> if mountpoint in contents: 2024-03-29T04:18:13.980 DEBUG:teuthology.orchestra.run.smithi151:> return f 2024-03-29T04:18:13.980 DEBUG:teuthology.orchestra.run.smithi151:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T04:18:13.980 DEBUG:teuthology.orchestra.run.smithi151:> 2024-03-29T04:18:13.980 DEBUG:teuthology.orchestra.run.smithi151:> print(_find_admin_socket("client.1")) 2024-03-29T04:18:13.980 DEBUG:teuthology.orchestra.run.smithi151:> ' 2024-03-29T04:18:14.098 INFO:teuthology.orchestra.run.smithi151.stdout:/var/run/ceph/ceph-client.1.33833.asok 2024-03-29T04:18:14.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:18:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:14.109 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33833.asok 2024-03-29T04:18:14.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:14.110 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33833.asok status 2024-03-29T04:18:14.296 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-03-29T04:18:14.296 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": { 2024-03-29T04:18:14.296 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_sha1": "69e2cd73c52284dfd5096c66b60c6e6b26a3c447", 2024-03-29T04:18:14.296 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_version": "ceph version 19.0.0-2520-g69e2cd73 (69e2cd73c52284dfd5096c66b60c6e6b26a3c447) squid (dev)", 2024-03-29T04:18:14.297 INFO:teuthology.orchestra.run.smithi151.stdout: "entity_id": "1", 2024-03-29T04:18:14.297 INFO:teuthology.orchestra.run.smithi151.stdout: "hostname": "smithi151", 2024-03-29T04:18:14.297 INFO:teuthology.orchestra.run.smithi151.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-03-29T04:18:14.297 INFO:teuthology.orchestra.run.smithi151.stdout: "pid": "33833", 2024-03-29T04:18:14.297 INFO:teuthology.orchestra.run.smithi151.stdout: "root": "/" 2024-03-29T04:18:14.297 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-03-29T04:18:14.297 INFO:teuthology.orchestra.run.smithi151.stdout: "dentry_count": 0, 2024-03-29T04:18:14.297 INFO:teuthology.orchestra.run.smithi151.stdout: "dentry_pinned_count": 0, 2024-03-29T04:18:14.297 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 4553, 2024-03-29T04:18:14.297 INFO:teuthology.orchestra.run.smithi151.stdout: "inst": { 2024-03-29T04:18:14.297 INFO:teuthology.orchestra.run.smithi151.stdout: "name": { 2024-03-29T04:18:14.297 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "client", 2024-03-29T04:18:14.297 INFO:teuthology.orchestra.run.smithi151.stdout: "num": 4553 2024-03-29T04:18:14.298 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-03-29T04:18:14.298 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": { 2024-03-29T04:18:14.298 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v1", 2024-03-29T04:18:14.298 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:18:14.298 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 1215776644 2024-03-29T04:18:14.298 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-03-29T04:18:14.298 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-03-29T04:18:14.298 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": { 2024-03-29T04:18:14.298 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v1", 2024-03-29T04:18:14.298 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:18:14.298 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 1215776644 2024-03-29T04:18:14.298 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-03-29T04:18:14.298 INFO:teuthology.orchestra.run.smithi151.stdout: "inst_str": "client.4553 192.168.0.1:0/1215776644", 2024-03-29T04:18:14.299 INFO:teuthology.orchestra.run.smithi151.stdout: "addr_str": "192.168.0.1:0/1215776644", 2024-03-29T04:18:14.299 INFO:teuthology.orchestra.run.smithi151.stdout: "inode_count": 1, 2024-03-29T04:18:14.299 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_epoch": 9, 2024-03-29T04:18:14.299 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_epoch": 26, 2024-03-29T04:18:14.299 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_epoch_barrier": 0, 2024-03-29T04:18:14.299 INFO:teuthology.orchestra.run.smithi151.stdout: "blocklisted": false, 2024-03-29T04:18:14.299 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs" 2024-03-29T04:18:14.299 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-03-29T04:18:14.316 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:18:14.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.468+0000 7f343854e640 1 Processor -- start 2024-03-29T04:18:14.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.468+0000 7f343854e640 1 -- start start 2024-03-29T04:18:14.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.468+0000 7f343854e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f343006c670 0x7f34300cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:14.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.468+0000 7f343854e640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f34300b41f0 0x7f343006e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:14.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.468+0000 7f343854e640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f343006e5c0 con 0x7f343006ca50 2024-03-29T04:18:14.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.468+0000 7f343854e640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f34300b34c0 con 0x7f343006c670 2024-03-29T04:18:14.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.468+0000 7f343854e640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f34300ccef0 con 0x7f34300b41f0 2024-03-29T04:18:14.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.468+0000 7f34362ea640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f343006c670 0x7f34300cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:14.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.468+0000 7f34362ea640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f343006c670 0x7f34300cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49752/0 (socket says 172.21.15.22:49752) 2024-03-29T04:18:14.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.468+0000 7f34362ea640 1 -- 172.21.15.22:0/1409678723 learned_addr learned my addr 172.21.15.22:0/1409678723 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:14.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.468+0000 7f3435ae9640 1 --2- 172.21.15.22:0/1409678723 >> v2:172.21.15.151:3300/0 conn(0x7f34300b41f0 0x7f343006e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:14.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.468+0000 7f34362ea640 1 -- 172.21.15.22:0/1409678723 >> v2:172.21.15.151:3300/0 conn(0x7f34300b41f0 msgr2=0x7f343006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:14.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.468+0000 7f34362ea640 1 --2- 172.21.15.22:0/1409678723 >> v2:172.21.15.151:3300/0 conn(0x7f34300b41f0 0x7f343006e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:14.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.468+0000 7f34362ea640 1 -- 172.21.15.22:0/1409678723 >> v1:172.21.15.151:6789/0 conn(0x7f343006ca50 legacy=0x7f343015fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:14.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.468+0000 7f34362ea640 1 -- 172.21.15.22:0/1409678723 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34300b3710 con 0x7f343006c670 2024-03-29T04:18:14.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f34362ea640 1 --2- 172.21.15.22:0/1409678723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f343006c670 0x7f34300cc910 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f343006ebe0 tx=0x7f342c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2944d22b25716f0f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:14.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f34352e8640 1 -- 172.21.15.22:0/1409678723 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c0a56c0 con 0x7f343006c670 2024-03-29T04:18:14.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f34352e8640 1 -- 172.21.15.22:0/1409678723 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f342c0a5860 con 0x7f343006c670 2024-03-29T04:18:14.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f34352e8640 1 -- 172.21.15.22:0/1409678723 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c0a5c30 con 0x7f343006c670 2024-03-29T04:18:14.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f343854e640 1 -- 172.21.15.22:0/1409678723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f343006c670 msgr2=0x7f34300cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:14.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f343854e640 1 --2- 172.21.15.22:0/1409678723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f343006c670 0x7f34300cc910 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f343006ebe0 tx=0x7f342c01c9f0 comp rx=0 tx=0).stop 2024-03-29T04:18:14.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f343854e640 1 -- 172.21.15.22:0/1409678723 shutdown_connections 2024-03-29T04:18:14.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f343854e640 1 --2- 172.21.15.22:0/1409678723 >> v2:172.21.15.151:3300/0 conn(0x7f34300b41f0 0x7f343006e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:14.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f343854e640 1 --2- 172.21.15.22:0/1409678723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f343006c670 0x7f34300cc910 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:14.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f343854e640 1 -- 172.21.15.22:0/1409678723 >> 172.21.15.22:0/1409678723 conn(0x7f34300cfee0 msgr2=0x7f34300d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:14.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f343854e640 1 -- 172.21.15.22:0/1409678723 shutdown_connections 2024-03-29T04:18:14.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f343854e640 1 -- 172.21.15.22:0/1409678723 wait complete. 2024-03-29T04:18:14.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f343854e640 1 Processor -- start 2024-03-29T04:18:14.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f343854e640 1 -- start start 2024-03-29T04:18:14.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f343854e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f343006c670 0x7f34301f8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:14.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f343854e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f343006ca50 0x7f343013b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:14.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f343854e640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34300b41f0 0x7f343013bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:14.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f343854e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f343006cfe0 con 0x7f343006c670 2024-03-29T04:18:14.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f343854e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f343006ce60 con 0x7f343006ca50 2024-03-29T04:18:14.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f343854e640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f3430168ef0 con 0x7f34300b41f0 2024-03-29T04:18:14.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f34362ea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f343006c670 0x7f34301f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:14.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f34362ea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f343006c670 0x7f34301f8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49768/0 (socket says 172.21.15.22:49768) 2024-03-29T04:18:14.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f34362ea640 1 -- 172.21.15.22:0/895676844 learned_addr learned my addr 172.21.15.22:0/895676844 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:14.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f3436aeb640 1 --2- 172.21.15.22:0/895676844 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34300b41f0 0x7f343013bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:14.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f3435ae9640 1 --2- 172.21.15.22:0/895676844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f343006ca50 0x7f343013b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:14.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f34362ea640 1 -- 172.21.15.22:0/895676844 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34300b41f0 msgr2=0x7f343013bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:14.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f34362ea640 1 --2- 172.21.15.22:0/895676844 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34300b41f0 0x7f343013bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:14.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f34362ea640 1 -- 172.21.15.22:0/895676844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f343006ca50 msgr2=0x7f343013b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:14.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f34362ea640 1 --2- 172.21.15.22:0/895676844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f343006ca50 0x7f343013b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:14.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f34362ea640 1 -- 172.21.15.22:0/895676844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3430143400 con 0x7f343006c670 2024-03-29T04:18:14.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f3435ae9640 1 --2- 172.21.15.22:0/895676844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f343006ca50 0x7f343013b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:14.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f34362ea640 1 --2- 172.21.15.22:0/895676844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f343006c670 0x7f34301f8590 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f342c01b0a0 tx=0x7f342c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:14.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f3426ffd640 1 -- 172.21.15.22:0/895676844 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c0b8670 con 0x7f343006c670 2024-03-29T04:18:14.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.472+0000 7f3426ffd640 1 -- 172.21.15.22:0/895676844 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f342c0be070 con 0x7f343006c670 2024-03-29T04:18:14.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.476+0000 7f343854e640 1 -- 172.21.15.22:0/895676844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f343015c000 con 0x7f343006c670 2024-03-29T04:18:14.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.476+0000 7f3426ffd640 1 -- 172.21.15.22:0/895676844 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c0b8a70 con 0x7f343006c670 2024-03-29T04:18:14.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.476+0000 7f343854e640 1 -- 172.21.15.22:0/895676844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34301f8020 con 0x7f343006c670 2024-03-29T04:18:14.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.476+0000 7f343854e640 1 -- 172.21.15.22:0/895676844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34301f8260 con 0x7f343006c670 2024-03-29T04:18:14.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.476+0000 7f3426ffd640 1 -- 172.21.15.22:0/895676844 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f342c0b8c10 con 0x7f343006c670 2024-03-29T04:18:14.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.476+0000 7f3426ffd640 1 --2- 172.21.15.22:0/895676844 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3414067fc0 0x7f341406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:14.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.476+0000 7f3426ffd640 1 -- 172.21.15.22:0/895676844 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f342c1218e0 con 0x7f343006c670 2024-03-29T04:18:14.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.476+0000 7f3435ae9640 1 --2- 172.21.15.22:0/895676844 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3414067fc0 0x7f341406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:14.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.476+0000 7f3435ae9640 1 --2- 172.21.15.22:0/895676844 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3414067fc0 0x7f341406a480 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3430166840 tx=0x7f3420008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:14.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.480+0000 7f3426ffd640 1 -- 172.21.15.22:0/895676844 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f34301f8260 con 0x7f343006c670 2024-03-29T04:18:14.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.680+0000 7f343854e640 1 -- 172.21.15.22:0/895676844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3430068890 con 0x7f343006c670 2024-03-29T04:18:14.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.680+0000 7f3426ffd640 1 -- 172.21.15.22:0/895676844 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f3430143400 con 0x7f343006c670 2024-03-29T04:18:14.663 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-29T04:18:14.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.684+0000 7f343854e640 1 -- 172.21.15.22:0/895676844 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3414067fc0 msgr2=0x7f341406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:14.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.684+0000 7f343854e640 1 --2- 172.21.15.22:0/895676844 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3414067fc0 0x7f341406a480 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3430166840 tx=0x7f3420008040 comp rx=0 tx=0).stop 2024-03-29T04:18:14.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.684+0000 7f343854e640 1 -- 172.21.15.22:0/895676844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f343006c670 msgr2=0x7f34301f8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:14.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.684+0000 7f343854e640 1 --2- 172.21.15.22:0/895676844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f343006c670 0x7f34301f8590 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f342c01b0a0 tx=0x7f342c019790 comp rx=0 tx=0).stop 2024-03-29T04:18:14.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.684+0000 7f343854e640 1 -- 172.21.15.22:0/895676844 shutdown_connections 2024-03-29T04:18:14.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.684+0000 7f343854e640 1 --2- 172.21.15.22:0/895676844 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f34300b41f0 0x7f343013bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:14.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.684+0000 7f343854e640 1 --2- 172.21.15.22:0/895676844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f343006ca50 0x7f343013b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:14.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.684+0000 7f343854e640 1 --2- 172.21.15.22:0/895676844 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3414067fc0 0x7f341406a480 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:14.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.684+0000 7f343854e640 1 --2- 172.21.15.22:0/895676844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f343006c670 0x7f34301f8590 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:14.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.684+0000 7f343854e640 1 -- 172.21.15.22:0/895676844 >> 172.21.15.22:0/895676844 conn(0x7f34300cfee0 msgr2=0x7f343015c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:14.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.684+0000 7f343854e640 1 -- 172.21.15.22:0/895676844 shutdown_connections 2024-03-29T04:18:14.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:14.684+0000 7f343854e640 1 -- 172.21.15.22:0/895676844 wait complete. 2024-03-29T04:18:14.682 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:18:14.682 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:18:14.682 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi151.front.sepia.ceph.com 2024-03-29T04:18:14.682 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-29T04:18:14.682 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-29T04:18:14.682 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:18:14.682 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:18:14.682 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:18:14.682 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-29T04:18:14.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:14.683 DEBUG:teuthology.orchestra.run.smithi151:> ip addr 2024-03-29T04:18:14.689 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:18:14.689 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:18:14.689 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:18:14.689 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:14.689 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-03-29T04:18:14.689 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:14.690 INFO:teuthology.orchestra.run.smithi151.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T04:18:14.690 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0c brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:14.690 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f0 2024-03-29T04:18:14.690 INFO:teuthology.orchestra.run.smithi151.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T04:18:14.690 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0d brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:14.690 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f1 2024-03-29T04:18:14.690 INFO:teuthology.orchestra.run.smithi151.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T04:18:14.690 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:62 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:14.690 INFO:teuthology.orchestra.run.smithi151.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:18:14.690 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:63 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:14.690 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.21.15.151/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-29T04:18:14.690 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft 42465sec preferred_lft 42465sec 2024-03-29T04:18:14.690 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::ec4:7aff:fe88:6f63/64 scope link 2024-03-29T04:18:14.690 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:14.691 INFO:teuthology.orchestra.run.smithi151.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T04:18:14.691 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 02:42:63:20:ce:96 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:14.691 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T04:18:14.691 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:14.691 INFO:teuthology.orchestra.run.smithi151.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:18:14.691 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-29T04:18:14.691 INFO:teuthology.orchestra.run.smithi151.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T04:18:14.691 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:14.691 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-29T04:18:14.691 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:14.691 INFO:teuthology.orchestra.run.smithi151.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:18:14.691 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:18:14.692 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-29T04:18:14.692 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:14.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:14.692 DEBUG:teuthology.orchestra.run.smithi151:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:18:14.744 INFO:teuthology.orchestra.run.smithi151.stdout:1 2024-03-29T04:18:14.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:14.746 DEBUG:teuthology.orchestra.run.smithi151:> route 2024-03-29T04:18:14.797 INFO:teuthology.orchestra.run.smithi151.stdout:Kernel IP routing table 2024-03-29T04:18:14.797 INFO:teuthology.orchestra.run.smithi151.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:18:14.797 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-29T04:18:14.797 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:18:14.797 INFO:teuthology.orchestra.run.smithi151.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-29T04:18:14.797 INFO:teuthology.orchestra.run.smithi151.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T04:18:14.797 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:18:14.797 INFO:teuthology.orchestra.run.smithi151.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-29T04:18:14.798 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-29T04:18:14.798 INFO:teuthology.orchestra.run.smithi151.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-29T04:18:14.798 INFO:teuthology.orchestra.run.smithi151.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:18:14.798 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:18:14.798 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-03-29T04:18:14.798 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:18:14.798 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:18:14.798 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:18:14.798 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-03-29T04:18:14.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:18:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:18:14.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:18:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:14.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:14.918 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-03-29T04:18:14.923 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-29T04:18:14.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:14.924 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list-id 2024-03-29T04:18:14.970 INFO:teuthology.orchestra.run.smithi151.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-29T04:18:14.971 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:18:14.971 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-03-29T04:18:14.971 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:18:14.971 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-29T04:18:14.971 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-03-29T04:18:15.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:18:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:18:15.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:18:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:15.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:15.085 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-29T04:18:15.141 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:18:15.141 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:18:15.141 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:18:15.141 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:15.141 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-03-29T04:18:15.141 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:15.141 INFO:teuthology.orchestra.run.smithi151.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:18:15.141 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 2e:de:f4:c4:1a:d5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T04:18:15.141 INFO:teuthology.orchestra.run.smithi151.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T04:18:15.141 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:15.141 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::2cde:f4ff:fec4:1ad5/64 scope link 2024-03-29T04:18:15.142 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:15.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:15.143 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-29T04:18:15.197 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:18:15.197 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:18:15.197 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:18:15.197 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:15.197 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-03-29T04:18:15.197 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:15.197 INFO:teuthology.orchestra.run.smithi151.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:18:15.197 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 2e:de:f4:c4:1a:d5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T04:18:15.197 INFO:teuthology.orchestra.run.smithi151.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T04:18:15.197 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:15.197 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::2cde:f4ff:fec4:1ad5/64 scope link 2024-03-29T04:18:15.197 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:18:15.198 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-29T04:18:15.199 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:18:15.199 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-03-29T04:18:15.199 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-29T04:18:15.199 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-29T04:18:15.199 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-29T04:18:15.199 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-29T04:18:15.199 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-29T04:18:15.199 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-03-29T04:18:15.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:18:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:18:15.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:18:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:15.335 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:18:15.335 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-03-29T04:18:15.335 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set brx.1 up 2024-03-29T04:18:15.335 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set dev brx.1 master ceph-brx 2024-03-29T04:18:15.335 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-03-29T04:18:15.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:18:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:18:15.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:18:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:15.439 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-03-29T04:18:15.440 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:18:15.440 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-29T04:18:15.489 INFO:teuthology.orchestra.run.smithi151.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-29T04:18:15.489 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:18:15.489 DEBUG:teuthology.orchestra.run.smithi151:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-29T04:18:15.537 DEBUG:teuthology.orchestra.run.smithi151:> sudo modprobe fuse 2024-03-29T04:18:15.599 DEBUG:teuthology.orchestra.run.smithi151:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:18:15.651 INFO:teuthology.orchestra.run.smithi151.stdout:/sys 2024-03-29T04:18:15.651 INFO:teuthology.orchestra.run.smithi151.stdout:/proc 2024-03-29T04:18:15.651 INFO:teuthology.orchestra.run.smithi151.stdout:/dev 2024-03-29T04:18:15.651 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/pts 2024-03-29T04:18:15.651 INFO:teuthology.orchestra.run.smithi151.stdout:/run 2024-03-29T04:18:15.651 INFO:teuthology.orchestra.run.smithi151.stdout:/ 2024-03-29T04:18:15.651 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/security 2024-03-29T04:18:15.651 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/shm 2024-03-29T04:18:15.651 INFO:teuthology.orchestra.run.smithi151.stdout:/run/lock 2024-03-29T04:18:15.652 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/cgroup 2024-03-29T04:18:15.652 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/pstore 2024-03-29T04:18:15.652 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/bpf 2024-03-29T04:18:15.652 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:18:15.652 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/hugepages 2024-03-29T04:18:15.652 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/mqueue 2024-03-29T04:18:15.652 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/debug 2024-03-29T04:18:15.652 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/tracing 2024-03-29T04:18:15.652 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/fs/nfsd 2024-03-29T04:18:15.652 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/fuse/connections 2024-03-29T04:18:15.652 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/config 2024-03-29T04:18:15.652 INFO:teuthology.orchestra.run.smithi151.stdout:/run/credentials/systemd-sysusers.service 2024-03-29T04:18:15.652 INFO:teuthology.orchestra.run.smithi151.stdout:/run/rpc_pipefs 2024-03-29T04:18:15.652 INFO:teuthology.orchestra.run.smithi151.stdout:/run/user/1000 2024-03-29T04:18:15.653 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph 2024-03-29T04:18:15.653 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-4 2024-03-29T04:18:15.653 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-5 2024-03-29T04:18:15.653 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-6 2024-03-29T04:18:15.653 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-7 2024-03-29T04:18:15.653 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns 2024-03-29T04:18:15.653 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:18:15.653 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:18:15.653 INFO:teuthology.orchestra.run.smithi151.stdout:/home/ubuntu/cephtest/mnt.1 2024-03-29T04:18:15.653 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:18:15.653 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:18:15.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:15.654 DEBUG:teuthology.orchestra.run.smithi151:> ls /sys/fs/fuse/connections 2024-03-29T04:18:15.702 INFO:teuthology.orchestra.run.smithi151.stdout:45 2024-03-29T04:18:15.702 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-03-29T04:18:15.702 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-03-29T04:18:15.747 DEBUG:teuthology.orchestra.run.smithi151:> sudo modprobe fuse 2024-03-29T04:18:15.759 DEBUG:teuthology.orchestra.run.smithi151:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:18:15.806 INFO:teuthology.orchestra.run.smithi151.stdout:/sys 2024-03-29T04:18:15.806 INFO:teuthology.orchestra.run.smithi151.stdout:/proc 2024-03-29T04:18:15.806 INFO:teuthology.orchestra.run.smithi151.stdout:/dev 2024-03-29T04:18:15.806 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/pts 2024-03-29T04:18:15.806 INFO:teuthology.orchestra.run.smithi151.stdout:/run 2024-03-29T04:18:15.806 INFO:teuthology.orchestra.run.smithi151.stdout:/ 2024-03-29T04:18:15.807 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/security 2024-03-29T04:18:15.807 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/shm 2024-03-29T04:18:15.807 INFO:teuthology.orchestra.run.smithi151.stdout:/run/lock 2024-03-29T04:18:15.807 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/cgroup 2024-03-29T04:18:15.807 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/pstore 2024-03-29T04:18:15.807 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/bpf 2024-03-29T04:18:15.807 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:18:15.807 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/hugepages 2024-03-29T04:18:15.807 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/mqueue 2024-03-29T04:18:15.807 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/debug 2024-03-29T04:18:15.807 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/tracing 2024-03-29T04:18:15.807 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/fs/nfsd 2024-03-29T04:18:15.807 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/fuse/connections 2024-03-29T04:18:15.807 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/config 2024-03-29T04:18:15.807 INFO:teuthology.orchestra.run.smithi151.stdout:/run/credentials/systemd-sysusers.service 2024-03-29T04:18:15.808 INFO:teuthology.orchestra.run.smithi151.stdout:/run/rpc_pipefs 2024-03-29T04:18:15.808 INFO:teuthology.orchestra.run.smithi151.stdout:/run/user/1000 2024-03-29T04:18:15.808 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph 2024-03-29T04:18:15.808 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-4 2024-03-29T04:18:15.808 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-5 2024-03-29T04:18:15.808 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-6 2024-03-29T04:18:15.808 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-7 2024-03-29T04:18:15.808 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns 2024-03-29T04:18:15.808 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:18:15.808 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:18:15.808 INFO:teuthology.orchestra.run.smithi151.stdout:/home/ubuntu/cephtest/mnt.1 2024-03-29T04:18:15.808 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:18:15.808 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:18:15.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:15.809 DEBUG:teuthology.orchestra.run.smithi151:> ls /sys/fs/fuse/connections 2024-03-29T04:18:15.819 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi151.stderr:2024-03-29T04:18:15.839+0000 7f7ccffa9580 -1 init, newargv = 0x5616796a9830 newargc=15 2024-03-29T04:18:15.819 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi151.stderr:ceph-fuse[33952]: starting ceph client 2024-03-29T04:18:15.823 INFO:teuthology.orchestra.run.smithi151.stdout:45 2024-03-29T04:18:15.825 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi151.stderr:ceph-fuse[33952]: starting fuse 2024-03-29T04:18:16.925 DEBUG:teuthology.orchestra.run.smithi151:> sudo modprobe fuse 2024-03-29T04:18:16.939 DEBUG:teuthology.orchestra.run.smithi151:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:18:16.987 INFO:teuthology.orchestra.run.smithi151.stdout:/sys 2024-03-29T04:18:16.987 INFO:teuthology.orchestra.run.smithi151.stdout:/proc 2024-03-29T04:18:16.987 INFO:teuthology.orchestra.run.smithi151.stdout:/dev 2024-03-29T04:18:16.987 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/pts 2024-03-29T04:18:16.987 INFO:teuthology.orchestra.run.smithi151.stdout:/run 2024-03-29T04:18:16.987 INFO:teuthology.orchestra.run.smithi151.stdout:/ 2024-03-29T04:18:16.987 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/security 2024-03-29T04:18:16.987 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/shm 2024-03-29T04:18:16.987 INFO:teuthology.orchestra.run.smithi151.stdout:/run/lock 2024-03-29T04:18:16.988 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/cgroup 2024-03-29T04:18:16.988 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/pstore 2024-03-29T04:18:16.988 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/bpf 2024-03-29T04:18:16.988 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:18:16.988 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/hugepages 2024-03-29T04:18:16.988 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/mqueue 2024-03-29T04:18:16.988 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/debug 2024-03-29T04:18:16.988 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/tracing 2024-03-29T04:18:16.988 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/fs/nfsd 2024-03-29T04:18:16.988 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/fuse/connections 2024-03-29T04:18:16.988 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/config 2024-03-29T04:18:16.988 INFO:teuthology.orchestra.run.smithi151.stdout:/run/credentials/systemd-sysusers.service 2024-03-29T04:18:16.988 INFO:teuthology.orchestra.run.smithi151.stdout:/run/rpc_pipefs 2024-03-29T04:18:16.988 INFO:teuthology.orchestra.run.smithi151.stdout:/run/user/1000 2024-03-29T04:18:16.989 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph 2024-03-29T04:18:16.989 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-4 2024-03-29T04:18:16.989 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-5 2024-03-29T04:18:16.989 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-6 2024-03-29T04:18:16.989 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-7 2024-03-29T04:18:16.989 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns 2024-03-29T04:18:16.989 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:18:16.989 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:18:16.989 INFO:teuthology.orchestra.run.smithi151.stdout:/home/ubuntu/cephtest/mnt.1 2024-03-29T04:18:16.989 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:18:16.989 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:18:16.989 INFO:teuthology.orchestra.run.smithi151.stdout:/home/ubuntu/cephtest/mnt.3 2024-03-29T04:18:16.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:16.990 DEBUG:teuthology.orchestra.run.smithi151:> ls /sys/fs/fuse/connections 2024-03-29T04:18:17.038 INFO:teuthology.orchestra.run.smithi151.stdout:45 2024-03-29T04:18:17.038 INFO:teuthology.orchestra.run.smithi151.stdout:46 2024-03-29T04:18:17.038 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-03-29T04:18:17.038 DEBUG:teuthology.orchestra.run.smithi151:> sudo stdin-killer -- python3 -c ' 2024-03-29T04:18:17.038 DEBUG:teuthology.orchestra.run.smithi151:> import glob 2024-03-29T04:18:17.038 DEBUG:teuthology.orchestra.run.smithi151:> import re 2024-03-29T04:18:17.039 DEBUG:teuthology.orchestra.run.smithi151:> import os 2024-03-29T04:18:17.039 DEBUG:teuthology.orchestra.run.smithi151:> import subprocess 2024-03-29T04:18:17.039 DEBUG:teuthology.orchestra.run.smithi151:> 2024-03-29T04:18:17.039 DEBUG:teuthology.orchestra.run.smithi151:> def _find_admin_socket(client_name): 2024-03-29T04:18:17.039 DEBUG:teuthology.orchestra.run.smithi151:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-03-29T04:18:17.039 DEBUG:teuthology.orchestra.run.smithi151:> files = glob.glob(asok_path) 2024-03-29T04:18:17.039 DEBUG:teuthology.orchestra.run.smithi151:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-03-29T04:18:17.039 DEBUG:teuthology.orchestra.run.smithi151:> 2024-03-29T04:18:17.039 DEBUG:teuthology.orchestra.run.smithi151:> # Given a non-glob path, it better be there 2024-03-29T04:18:17.039 DEBUG:teuthology.orchestra.run.smithi151:> if "*" not in asok_path: 2024-03-29T04:18:17.039 DEBUG:teuthology.orchestra.run.smithi151:> assert(len(files) == 1) 2024-03-29T04:18:17.039 DEBUG:teuthology.orchestra.run.smithi151:> return files[0] 2024-03-29T04:18:17.039 DEBUG:teuthology.orchestra.run.smithi151:> 2024-03-29T04:18:17.039 DEBUG:teuthology.orchestra.run.smithi151:> for f in files: 2024-03-29T04:18:17.039 DEBUG:teuthology.orchestra.run.smithi151:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T04:18:17.040 DEBUG:teuthology.orchestra.run.smithi151:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T04:18:17.040 DEBUG:teuthology.orchestra.run.smithi151:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T04:18:17.040 DEBUG:teuthology.orchestra.run.smithi151:> contents = proc_f.read() 2024-03-29T04:18:17.040 DEBUG:teuthology.orchestra.run.smithi151:> if mountpoint in contents: 2024-03-29T04:18:17.040 DEBUG:teuthology.orchestra.run.smithi151:> return f 2024-03-29T04:18:17.040 DEBUG:teuthology.orchestra.run.smithi151:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T04:18:17.040 DEBUG:teuthology.orchestra.run.smithi151:> 2024-03-29T04:18:17.040 DEBUG:teuthology.orchestra.run.smithi151:> print(_find_admin_socket("client.3")) 2024-03-29T04:18:17.040 DEBUG:teuthology.orchestra.run.smithi151:> ' 2024-03-29T04:18:17.159 INFO:teuthology.orchestra.run.smithi151.stdout:/var/run/ceph/ceph-client.3.33952.asok 2024-03-29T04:18:17.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:18:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:18:17.171 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33952.asok 2024-03-29T04:18:17.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:17.171 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33952.asok status 2024-03-29T04:18:17.321 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-03-29T04:18:17.321 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": { 2024-03-29T04:18:17.321 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_sha1": "69e2cd73c52284dfd5096c66b60c6e6b26a3c447", 2024-03-29T04:18:17.321 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_version": "ceph version 19.0.0-2520-g69e2cd73 (69e2cd73c52284dfd5096c66b60c6e6b26a3c447) squid (dev)", 2024-03-29T04:18:17.322 INFO:teuthology.orchestra.run.smithi151.stdout: "entity_id": "3", 2024-03-29T04:18:17.322 INFO:teuthology.orchestra.run.smithi151.stdout: "hostname": "smithi151", 2024-03-29T04:18:17.322 INFO:teuthology.orchestra.run.smithi151.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-03-29T04:18:17.322 INFO:teuthology.orchestra.run.smithi151.stdout: "pid": "33952", 2024-03-29T04:18:17.322 INFO:teuthology.orchestra.run.smithi151.stdout: "root": "/" 2024-03-29T04:18:17.322 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-03-29T04:18:17.322 INFO:teuthology.orchestra.run.smithi151.stdout: "dentry_count": 0, 2024-03-29T04:18:17.322 INFO:teuthology.orchestra.run.smithi151.stdout: "dentry_pinned_count": 0, 2024-03-29T04:18:17.322 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 4562, 2024-03-29T04:18:17.322 INFO:teuthology.orchestra.run.smithi151.stdout: "inst": { 2024-03-29T04:18:17.322 INFO:teuthology.orchestra.run.smithi151.stdout: "name": { 2024-03-29T04:18:17.322 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "client", 2024-03-29T04:18:17.322 INFO:teuthology.orchestra.run.smithi151.stdout: "num": 4562 2024-03-29T04:18:17.322 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-03-29T04:18:17.322 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": { 2024-03-29T04:18:17.323 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v1", 2024-03-29T04:18:17.323 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "192.168.0.2:0", 2024-03-29T04:18:17.323 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 3225702326 2024-03-29T04:18:17.323 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-03-29T04:18:17.323 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-03-29T04:18:17.323 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": { 2024-03-29T04:18:17.323 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v1", 2024-03-29T04:18:17.323 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "192.168.0.2:0", 2024-03-29T04:18:17.323 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 3225702326 2024-03-29T04:18:17.323 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-03-29T04:18:17.323 INFO:teuthology.orchestra.run.smithi151.stdout: "inst_str": "client.4562 192.168.0.2:0/3225702326", 2024-03-29T04:18:17.323 INFO:teuthology.orchestra.run.smithi151.stdout: "addr_str": "192.168.0.2:0/3225702326", 2024-03-29T04:18:17.323 INFO:teuthology.orchestra.run.smithi151.stdout: "inode_count": 1, 2024-03-29T04:18:17.323 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_epoch": 9, 2024-03-29T04:18:17.323 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_epoch": 26, 2024-03-29T04:18:17.324 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_epoch_barrier": 0, 2024-03-29T04:18:17.324 INFO:teuthology.orchestra.run.smithi151.stdout: "blocklisted": false, 2024-03-29T04:18:17.324 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs" 2024-03-29T04:18:17.324 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-03-29T04:18:17.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:17.335 DEBUG:teuthology.orchestra.run.smithi022:> stat --file-system '--printf=%T 2024-03-29T04:18:17.335 DEBUG:teuthology.orchestra.run.smithi022:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T04:18:17.343 INFO:teuthology.orchestra.run.smithi022.stdout:fuseblk 2024-03-29T04:18:17.343 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T04:18:17.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:17.343 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T04:18:17.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:17.402 DEBUG:teuthology.orchestra.run.smithi022:> stat --file-system '--printf=%T 2024-03-29T04:18:17.402 DEBUG:teuthology.orchestra.run.smithi022:> ' -- /home/ubuntu/cephtest/mnt.2 2024-03-29T04:18:17.449 INFO:teuthology.orchestra.run.smithi022.stdout:fuseblk 2024-03-29T04:18:17.449 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-03-29T04:18:17.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:17.449 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-29T04:18:17.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:17.506 DEBUG:teuthology.orchestra.run.smithi151:> stat --file-system '--printf=%T 2024-03-29T04:18:17.506 DEBUG:teuthology.orchestra.run.smithi151:> ' -- /home/ubuntu/cephtest/mnt.1 2024-03-29T04:18:17.514 INFO:teuthology.orchestra.run.smithi151.stdout:fuseblk 2024-03-29T04:18:17.514 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-03-29T04:18:17.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:17.515 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-29T04:18:17.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:17.571 DEBUG:teuthology.orchestra.run.smithi151:> stat --file-system '--printf=%T 2024-03-29T04:18:17.571 DEBUG:teuthology.orchestra.run.smithi151:> ' -- /home/ubuntu/cephtest/mnt.3 2024-03-29T04:18:17.619 INFO:teuthology.orchestra.run.smithi151.stdout:fuseblk 2024-03-29T04:18:17.619 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-03-29T04:18:17.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:17.619 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-29T04:18:17.677 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-29T04:18:17.694 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:17.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f536f842640 1 Processor -- start 2024-03-29T04:18:17.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f536f842640 1 -- start start 2024-03-29T04:18:17.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f536f842640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5368165120 0x7f5368167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:17.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f536f842640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5368160bc0 con 0x7f536815c230 2024-03-29T04:18:17.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f536f842640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f536815c610 con 0x7f536806f390 2024-03-29T04:18:17.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f536f842640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f53680b4fa0 con 0x7f5368165120 2024-03-29T04:18:17.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f536d5de640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f536815c230 0x7f536815fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57914/0 (socket says 172.21.15.22:57914) 2024-03-29T04:18:17.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f536d5de640 1 -- 172.21.15.22:0/155112229 learned_addr learned my addr 172.21.15.22:0/155112229 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:17.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f536dddf640 1 --2- 172.21.15.22:0/155112229 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5368165120 0x7f5368167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:17.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f535ffff640 1 -- 172.21.15.22:0/155112229 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2750936930 0 0) 0x7f5368160bc0 con 0x7f536815c230 2024-03-29T04:18:17.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f535ffff640 1 -- 172.21.15.22:0/155112229 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5354003530 con 0x7f536815c230 2024-03-29T04:18:17.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f535ffff640 1 -- 172.21.15.22:0/155112229 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 868588666 0 0) 0x7f5354003530 con 0x7f536815c230 2024-03-29T04:18:17.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f535ffff640 1 -- 172.21.15.22:0/155112229 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5368165120 msgr2=0x7f5368167510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:17.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f535ffff640 1 --2- 172.21.15.22:0/155112229 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5368165120 0x7f5368167510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:17.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f535ffff640 1 -- 172.21.15.22:0/155112229 >> v1:172.21.15.151:6789/0 conn(0x7f536806f390 legacy=0x7f53681604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:17.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f535ffff640 1 -- 172.21.15.22:0/155112229 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53680d0d30 con 0x7f536815c230 2024-03-29T04:18:17.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f535ffff640 1 -- 172.21.15.22:0/155112229 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f5364001680 con 0x7f536815c230 2024-03-29T04:18:17.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f535ffff640 1 -- 172.21.15.22:0/155112229 >> v1:172.21.15.22:6789/0 conn(0x7f536815c230 legacy=0x7f536815fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:17.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f535ffff640 1 --2- 172.21.15.22:0/155112229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53540039f0 0x7f5354003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:17.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f535ffff640 1 --2- 172.21.15.22:0/155112229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5354004810 0x7f5354004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:17.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f535ffff640 1 --2- 172.21.15.22:0/155112229 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5368165120 0x7f535400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:17.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f535ffff640 1 -- 172.21.15.22:0/155112229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5354005500 con 0x7f5354004810 2024-03-29T04:18:17.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f535ffff640 1 -- 172.21.15.22:0/155112229 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f535400bf10 con 0x7f53540039f0 2024-03-29T04:18:17.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f535ffff640 1 -- 172.21.15.22:0/155112229 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f535400c090 con 0x7f5368165120 2024-03-29T04:18:17.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f536cddd640 1 --2- 172.21.15.22:0/155112229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5354004810 0x7f5354004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:17.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f536dddf640 1 --2- 172.21.15.22:0/155112229 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5368165120 0x7f535400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:17.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f536d5de640 1 --2- 172.21.15.22:0/155112229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53540039f0 0x7f5354003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:17.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f536cddd640 1 -- 172.21.15.22:0/155112229 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5368165120 msgr2=0x7f535400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:17.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f536cddd640 1 --2- 172.21.15.22:0/155112229 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5368165120 0x7f535400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:17.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f536cddd640 1 -- 172.21.15.22:0/155112229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53540039f0 msgr2=0x7f5354003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:17.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f536cddd640 1 --2- 172.21.15.22:0/155112229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53540039f0 0x7f5354003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:17.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f536cddd640 1 -- 172.21.15.22:0/155112229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5354003530 con 0x7f5354004810 2024-03-29T04:18:17.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f536dddf640 1 --2- 172.21.15.22:0/155112229 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5368165120 0x7f535400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:17.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.848+0000 7f536d5de640 1 --2- 172.21.15.22:0/155112229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53540039f0 0x7f5354003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:17.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.852+0000 7f536cddd640 1 --2- 172.21.15.22:0/155112229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5354004810 0x7f5354004c90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5358001780 tx=0x7f535801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6568892bcea1fd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:17.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.852+0000 7f535ffff640 1 -- 172.21.15.22:0/155112229 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53580013a0 con 0x7f5354004810 2024-03-29T04:18:17.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.852+0000 7f535ffff640 1 -- 172.21.15.22:0/155112229 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53580052f0 con 0x7f5354004810 2024-03-29T04:18:17.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.852+0000 7f535ffff640 1 -- 172.21.15.22:0/155112229 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358003840 con 0x7f5354004810 2024-03-29T04:18:17.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.852+0000 7f536f842640 1 -- 172.21.15.22:0/155112229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5354004810 msgr2=0x7f5354004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:17.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.852+0000 7f536f842640 1 --2- 172.21.15.22:0/155112229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5354004810 0x7f5354004c90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5358001780 tx=0x7f535801c420 comp rx=0 tx=0).stop 2024-03-29T04:18:17.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.852+0000 7f536dddf640 1 -- 172.21.15.22:0/155112229 reap_dead start 2024-03-29T04:18:17.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.852+0000 7f536f842640 1 -- 172.21.15.22:0/155112229 shutdown_connections 2024-03-29T04:18:17.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.852+0000 7f536f842640 1 -- 172.21.15.22:0/155112229 >> 172.21.15.22:0/155112229 conn(0x7f53680b3c80 msgr2=0x7f53680b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:17.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.852+0000 7f536f842640 1 -- 172.21.15.22:0/155112229 shutdown_connections 2024-03-29T04:18:17.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.852+0000 7f536f842640 1 -- 172.21.15.22:0/155112229 wait complete. 2024-03-29T04:18:17.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.852+0000 7f536f842640 1 Processor -- start 2024-03-29T04:18:17.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.852+0000 7f536f842640 1 -- start start 2024-03-29T04:18:17.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f536f842640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5354004810 0x7f5368156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:17.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f536f842640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5368153410 0x7f536814f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:17.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f536f842640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53680b6f00 0x7f5368154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:17.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f536f842640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f53680cc7a0 con 0x7f5354004810 2024-03-29T04:18:17.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f536f842640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f53680cf4f0 con 0x7f53680b6f00 2024-03-29T04:18:17.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f536f842640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f53680d1840 con 0x7f5368153410 2024-03-29T04:18:17.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f536d5de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5354004810 0x7f5368156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:17.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f536d5de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5354004810 0x7f5368156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59334/0 (socket says 172.21.15.22:59334) 2024-03-29T04:18:17.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f536d5de640 1 -- 172.21.15.22:0/4162400179 learned_addr learned my addr 172.21.15.22:0/4162400179 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:17.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f536cddd640 1 --2- 172.21.15.22:0/4162400179 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5368153410 0x7f536814f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:17.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f536dddf640 1 --2- 172.21.15.22:0/4162400179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53680b6f00 0x7f5368154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:17.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f536d5de640 1 -- 172.21.15.22:0/4162400179 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5368153410 msgr2=0x7f536814f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:17.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f536d5de640 1 --2- 172.21.15.22:0/4162400179 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5368153410 0x7f536814f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:17.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f536d5de640 1 -- 172.21.15.22:0/4162400179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53680b6f00 msgr2=0x7f5368154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:17.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f536d5de640 1 --2- 172.21.15.22:0/4162400179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53680b6f00 0x7f5368154e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:17.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f536d5de640 1 -- 172.21.15.22:0/4162400179 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53680b5e60 con 0x7f5354004810 2024-03-29T04:18:17.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f536d5de640 1 --2- 172.21.15.22:0/4162400179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5354004810 0x7f5368156d60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f536400e7f0 tx=0x7f536400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:17.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f535dffb640 1 -- 172.21.15.22:0/4162400179 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536400c8d0 con 0x7f5354004810 2024-03-29T04:18:17.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f535dffb640 1 -- 172.21.15.22:0/4162400179 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5364018070 con 0x7f5354004810 2024-03-29T04:18:17.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f536f842640 1 -- 172.21.15.22:0/4162400179 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53680b6140 con 0x7f5354004810 2024-03-29T04:18:17.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f535dffb640 1 -- 172.21.15.22:0/4162400179 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536400cc70 con 0x7f5354004810 2024-03-29T04:18:17.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f536f842640 1 -- 172.21.15.22:0/4162400179 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53680b6700 con 0x7f5354004810 2024-03-29T04:18:17.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f536f842640 1 -- 172.21.15.22:0/4162400179 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5368068890 con 0x7f5354004810 2024-03-29T04:18:17.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f535dffb640 1 -- 172.21.15.22:0/4162400179 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f536401d460 con 0x7f5354004810 2024-03-29T04:18:17.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f535dffb640 1 --2- 172.21.15.22:0/4162400179 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f534c067ff0 0x7f534c06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:17.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f535dffb640 1 -- 172.21.15.22:0/4162400179 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f5364086800 con 0x7f5354004810 2024-03-29T04:18:17.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.856+0000 7f536cddd640 1 --2- 172.21.15.22:0/4162400179 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f534c067ff0 0x7f534c06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:17.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.860+0000 7f536cddd640 1 --2- 172.21.15.22:0/4162400179 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f534c067ff0 0x7f534c06a4b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5358001750 tx=0x7f53580011a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:17.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:17.864+0000 7f535dffb640 1 -- 172.21.15.22:0/4162400179 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f5364056c40 con 0x7f5354004810 2024-03-29T04:18:18.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.024+0000 7f536f842640 1 -- 172.21.15.22:0/4162400179 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f536815a430 con 0x7f5354004810 2024-03-29T04:18:18.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.028+0000 7f535dffb640 1 -- 172.21.15.22:0/4162400179 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18847 (secure 0 0 0) 0x7f5364056540 con 0x7f5354004810 2024-03-29T04:18:18.008 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:18.008 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:18:16.893439+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:18.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.028+0000 7f536f842640 1 -- 172.21.15.22:0/4162400179 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f534c067ff0 msgr2=0x7f534c06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.028+0000 7f536f842640 1 --2- 172.21.15.22:0/4162400179 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f534c067ff0 0x7f534c06a4b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5358001750 tx=0x7f53580011a0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.028+0000 7f536f842640 1 -- 172.21.15.22:0/4162400179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5354004810 msgr2=0x7f5368156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.028+0000 7f536f842640 1 --2- 172.21.15.22:0/4162400179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5354004810 0x7f5368156d60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f536400e7f0 tx=0x7f536400ecb0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.028+0000 7f536f842640 1 -- 172.21.15.22:0/4162400179 shutdown_connections 2024-03-29T04:18:18.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.028+0000 7f536f842640 1 --2- 172.21.15.22:0/4162400179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53680b6f00 0x7f5368154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.028+0000 7f536f842640 1 --2- 172.21.15.22:0/4162400179 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5368153410 0x7f536814f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.028+0000 7f536f842640 1 --2- 172.21.15.22:0/4162400179 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f534c067ff0 0x7f534c06a4b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.028+0000 7f536f842640 1 --2- 172.21.15.22:0/4162400179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5354004810 0x7f5368156d60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.028+0000 7f536f842640 1 -- 172.21.15.22:0/4162400179 >> 172.21.15.22:0/4162400179 conn(0x7f53680b3c80 msgr2=0x7f53680b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:18.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.028+0000 7f536f842640 1 -- 172.21.15.22:0/4162400179 shutdown_connections 2024-03-29T04:18:18.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.028+0000 7f536f842640 1 -- 172.21.15.22:0/4162400179 wait complete. 2024-03-29T04:18:18.028 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:17:34.034142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:17:34.886659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:17:38.628632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:18:18.029 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:18:18.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.168+0000 7fd0ae3cd640 1 Processor -- start 2024-03-29T04:18:18.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.168+0000 7fd0ae3cd640 1 -- start start 2024-03-29T04:18:18.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.172+0000 7fd0ae3cd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd0a80b9e30 0x7fd0a80cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:18.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.172+0000 7fd0ae3cd640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0a80cf4f0 0x7fd0a80d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:18.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.172+0000 7fd0ae3cd640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a806c850 con 0x7fd0a815e4c0 2024-03-29T04:18:18.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.172+0000 7fd0ae3cd640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fd0a80ba290 con 0x7fd0a80b9e30 2024-03-29T04:18:18.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.172+0000 7fd0ae3cd640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a815e8a0 con 0x7fd0a80cf4f0 2024-03-29T04:18:18.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.172+0000 7fd0a77fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd0a80b9e30 0x7fd0a80cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:18.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.172+0000 7fd0a7fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fd0a815e4c0 0x7fd0a806c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57918/0 (socket says 172.21.15.22:57918) 2024-03-29T04:18:18.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.172+0000 7fd0a7fff640 1 -- 172.21.15.22:0/1381429773 learned_addr learned my addr 172.21.15.22:0/1381429773 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:18.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.172+0000 7fd0ac96a640 1 --2- 172.21.15.22:0/1381429773 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0a80cf4f0 0x7fd0a80d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:18.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.172+0000 7fd0a77fe640 1 -- 172.21.15.22:0/1381429773 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0a80cf4f0 msgr2=0x7fd0a80d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:18.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.172+0000 7fd0a77fe640 1 --2- 172.21.15.22:0/1381429773 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0a80cf4f0 0x7fd0a80d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.172+0000 7fd0a77fe640 1 -- 172.21.15.22:0/1381429773 >> v1:172.21.15.22:6789/0 conn(0x7fd0a815e4c0 legacy=0x7fd0a806c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.172+0000 7fd0a77fe640 1 -- 172.21.15.22:0/1381429773 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a806cb30 con 0x7fd0a80b9e30 2024-03-29T04:18:18.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.172+0000 7fd0a6ffd640 1 -- 172.21.15.22:0/1381429773 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2808676640 0 0) 0x7fd0a806c850 con 0x7fd0a815e4c0 2024-03-29T04:18:18.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.172+0000 7fd0a77fe640 1 --2- 172.21.15.22:0/1381429773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a80b9e30 0x7fd0a80cefb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0097c0 tx=0x7fd09c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf71484b6ec0ed99 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:18.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.172+0000 7fd0a6ffd640 1 -- 172.21.15.22:0/1381429773 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0b0070 con 0x7fd0a80b9e30 2024-03-29T04:18:18.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.172+0000 7fd0a6ffd640 1 -- 172.21.15.22:0/1381429773 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd09c0ab600 con 0x7fd0a80b9e30 2024-03-29T04:18:18.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.172+0000 7fd0a6ffd640 1 -- 172.21.15.22:0/1381429773 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0ab9d0 con 0x7fd0a80b9e30 2024-03-29T04:18:18.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0ae3cd640 1 -- 172.21.15.22:0/1381429773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a80b9e30 msgr2=0x7fd0a80cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0ae3cd640 1 --2- 172.21.15.22:0/1381429773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a80b9e30 0x7fd0a80cefb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0097c0 tx=0x7fd09c01c4a0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0ae3cd640 1 -- 172.21.15.22:0/1381429773 shutdown_connections 2024-03-29T04:18:18.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0ae3cd640 1 --2- 172.21.15.22:0/1381429773 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0a80cf4f0 0x7fd0a80d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0ae3cd640 1 --2- 172.21.15.22:0/1381429773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a80b9e30 0x7fd0a80cefb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0ae3cd640 1 -- 172.21.15.22:0/1381429773 >> 172.21.15.22:0/1381429773 conn(0x7fd0a80b6410 msgr2=0x7fd0a80d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:18.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0ae3cd640 1 -- 172.21.15.22:0/1381429773 shutdown_connections 2024-03-29T04:18:18.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0ae3cd640 1 -- 172.21.15.22:0/1381429773 wait complete. 2024-03-29T04:18:18.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0ae3cd640 1 Processor -- start 2024-03-29T04:18:18.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0ae3cd640 1 -- start start 2024-03-29T04:18:18.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0ae3cd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a80b9e30 0x7fd0a8161310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:18.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0ae3cd640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0a80cf4f0 0x7fd0a80c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:18.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0ae3cd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0a815e4c0 0x7fd0a814fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:18.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0ae3cd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a80d3ed0 con 0x7fd0a80b9e30 2024-03-29T04:18:18.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0ae3cd640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a80d3d50 con 0x7fd0a815e4c0 2024-03-29T04:18:18.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0ae3cd640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a80d4050 con 0x7fd0a80cf4f0 2024-03-29T04:18:18.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0a7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a80b9e30 0x7fd0a8161310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:18.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0a7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a80b9e30 0x7fd0a8161310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59344/0 (socket says 172.21.15.22:59344) 2024-03-29T04:18:18.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0a7fff640 1 -- 172.21.15.22:0/442952804 learned_addr learned my addr 172.21.15.22:0/442952804 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:18.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0a77fe640 1 --2- 172.21.15.22:0/442952804 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0a80cf4f0 0x7fd0a80c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0ac96a640 1 --2- 172.21.15.22:0/442952804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0a815e4c0 0x7fd0a814fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0a7fff640 1 -- 172.21.15.22:0/442952804 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0a80cf4f0 msgr2=0x7fd0a80c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0a7fff640 1 --2- 172.21.15.22:0/442952804 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0a80cf4f0 0x7fd0a80c9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0a7fff640 1 -- 172.21.15.22:0/442952804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0a815e4c0 msgr2=0x7fd0a814fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0a7fff640 1 --2- 172.21.15.22:0/442952804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0a815e4c0 0x7fd0a814fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0a7fff640 1 -- 172.21.15.22:0/442952804 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a8161850 con 0x7fd0a80b9e30 2024-03-29T04:18:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0a7fff640 1 --2- 172.21.15.22:0/442952804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a80b9e30 0x7fd0a8161310 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd09800c7f0 tx=0x7fd09800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0a4ff9640 1 -- 172.21.15.22:0/442952804 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098018070 con 0x7fd0a80b9e30 2024-03-29T04:18:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0a4ff9640 1 -- 172.21.15.22:0/442952804 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd09800a640 con 0x7fd0a80b9e30 2024-03-29T04:18:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0ae3cd640 1 -- 172.21.15.22:0/442952804 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a8160550 con 0x7fd0a80b9e30 2024-03-29T04:18:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0a4ff9640 1 -- 172.21.15.22:0/442952804 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09800aa30 con 0x7fd0a80b9e30 2024-03-29T04:18:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.176+0000 7fd0ae3cd640 1 -- 172.21.15.22:0/442952804 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a80c8a70 con 0x7fd0a80b9e30 2024-03-29T04:18:18.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.180+0000 7fd0ae3cd640 1 -- 172.21.15.22:0/442952804 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a8068900 con 0x7fd0a80b9e30 2024-03-29T04:18:18.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.180+0000 7fd0a4ff9640 1 -- 172.21.15.22:0/442952804 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fd09801b070 con 0x7fd0a80b9e30 2024-03-29T04:18:18.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.180+0000 7fd0a4ff9640 1 --2- 172.21.15.22:0/442952804 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd094067ff0 0x7fd09406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:18.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.180+0000 7fd0a4ff9640 1 -- 172.21.15.22:0/442952804 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fd098085f50 con 0x7fd0a80b9e30 2024-03-29T04:18:18.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.180+0000 7fd0a77fe640 1 --2- 172.21.15.22:0/442952804 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd094067ff0 0x7fd09406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:18.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.180+0000 7fd0a77fe640 1 --2- 172.21.15.22:0/442952804 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd094067ff0 0x7fd09406a4b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0ad000 tx=0x7fd09c0a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:18.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.184+0000 7fd0a4ff9640 1 -- 172.21.15.22:0/442952804 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fd098056260 con 0x7fd0a80b9e30 2024-03-29T04:18:18.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.344+0000 7fd0ae3cd640 1 -- 172.21.15.22:0/442952804 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd0a8155340 con 0x7fd0a80b9e30 2024-03-29T04:18:18.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.344+0000 7fd0a4ff9640 1 -- 172.21.15.22:0/442952804 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7fd098055b60 con 0x7fd0a80b9e30 2024-03-29T04:18:18.327 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:18:18.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.348+0000 7fd0ae3cd640 1 -- 172.21.15.22:0/442952804 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd094067ff0 msgr2=0x7fd09406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.348+0000 7fd0ae3cd640 1 --2- 172.21.15.22:0/442952804 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd094067ff0 0x7fd09406a4b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0ad000 tx=0x7fd09c0a6000 comp rx=0 tx=0).stop 2024-03-29T04:18:18.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.348+0000 7fd0ae3cd640 1 -- 172.21.15.22:0/442952804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a80b9e30 msgr2=0x7fd0a8161310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.348+0000 7fd0ae3cd640 1 --2- 172.21.15.22:0/442952804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a80b9e30 0x7fd0a8161310 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd09800c7f0 tx=0x7fd09800ccb0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.348+0000 7fd0ae3cd640 1 -- 172.21.15.22:0/442952804 shutdown_connections 2024-03-29T04:18:18.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.348+0000 7fd0ae3cd640 1 --2- 172.21.15.22:0/442952804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0a815e4c0 0x7fd0a814fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.348+0000 7fd0ae3cd640 1 --2- 172.21.15.22:0/442952804 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd0a80cf4f0 0x7fd0a80c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.348+0000 7fd0ae3cd640 1 --2- 172.21.15.22:0/442952804 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd094067ff0 0x7fd09406a4b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.348+0000 7fd0ae3cd640 1 --2- 172.21.15.22:0/442952804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a80b9e30 0x7fd0a8161310 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.348+0000 7fd0ae3cd640 1 -- 172.21.15.22:0/442952804 >> 172.21.15.22:0/442952804 conn(0x7fd0a80b6410 msgr2=0x7fd0a8154470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:18.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.348+0000 7fd0ae3cd640 1 -- 172.21.15.22:0/442952804 shutdown_connections 2024-03-29T04:18:18.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.348+0000 7fd0ae3cd640 1 -- 172.21.15.22:0/442952804 wait complete. 2024-03-29T04:18:18.347 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:18:18.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.488+0000 7ff9912aa640 1 Processor -- start 2024-03-29T04:18:18.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.488+0000 7ff9912aa640 1 -- start start 2024-03-29T04:18:18.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.492+0000 7ff9912aa640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff98c06cd80 0x7ff98c165de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:18.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.492+0000 7ff9912aa640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff98c0b3660 con 0x7ff98c0b4870 2024-03-29T04:18:18.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.492+0000 7ff9912aa640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff98c0b3940 con 0x7ff98c144930 2024-03-29T04:18:18.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.492+0000 7ff9912aa640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7ff98c0cdb90 con 0x7ff98c06cd80 2024-03-29T04:18:18.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.492+0000 7ff9827fc640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff98c0b4870 0x7ff98c06c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57932/0 (socket says 172.21.15.22:57932) 2024-03-29T04:18:18.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.492+0000 7ff9827fc640 1 -- 172.21.15.22:0/4187877904 learned_addr learned my addr 172.21.15.22:0/4187877904 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:18.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.492+0000 7ff98b7fe640 1 --2- 172.21.15.22:0/4187877904 >> v2:172.21.15.22:3300/0 conn(0x7ff98c06cd80 0x7ff98c165de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:18.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.492+0000 7ff98a7fc640 1 -- 172.21.15.22:0/4187877904 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1500945856 0 0) 0x7ff98c0b3660 con 0x7ff98c0b4870 2024-03-29T04:18:18.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.492+0000 7ff98a7fc640 1 -- 172.21.15.22:0/4187877904 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff978005740 con 0x7ff98c0b4870 2024-03-29T04:18:18.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.492+0000 7ff98b7fe640 1 -- 172.21.15.22:0/4187877904 >> v1:172.21.15.151:6789/0 conn(0x7ff98c144930 legacy=0x7ff98c0b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.492+0000 7ff98b7fe640 1 -- 172.21.15.22:0/4187877904 >> v1:172.21.15.22:6789/0 conn(0x7ff98c0b4870 legacy=0x7ff98c06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.492+0000 7ff98b7fe640 1 -- 172.21.15.22:0/4187877904 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff98c0b3c20 con 0x7ff98c06cd80 2024-03-29T04:18:18.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff98b7fe640 1 --2- 172.21.15.22:0/4187877904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff98c06cd80 0x7ff98c165de0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff984009d90 tx=0x7ff98401c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7352d525b905cd86 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:18.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff98a7fc640 1 -- 172.21.15.22:0/4187877904 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9840b2070 con 0x7ff98c06cd80 2024-03-29T04:18:18.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff98a7fc640 1 -- 172.21.15.22:0/4187877904 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9840a5860 con 0x7ff98c06cd80 2024-03-29T04:18:18.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff98a7fc640 1 -- 172.21.15.22:0/4187877904 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9840a5c30 con 0x7ff98c06cd80 2024-03-29T04:18:18.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff9912aa640 1 -- 172.21.15.22:0/4187877904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff98c06cd80 msgr2=0x7ff98c165de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff9912aa640 1 --2- 172.21.15.22:0/4187877904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff98c06cd80 0x7ff98c165de0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff984009d90 tx=0x7ff98401c880 comp rx=0 tx=0).stop 2024-03-29T04:18:18.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff9912aa640 1 -- 172.21.15.22:0/4187877904 shutdown_connections 2024-03-29T04:18:18.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff9912aa640 1 --2- 172.21.15.22:0/4187877904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff98c06cd80 0x7ff98c165de0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff9912aa640 1 -- 172.21.15.22:0/4187877904 >> 172.21.15.22:0/4187877904 conn(0x7ff98c0d00a0 msgr2=0x7ff98c0d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:18.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff9912aa640 1 -- 172.21.15.22:0/4187877904 shutdown_connections 2024-03-29T04:18:18.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff9912aa640 1 -- 172.21.15.22:0/4187877904 wait complete. 2024-03-29T04:18:18.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff9912aa640 1 Processor -- start 2024-03-29T04:18:18.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff9912aa640 1 -- start start 2024-03-29T04:18:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff9912aa640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff98c06cd80 0x7ff98c15aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff9912aa640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff98c0b4870 0x7ff98c133cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff9912aa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff98c144930 0x7ff98c0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff9912aa640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff98c06e2d0 con 0x7ff98c144930 2024-03-29T04:18:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff9912aa640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff98c06e150 con 0x7ff98c06cd80 2024-03-29T04:18:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff9912aa640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff98c06e450 con 0x7ff98c0b4870 2024-03-29T04:18:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff98b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff98c144930 0x7ff98c0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff98b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff98c144930 0x7ff98c0ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59350/0 (socket says 172.21.15.22:59350) 2024-03-29T04:18:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff98b7fe640 1 -- 172.21.15.22:0/4076174285 learned_addr learned my addr 172.21.15.22:0/4076174285 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff98affd640 1 --2- 172.21.15.22:0/4076174285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff98c06cd80 0x7ff98c15aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff9827fc640 1 --2- 172.21.15.22:0/4076174285 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff98c0b4870 0x7ff98c133cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff98b7fe640 1 -- 172.21.15.22:0/4076174285 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff98c0b4870 msgr2=0x7ff98c133cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff98b7fe640 1 --2- 172.21.15.22:0/4076174285 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff98c0b4870 0x7ff98c133cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff98b7fe640 1 -- 172.21.15.22:0/4076174285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff98c06cd80 msgr2=0x7ff98c15aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff98b7fe640 1 --2- 172.21.15.22:0/4076174285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff98c06cd80 0x7ff98c15aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff98b7fe640 1 -- 172.21.15.22:0/4076174285 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff98c15af60 con 0x7ff98c144930 2024-03-29T04:18:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.496+0000 7ff98b7fe640 1 --2- 172.21.15.22:0/4076174285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff98c144930 0x7ff98c0ce020 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff98401a510 tx=0x7ff984019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.500+0000 7ff983fff640 1 -- 172.21.15.22:0/4076174285 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9840b2050 con 0x7ff98c144930 2024-03-29T04:18:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.500+0000 7ff983fff640 1 -- 172.21.15.22:0/4076174285 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9840be070 con 0x7ff98c144930 2024-03-29T04:18:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.500+0000 7ff9912aa640 1 -- 172.21.15.22:0/4076174285 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff98c13e500 con 0x7ff98c144930 2024-03-29T04:18:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.500+0000 7ff983fff640 1 -- 172.21.15.22:0/4076174285 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9840ad850 con 0x7ff98c144930 2024-03-29T04:18:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.500+0000 7ff9912aa640 1 -- 172.21.15.22:0/4076174285 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff98c1335c0 con 0x7ff98c144930 2024-03-29T04:18:18.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.500+0000 7ff9912aa640 1 -- 172.21.15.22:0/4076174285 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff98c068900 con 0x7ff98c144930 2024-03-29T04:18:18.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.500+0000 7ff983fff640 1 -- 172.21.15.22:0/4076174285 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff9840ad9f0 con 0x7ff98c144930 2024-03-29T04:18:18.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.500+0000 7ff983fff640 1 --2- 172.21.15.22:0/4076174285 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff96c067fc0 0x7ff96c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:18.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.500+0000 7ff983fff640 1 -- 172.21.15.22:0/4076174285 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7ff984121ad0 con 0x7ff98c144930 2024-03-29T04:18:18.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.500+0000 7ff98affd640 1 --2- 172.21.15.22:0/4076174285 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff96c067fc0 0x7ff96c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:18.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.500+0000 7ff98affd640 1 --2- 172.21.15.22:0/4076174285 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff96c067fc0 0x7ff96c06a480 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff98c144350 tx=0x7ff97c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:18.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.504+0000 7ff983fff640 1 -- 172.21.15.22:0/4076174285 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff984016560 con 0x7ff98c144930 2024-03-29T04:18:18.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.664+0000 7ff9912aa640 1 -- 172.21.15.22:0/4076174285 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff98c13e6d0 con 0x7ff98c144930 2024-03-29T04:18:18.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.664+0000 7ff983fff640 1 -- 172.21.15.22:0/4076174285 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7ff9840f1fa0 con 0x7ff98c144930 2024-03-29T04:18:18.647 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:18.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.668+0000 7ff9912aa640 1 -- 172.21.15.22:0/4076174285 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff96c067fc0 msgr2=0x7ff96c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.668+0000 7ff9912aa640 1 --2- 172.21.15.22:0/4076174285 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff96c067fc0 0x7ff96c06a480 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff98c144350 tx=0x7ff97c009040 comp rx=0 tx=0).stop 2024-03-29T04:18:18.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.668+0000 7ff9912aa640 1 -- 172.21.15.22:0/4076174285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff98c144930 msgr2=0x7ff98c0ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.668+0000 7ff9912aa640 1 --2- 172.21.15.22:0/4076174285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff98c144930 0x7ff98c0ce020 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff98401a510 tx=0x7ff984019680 comp rx=0 tx=0).stop 2024-03-29T04:18:18.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.668+0000 7ff9912aa640 1 -- 172.21.15.22:0/4076174285 shutdown_connections 2024-03-29T04:18:18.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.668+0000 7ff9912aa640 1 --2- 172.21.15.22:0/4076174285 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff96c067fc0 0x7ff96c06a480 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.668+0000 7ff9912aa640 1 --2- 172.21.15.22:0/4076174285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff98c144930 0x7ff98c0ce020 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.668+0000 7ff9912aa640 1 --2- 172.21.15.22:0/4076174285 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff98c0b4870 0x7ff98c133cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.668+0000 7ff9912aa640 1 --2- 172.21.15.22:0/4076174285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff98c06cd80 0x7ff98c15aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.668+0000 7ff9912aa640 1 -- 172.21.15.22:0/4076174285 >> 172.21.15.22:0/4076174285 conn(0x7ff98c0d00a0 msgr2=0x7ff98c0cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:18.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.668+0000 7ff9912aa640 1 -- 172.21.15.22:0/4076174285 shutdown_connections 2024-03-29T04:18:18.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.668+0000 7ff9912aa640 1 -- 172.21.15.22:0/4076174285 wait complete. 2024-03-29T04:18:18.666 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:18:18.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.804+0000 7f3076cbd640 1 Processor -- start 2024-03-29T04:18:18.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.804+0000 7f3076cbd640 1 -- start start 2024-03-29T04:18:18.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.804+0000 7f3076cbd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f30700b9e30 0x7f30700cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:18.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.804+0000 7f3076cbd640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f307006c850 con 0x7f30700cf4f0 2024-03-29T04:18:18.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.804+0000 7f3076cbd640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f307006cb30 con 0x7f307015e4c0 2024-03-29T04:18:18.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.804+0000 7f3076cbd640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f30700ba290 con 0x7f30700b9e30 2024-03-29T04:18:18.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.804+0000 7f306ffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f30700b9e30 0x7f30700cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:18.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.804+0000 7f307525a640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f30700cf4f0 0x7f30700d2d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57946/0 (socket says 172.21.15.22:57946) 2024-03-29T04:18:18.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.804+0000 7f306ffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f30700b9e30 0x7f30700cefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59358/0 (socket says 172.21.15.22:59358) 2024-03-29T04:18:18.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.804+0000 7f307525a640 1 -- 172.21.15.22:0/3598288945 learned_addr learned my addr 172.21.15.22:0/3598288945 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:18.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.804+0000 7f3074a59640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f307015e4c0 0x7f307006c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.22:60384/0 (socket says 172.21.15.22:60384) 2024-03-29T04:18:18.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.804+0000 7f306ffff640 1 -- 172.21.15.22:0/3598288945 >> v1:172.21.15.151:6789/0 conn(0x7f307015e4c0 legacy=0x7f307006c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.804+0000 7f306ffff640 1 -- 172.21.15.22:0/3598288945 >> v1:172.21.15.22:6789/0 conn(0x7f30700cf4f0 legacy=0x7f30700d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.804+0000 7f306ffff640 1 -- 172.21.15.22:0/3598288945 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307006ce10 con 0x7f30700b9e30 2024-03-29T04:18:18.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f306ffff640 1 --2- 172.21.15.22:0/3598288945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30700b9e30 0x7f30700cefb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f30580097c0 tx=0x7f305801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0aa3f0acf7beea6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:18.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f306f7fe640 1 -- 172.21.15.22:0/3598288945 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30580b0070 con 0x7f30700b9e30 2024-03-29T04:18:18.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f306f7fe640 1 -- 172.21.15.22:0/3598288945 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30580ab600 con 0x7f30700b9e30 2024-03-29T04:18:18.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f306f7fe640 1 -- 172.21.15.22:0/3598288945 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30580ab9d0 con 0x7f30700b9e30 2024-03-29T04:18:18.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f3076cbd640 1 -- 172.21.15.22:0/3598288945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30700b9e30 msgr2=0x7f30700cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f3076cbd640 1 --2- 172.21.15.22:0/3598288945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30700b9e30 0x7f30700cefb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f30580097c0 tx=0x7f305801c4a0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f3076cbd640 1 -- 172.21.15.22:0/3598288945 shutdown_connections 2024-03-29T04:18:18.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f3076cbd640 1 --2- 172.21.15.22:0/3598288945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30700b9e30 0x7f30700cefb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f3076cbd640 1 -- 172.21.15.22:0/3598288945 >> 172.21.15.22:0/3598288945 conn(0x7f30700b6410 msgr2=0x7f30700d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:18.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f3076cbd640 1 -- 172.21.15.22:0/3598288945 shutdown_connections 2024-03-29T04:18:18.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f3076cbd640 1 -- 172.21.15.22:0/3598288945 wait complete. 2024-03-29T04:18:18.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f3076cbd640 1 Processor -- start 2024-03-29T04:18:18.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f3076cbd640 1 -- start start 2024-03-29T04:18:18.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f3076cbd640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30700b9e30 0x7f30700cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:18.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f3076cbd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30700cf4f0 0x7f307014bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:18.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f3076cbd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f307015e4c0 0x7f3070160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:18.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f3076cbd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f307015e8a0 con 0x7f307015e4c0 2024-03-29T04:18:18.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f3076cbd640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f30700d3f20 con 0x7f30700cf4f0 2024-03-29T04:18:18.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f3076cbd640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f30700d40a0 con 0x7f30700b9e30 2024-03-29T04:18:18.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f307525a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f307015e4c0 0x7f3070160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:18.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f307525a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f307015e4c0 0x7f3070160750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59374/0 (socket says 172.21.15.22:59374) 2024-03-29T04:18:18.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f307525a640 1 -- 172.21.15.22:0/526545962 learned_addr learned my addr 172.21.15.22:0/526545962 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:18.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.808+0000 7f306ffff640 1 --2- 172.21.15.22:0/526545962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30700cf4f0 0x7f307014bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:18.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.812+0000 7f3074a59640 1 --2- 172.21.15.22:0/526545962 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30700b9e30 0x7f30700cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:18.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.812+0000 7f307525a640 1 -- 172.21.15.22:0/526545962 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30700b9e30 msgr2=0x7f30700cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.812+0000 7f307525a640 1 --2- 172.21.15.22:0/526545962 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30700b9e30 0x7f30700cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.812+0000 7f307525a640 1 -- 172.21.15.22:0/526545962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30700cf4f0 msgr2=0x7f307014bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.812+0000 7f307525a640 1 --2- 172.21.15.22:0/526545962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30700cf4f0 0x7f307014bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.812+0000 7f307525a640 1 -- 172.21.15.22:0/526545962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30700c7f40 con 0x7f307015e4c0 2024-03-29T04:18:18.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.812+0000 7f306ffff640 1 --2- 172.21.15.22:0/526545962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30700cf4f0 0x7f307014bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:18.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.812+0000 7f307525a640 1 --2- 172.21.15.22:0/526545962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f307015e4c0 0x7f3070160750 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f306800cbc0 tx=0x7f306800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:18.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.812+0000 7f306d7fa640 1 -- 172.21.15.22:0/526545962 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068017070 con 0x7f307015e4c0 2024-03-29T04:18:18.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.812+0000 7f306d7fa640 1 -- 172.21.15.22:0/526545962 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f306800ad40 con 0x7f307015e4c0 2024-03-29T04:18:18.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.812+0000 7f306d7fa640 1 -- 172.21.15.22:0/526545962 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068012600 con 0x7f307015e4c0 2024-03-29T04:18:18.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.812+0000 7f3076cbd640 1 -- 172.21.15.22:0/526545962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30700c8220 con 0x7f307015e4c0 2024-03-29T04:18:18.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.812+0000 7f3076cbd640 1 -- 172.21.15.22:0/526545962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f307014c520 con 0x7f307015e4c0 2024-03-29T04:18:18.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.812+0000 7f3076cbd640 1 -- 172.21.15.22:0/526545962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3070068900 con 0x7f307015e4c0 2024-03-29T04:18:18.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.812+0000 7f306d7fa640 1 -- 172.21.15.22:0/526545962 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f306801a070 con 0x7f307015e4c0 2024-03-29T04:18:18.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.812+0000 7f306d7fa640 1 --2- 172.21.15.22:0/526545962 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3050067fc0 0x7f305006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:18.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.812+0000 7f306d7fa640 1 -- 172.21.15.22:0/526545962 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f3068086530 con 0x7f307015e4c0 2024-03-29T04:18:18.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.812+0000 7f3074a59640 1 --2- 172.21.15.22:0/526545962 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3050067fc0 0x7f305006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:18.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.812+0000 7f3074a59640 1 --2- 172.21.15.22:0/526545962 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3050067fc0 0x7f305006a480 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3060000f70 tx=0x7f3060007c90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:18.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.816+0000 7f306d7fa640 1 -- 172.21.15.22:0/526545962 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f3068056320 con 0x7f307015e4c0 2024-03-29T04:18:18.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.980+0000 7f3076cbd640 1 -- 172.21.15.22:0/526545962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f30700b7bc0 con 0x7f307015e4c0 2024-03-29T04:18:18.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.980+0000 7f306d7fa640 1 -- 172.21.15.22:0/526545962 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f306805a620 con 0x7f307015e4c0 2024-03-29T04:18:18.962 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-29T04:18:18.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.984+0000 7f3076cbd640 1 -- 172.21.15.22:0/526545962 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3050067fc0 msgr2=0x7f305006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.984+0000 7f3076cbd640 1 --2- 172.21.15.22:0/526545962 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3050067fc0 0x7f305006a480 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3060000f70 tx=0x7f3060007c90 comp rx=0 tx=0).stop 2024-03-29T04:18:18.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.984+0000 7f3076cbd640 1 -- 172.21.15.22:0/526545962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f307015e4c0 msgr2=0x7f3070160750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:18.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.984+0000 7f3076cbd640 1 --2- 172.21.15.22:0/526545962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f307015e4c0 0x7f3070160750 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f306800cbc0 tx=0x7f306800a430 comp rx=0 tx=0).stop 2024-03-29T04:18:18.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.984+0000 7f3076cbd640 1 -- 172.21.15.22:0/526545962 shutdown_connections 2024-03-29T04:18:18.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.984+0000 7f3076cbd640 1 --2- 172.21.15.22:0/526545962 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3050067fc0 0x7f305006a480 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.984+0000 7f3076cbd640 1 --2- 172.21.15.22:0/526545962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f307015e4c0 0x7f3070160750 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.984+0000 7f3076cbd640 1 --2- 172.21.15.22:0/526545962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30700cf4f0 0x7f307014bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.984+0000 7f3076cbd640 1 --2- 172.21.15.22:0/526545962 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30700b9e30 0x7f30700cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:18.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.984+0000 7f3076cbd640 1 -- 172.21.15.22:0/526545962 >> 172.21.15.22:0/526545962 conn(0x7f30700b6410 msgr2=0x7f30700b7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:18.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.984+0000 7f3076cbd640 1 -- 172.21.15.22:0/526545962 shutdown_connections 2024-03-29T04:18:18.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:18.984+0000 7f3076cbd640 1 -- 172.21.15.22:0/526545962 wait complete. 2024-03-29T04:18:18.982 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:18:19.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87c69a0640 1 Processor -- start 2024-03-29T04:18:19.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87c69a0640 1 -- start start 2024-03-29T04:18:19.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87c69a0640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f87c0144930 0x7f87c00b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87c69a0640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f87c006cd80 0x7f87c00ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87c69a0640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87c00b33c0 con 0x7f87c00b4870 2024-03-29T04:18:19.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87c69a0640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f87c00cdb90 con 0x7f87c0144930 2024-03-29T04:18:19.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87c69a0640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f87c00b3660 con 0x7f87c006cd80 2024-03-29T04:18:19.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87bf7fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f87c00b4870 0x7f87c006c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57958/0 (socket says 172.21.15.22:57958) 2024-03-29T04:18:19.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87bf7fe640 1 -- 172.21.15.22:0/3330587958 learned_addr learned my addr 172.21.15.22:0/3330587958 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:19.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87bffff640 1 --2- 172.21.15.22:0/3330587958 >> v2:172.21.15.151:3300/0 conn(0x7f87c0144930 0x7f87c00b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87c4f3d640 1 --2- 172.21.15.22:0/3330587958 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f87c006cd80 0x7f87c00ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87beffd640 1 -- 172.21.15.22:0/3330587958 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2614363447 0 0) 0x7f87c00b33c0 con 0x7f87c00b4870 2024-03-29T04:18:19.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87beffd640 1 -- 172.21.15.22:0/3330587958 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87a8005740 con 0x7f87c00b4870 2024-03-29T04:18:19.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87beffd640 1 -- 172.21.15.22:0/3330587958 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1837062049 0 0) 0x7f87a8005740 con 0x7f87c00b4870 2024-03-29T04:18:19.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87beffd640 1 -- 172.21.15.22:0/3330587958 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f87c006cd80 msgr2=0x7f87c00ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:19.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87beffd640 1 --2- 172.21.15.22:0/3330587958 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f87c006cd80 0x7f87c00ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87beffd640 1 -- 172.21.15.22:0/3330587958 >> v2:172.21.15.151:3300/0 conn(0x7f87c0144930 msgr2=0x7f87c00b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:19.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87beffd640 1 --2- 172.21.15.22:0/3330587958 >> v2:172.21.15.151:3300/0 conn(0x7f87c0144930 0x7f87c00b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87beffd640 1 -- 172.21.15.22:0/3330587958 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c00b3940 con 0x7f87c00b4870 2024-03-29T04:18:19.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87beffd640 1 -- 172.21.15.22:0/3330587958 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f87b40016a0 con 0x7f87c00b4870 2024-03-29T04:18:19.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87beffd640 1 -- 172.21.15.22:0/3330587958 >> v1:172.21.15.22:6789/0 conn(0x7f87c00b4870 legacy=0x7f87c006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:19.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87beffd640 1 --2- 172.21.15.22:0/3330587958 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f87c006cd80 0x7f87a8005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87beffd640 1 --2- 172.21.15.22:0/3330587958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87a8006150 0x7f87a8006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87beffd640 1 --2- 172.21.15.22:0/3330587958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87a8006f60 0x7f87a80073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87beffd640 1 -- 172.21.15.22:0/3330587958 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f87c00cdb90 con 0x7f87a8006150 2024-03-29T04:18:19.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87beffd640 1 -- 172.21.15.22:0/3330587958 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f87c00b3660 con 0x7f87a8006f60 2024-03-29T04:18:19.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87beffd640 1 -- 172.21.15.22:0/3330587958 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f87a800dc30 con 0x7f87c006cd80 2024-03-29T04:18:19.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87bf7fe640 1 --2- 172.21.15.22:0/3330587958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87a8006150 0x7f87a8006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87bffff640 1 --2- 172.21.15.22:0/3330587958 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f87c006cd80 0x7f87a8005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87c4f3d640 1 --2- 172.21.15.22:0/3330587958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87a8006f60 0x7f87a80073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87bf7fe640 1 -- 172.21.15.22:0/3330587958 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f87c006cd80 msgr2=0x7f87a8005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:19.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87bf7fe640 1 --2- 172.21.15.22:0/3330587958 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f87c006cd80 0x7f87a8005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87bf7fe640 1 -- 172.21.15.22:0/3330587958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87a8006f60 msgr2=0x7f87a80073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:19.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87bf7fe640 1 --2- 172.21.15.22:0/3330587958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87a8006f60 0x7f87a80073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87bf7fe640 1 -- 172.21.15.22:0/3330587958 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a8005740 con 0x7f87a8006150 2024-03-29T04:18:19.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.124+0000 7f87bffff640 1 --2- 172.21.15.22:0/3330587958 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f87c006cd80 0x7f87a8005bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:19.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.128+0000 7f87bf7fe640 1 --2- 172.21.15.22:0/3330587958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87a8006150 0x7f87a8006a20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f87b400c240 tx=0x7f87b401cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4f5814e902e5d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:19.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.128+0000 7f87beffd640 1 -- 172.21.15.22:0/3330587958 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b4007d70 con 0x7f87a8006150 2024-03-29T04:18:19.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.128+0000 7f87beffd640 1 -- 172.21.15.22:0/3330587958 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87b40a9020 con 0x7f87a8006150 2024-03-29T04:18:19.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.128+0000 7f87beffd640 1 -- 172.21.15.22:0/3330587958 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b4005ae0 con 0x7f87a8006150 2024-03-29T04:18:19.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.128+0000 7f87c69a0640 1 -- 172.21.15.22:0/3330587958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87a8006150 msgr2=0x7f87a8006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:19.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.128+0000 7f87c69a0640 1 --2- 172.21.15.22:0/3330587958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87a8006150 0x7f87a8006a20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f87b400c240 tx=0x7f87b401cc50 comp rx=0 tx=0).stop 2024-03-29T04:18:19.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.128+0000 7f87c4f3d640 1 -- 172.21.15.22:0/3330587958 reap_dead start 2024-03-29T04:18:19.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.128+0000 7f87c69a0640 1 -- 172.21.15.22:0/3330587958 shutdown_connections 2024-03-29T04:18:19.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.128+0000 7f87c69a0640 1 -- 172.21.15.22:0/3330587958 >> 172.21.15.22:0/3330587958 conn(0x7f87c00d00a0 msgr2=0x7f87c00d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:19.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.128+0000 7f87c69a0640 1 -- 172.21.15.22:0/3330587958 shutdown_connections 2024-03-29T04:18:19.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.128+0000 7f87c69a0640 1 -- 172.21.15.22:0/3330587958 wait complete. 2024-03-29T04:18:19.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.128+0000 7f87c69a0640 1 Processor -- start 2024-03-29T04:18:19.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.128+0000 7f87c69a0640 1 -- start start 2024-03-29T04:18:19.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87c69a0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87a8006150 0x7f87c013d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87c69a0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87c0135db0 0x7f87c00cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87c69a0640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f87c013cef0 0x7f87c01f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87c69a0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f87c006e000 con 0x7f87c0135db0 2024-03-29T04:18:19.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87c69a0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f87c006de80 con 0x7f87a8006150 2024-03-29T04:18:19.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87c69a0640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f87c006e180 con 0x7f87c013cef0 2024-03-29T04:18:19.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87bf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87c0135db0 0x7f87c00cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87bf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87c0135db0 0x7f87c00cf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59386/0 (socket says 172.21.15.22:59386) 2024-03-29T04:18:19.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87bf7fe640 1 -- 172.21.15.22:0/3896073366 learned_addr learned my addr 172.21.15.22:0/3896073366 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:19.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87bffff640 1 --2- 172.21.15.22:0/3896073366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87a8006150 0x7f87c013d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87c4f3d640 1 --2- 172.21.15.22:0/3896073366 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f87c013cef0 0x7f87c01f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87bf7fe640 1 -- 172.21.15.22:0/3896073366 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f87c013cef0 msgr2=0x7f87c01f65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:19.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87bf7fe640 1 --2- 172.21.15.22:0/3896073366 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f87c013cef0 0x7f87c01f65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87bf7fe640 1 -- 172.21.15.22:0/3896073366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87a8006150 msgr2=0x7f87c013d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:19.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87bf7fe640 1 --2- 172.21.15.22:0/3896073366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87a8006150 0x7f87c013d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87bf7fe640 1 -- 172.21.15.22:0/3896073366 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c015ad20 con 0x7f87c0135db0 2024-03-29T04:18:19.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87bf7fe640 1 --2- 172.21.15.22:0/3896073366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87c0135db0 0x7f87c00cf1d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f87b4007040 tx=0x7f87a8007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:19.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87bcff9640 1 -- 172.21.15.22:0/3896073366 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b40a5ab0 con 0x7f87c0135db0 2024-03-29T04:18:19.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87bcff9640 1 -- 172.21.15.22:0/3896073366 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87b40a5c50 con 0x7f87c0135db0 2024-03-29T04:18:19.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87c69a0640 1 -- 172.21.15.22:0/3896073366 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c013d490 con 0x7f87c0135db0 2024-03-29T04:18:19.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87bcff9640 1 -- 172.21.15.22:0/3896073366 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b40047f0 con 0x7f87c0135db0 2024-03-29T04:18:19.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87c69a0640 1 -- 172.21.15.22:0/3896073366 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c01352d0 con 0x7f87c0135db0 2024-03-29T04:18:19.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87c69a0640 1 -- 172.21.15.22:0/3896073366 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87c0068900 con 0x7f87c0135db0 2024-03-29T04:18:19.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87bcff9640 1 -- 172.21.15.22:0/3896073366 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f87b4004990 con 0x7f87c0135db0 2024-03-29T04:18:19.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87bcff9640 1 --2- 172.21.15.22:0/3896073366 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f87ac067fc0 0x7f87ac06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87bcff9640 1 -- 172.21.15.22:0/3896073366 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f87b4120160 con 0x7f87c0135db0 2024-03-29T04:18:19.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.132+0000 7f87bffff640 1 --2- 172.21.15.22:0/3896073366 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f87ac067fc0 0x7f87ac06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.136+0000 7f87bffff640 1 --2- 172.21.15.22:0/3896073366 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f87ac067fc0 0x7f87ac06a480 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f87b0006f90 tx=0x7f87b0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:19.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.140+0000 7f87bcff9640 1 -- 172.21.15.22:0/3896073366 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f87b4016990 con 0x7f87c0135db0 2024-03-29T04:18:19.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.292+0000 7f87c69a0640 1 -- 172.21.15.22:0/3896073366 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f87c006b700 con 0x7f87c0135db0 2024-03-29T04:18:19.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.292+0000 7f87bcff9640 1 -- 172.21.15.22:0/3896073366 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f87b40f0470 con 0x7f87c0135db0 2024-03-29T04:18:19.274 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:19.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.296+0000 7f87c69a0640 1 -- 172.21.15.22:0/3896073366 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f87ac067fc0 msgr2=0x7f87ac06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:19.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.296+0000 7f87c69a0640 1 --2- 172.21.15.22:0/3896073366 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f87ac067fc0 0x7f87ac06a480 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f87b0006f90 tx=0x7f87b0008040 comp rx=0 tx=0).stop 2024-03-29T04:18:19.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.296+0000 7f87c69a0640 1 -- 172.21.15.22:0/3896073366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87c0135db0 msgr2=0x7f87c00cf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:19.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.296+0000 7f87c69a0640 1 --2- 172.21.15.22:0/3896073366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87c0135db0 0x7f87c00cf1d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f87b4007040 tx=0x7f87a8007a50 comp rx=0 tx=0).stop 2024-03-29T04:18:19.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.296+0000 7f87c69a0640 1 -- 172.21.15.22:0/3896073366 shutdown_connections 2024-03-29T04:18:19.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.296+0000 7f87c69a0640 1 --2- 172.21.15.22:0/3896073366 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f87c013cef0 0x7f87c01f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.296+0000 7f87c69a0640 1 --2- 172.21.15.22:0/3896073366 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f87ac067fc0 0x7f87ac06a480 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.296+0000 7f87c69a0640 1 --2- 172.21.15.22:0/3896073366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87c0135db0 0x7f87c00cf1d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.296+0000 7f87c69a0640 1 --2- 172.21.15.22:0/3896073366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87a8006150 0x7f87c013d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.296+0000 7f87c69a0640 1 -- 172.21.15.22:0/3896073366 >> 172.21.15.22:0/3896073366 conn(0x7f87c00d00a0 msgr2=0x7f87c0163120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:19.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.296+0000 7f87c69a0640 1 -- 172.21.15.22:0/3896073366 shutdown_connections 2024-03-29T04:18:19.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.296+0000 7f87c69a0640 1 -- 172.21.15.22:0/3896073366 wait complete. 2024-03-29T04:18:19.293 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:19.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.432+0000 7f129dc07640 1 Processor -- start 2024-03-29T04:18:19.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.432+0000 7f129dc07640 1 -- start start 2024-03-29T04:18:19.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f129dc07640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f12980d1810 0x7f12980b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f129dc07640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12980d3e60 con 0x7f129815e600 2024-03-29T04:18:19.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f129dc07640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12980b7680 con 0x7f12980b9170 2024-03-29T04:18:19.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f129dc07640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f12980b7450 con 0x7f12980d1810 2024-03-29T04:18:19.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f12977fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f129815e600 0x7f12980d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57960/0 (socket says 172.21.15.22:57960) 2024-03-29T04:18:19.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f12977fe640 1 -- 172.21.15.22:0/903244445 learned_addr learned my addr 172.21.15.22:0/903244445 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:19.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f1296ffd640 1 --2- 172.21.15.22:0/903244445 >> v2:172.21.15.151:3300/0 conn(0x7f12980d1810 0x7f12980b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f12967fc640 1 -- 172.21.15.22:0/903244445 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3256677863 0 0) 0x7f12980d3e60 con 0x7f129815e600 2024-03-29T04:18:19.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f12967fc640 1 -- 172.21.15.22:0/903244445 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1280003530 con 0x7f129815e600 2024-03-29T04:18:19.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f12967fc640 1 -- 172.21.15.22:0/903244445 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3749510579 0 0) 0x7f1280003530 con 0x7f129815e600 2024-03-29T04:18:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f12967fc640 1 -- 172.21.15.22:0/903244445 >> v2:172.21.15.151:3300/0 conn(0x7f12980d1810 msgr2=0x7f12980b9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f12967fc640 1 --2- 172.21.15.22:0/903244445 >> v2:172.21.15.151:3300/0 conn(0x7f12980d1810 0x7f12980b9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f12967fc640 1 -- 172.21.15.22:0/903244445 >> v1:172.21.15.151:6789/0 conn(0x7f12980b9170 legacy=0x7f12980d2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f12967fc640 1 -- 172.21.15.22:0/903244445 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12980b7960 con 0x7f129815e600 2024-03-29T04:18:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f12967fc640 1 -- 172.21.15.22:0/903244445 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f1288001680 con 0x7f129815e600 2024-03-29T04:18:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f12967fc640 1 -- 172.21.15.22:0/903244445 >> v1:172.21.15.22:6789/0 conn(0x7f129815e600 legacy=0x7f12980d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f12967fc640 1 --2- 172.21.15.22:0/903244445 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12800039c0 0x7f1280003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f12967fc640 1 --2- 172.21.15.22:0/903244445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12800043c0 0x7f1280004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f12967fc640 1 --2- 172.21.15.22:0/903244445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1280005130 0x7f128000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f12967fc640 1 -- 172.21.15.22:0/903244445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f128000c040 con 0x7f1280005130 2024-03-29T04:18:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f12967fc640 1 -- 172.21.15.22:0/903244445 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f128000c1f0 con 0x7f12800043c0 2024-03-29T04:18:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f12967fc640 1 -- 172.21.15.22:0/903244445 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f128000c3f0 con 0x7f12800039c0 2024-03-29T04:18:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f1297fff640 1 --2- 172.21.15.22:0/903244445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1280005130 0x7f128000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f1296ffd640 1 --2- 172.21.15.22:0/903244445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12800043c0 0x7f1280004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f12977fe640 1 --2- 172.21.15.22:0/903244445 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12800039c0 0x7f1280003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f1297fff640 1 -- 172.21.15.22:0/903244445 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12800039c0 msgr2=0x7f1280003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:19.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f1297fff640 1 --2- 172.21.15.22:0/903244445 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12800039c0 0x7f1280003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f1297fff640 1 -- 172.21.15.22:0/903244445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12800043c0 msgr2=0x7f1280004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:19.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f1297fff640 1 --2- 172.21.15.22:0/903244445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12800043c0 0x7f1280004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.436+0000 7f1297fff640 1 -- 172.21.15.22:0/903244445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1280003530 con 0x7f1280005130 2024-03-29T04:18:19.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f1297fff640 1 -- 172.21.15.22:0/903244445 reap_dead start 2024-03-29T04:18:19.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f1297fff640 1 --2- 172.21.15.22:0/903244445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1280005130 0x7f128000b9d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1290001f60 tx=0x7f129001eff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a452faaff78640 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:19.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f12967fc640 1 -- 172.21.15.22:0/903244445 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290001b60 con 0x7f1280005130 2024-03-29T04:18:19.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f12967fc640 1 -- 172.21.15.22:0/903244445 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12900a8c10 con 0x7f1280005130 2024-03-29T04:18:19.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f12967fc640 1 -- 172.21.15.22:0/903244445 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290003980 con 0x7f1280005130 2024-03-29T04:18:19.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f129dc07640 1 -- 172.21.15.22:0/903244445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1280005130 msgr2=0x7f128000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:19.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f129dc07640 1 --2- 172.21.15.22:0/903244445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1280005130 0x7f128000b9d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1290001f60 tx=0x7f129001eff0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f129dc07640 1 -- 172.21.15.22:0/903244445 shutdown_connections 2024-03-29T04:18:19.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f129dc07640 1 --2- 172.21.15.22:0/903244445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1280005130 0x7f128000b9d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f129dc07640 1 -- 172.21.15.22:0/903244445 >> 172.21.15.22:0/903244445 conn(0x7f12980b6230 msgr2=0x7f12980b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:19.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f129dc07640 1 -- 172.21.15.22:0/903244445 shutdown_connections 2024-03-29T04:18:19.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f129dc07640 1 -- 172.21.15.22:0/903244445 wait complete. 2024-03-29T04:18:19.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f129dc07640 1 Processor -- start 2024-03-29T04:18:19.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f129dc07640 1 -- start start 2024-03-29T04:18:19.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f129dc07640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1280005130 0x7f129814a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f129dc07640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12981f3e60 0x7f12980cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f129dc07640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12980cfc30 0x7f129815eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f129dc07640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f12980b9f00 con 0x7f1280005130 2024-03-29T04:18:19.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f129dc07640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f12980d4140 con 0x7f12980cfc30 2024-03-29T04:18:19.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f129dc07640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f12980ba080 con 0x7f12981f3e60 2024-03-29T04:18:19.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f12977fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1280005130 0x7f129814a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f1297fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12980cfc30 0x7f129815eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f1297fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12980cfc30 0x7f129815eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:50390/0 (socket says 172.21.15.22:50390) 2024-03-29T04:18:19.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f1297fff640 1 -- 172.21.15.22:0/2950402720 learned_addr learned my addr 172.21.15.22:0/2950402720 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:19.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f12977fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1280005130 0x7f129814a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59400/0 (socket says 172.21.15.22:59400) 2024-03-29T04:18:19.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.440+0000 7f1296ffd640 1 --2- 172.21.15.22:0/2950402720 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12981f3e60 0x7f12980cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.444+0000 7f12977fe640 1 -- 172.21.15.22:0/2950402720 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12981f3e60 msgr2=0x7f12980cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:19.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.444+0000 7f12977fe640 1 --2- 172.21.15.22:0/2950402720 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12981f3e60 0x7f12980cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.444+0000 7f12977fe640 1 -- 172.21.15.22:0/2950402720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12980cfc30 msgr2=0x7f129815eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:19.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.444+0000 7f12977fe640 1 --2- 172.21.15.22:0/2950402720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12980cfc30 0x7f129815eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.444+0000 7f12977fe640 1 -- 172.21.15.22:0/2950402720 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129815f810 con 0x7f1280005130 2024-03-29T04:18:19.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.444+0000 7f12977fe640 1 --2- 172.21.15.22:0/2950402720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1280005130 0x7f129814a660 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f128800e5b0 tx=0x7f128800e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:19.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.444+0000 7f127ffff640 1 -- 172.21.15.22:0/2950402720 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288012590 con 0x7f1280005130 2024-03-29T04:18:19.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.444+0000 7f127ffff640 1 -- 172.21.15.22:0/2950402720 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1288018070 con 0x7f1280005130 2024-03-29T04:18:19.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.444+0000 7f129dc07640 1 -- 172.21.15.22:0/2950402720 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129815ff90 con 0x7f1280005130 2024-03-29T04:18:19.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.444+0000 7f127ffff640 1 -- 172.21.15.22:0/2950402720 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288012970 con 0x7f1280005130 2024-03-29T04:18:19.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.444+0000 7f129dc07640 1 -- 172.21.15.22:0/2950402720 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1298160550 con 0x7f1280005130 2024-03-29T04:18:19.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.444+0000 7f129dc07640 1 -- 172.21.15.22:0/2950402720 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12980d0040 con 0x7f1280005130 2024-03-29T04:18:19.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.444+0000 7f127ffff640 1 -- 172.21.15.22:0/2950402720 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1288012b10 con 0x7f1280005130 2024-03-29T04:18:19.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.444+0000 7f127ffff640 1 --2- 172.21.15.22:0/2950402720 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1284067ff0 0x7f128406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.444+0000 7f127ffff640 1 -- 172.21.15.22:0/2950402720 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f1288086320 con 0x7f1280005130 2024-03-29T04:18:19.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.444+0000 7f1296ffd640 1 --2- 172.21.15.22:0/2950402720 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1284067ff0 0x7f128406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.444+0000 7f1296ffd640 1 --2- 172.21.15.22:0/2950402720 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1284067ff0 0x7f128406a4b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f12980cbb40 tx=0x7f128c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:19.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.448+0000 7f127ffff640 1 -- 172.21.15.22:0/2950402720 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f12980d0040 con 0x7f1280005130 2024-03-29T04:18:19.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.612+0000 7f129dc07640 1 -- 172.21.15.22:0/2950402720 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1298068890 con 0x7f1280005130 2024-03-29T04:18:19.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.616+0000 7f127ffff640 1 -- 172.21.15.22:0/2950402720 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18847 (secure 0 0 0) 0x7f1288056760 con 0x7f1280005130 2024-03-29T04:18:19.598 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:19.598 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:18:16.893439+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:19.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.620+0000 7f129dc07640 1 -- 172.21.15.22:0/2950402720 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1284067ff0 msgr2=0x7f128406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:19.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.620+0000 7f129dc07640 1 --2- 172.21.15.22:0/2950402720 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1284067ff0 0x7f128406a4b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f12980cbb40 tx=0x7f128c009190 comp rx=0 tx=0).stop 2024-03-29T04:18:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.620+0000 7f129dc07640 1 -- 172.21.15.22:0/2950402720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1280005130 msgr2=0x7f129814a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.620+0000 7f129dc07640 1 --2- 172.21.15.22:0/2950402720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1280005130 0x7f129814a660 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f128800e5b0 tx=0x7f128800e9b0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.620+0000 7f129dc07640 1 -- 172.21.15.22:0/2950402720 shutdown_connections 2024-03-29T04:18:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.620+0000 7f129dc07640 1 --2- 172.21.15.22:0/2950402720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12980cfc30 0x7f129815eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.620+0000 7f129dc07640 1 --2- 172.21.15.22:0/2950402720 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12981f3e60 0x7f12980cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.620+0000 7f129dc07640 1 --2- 172.21.15.22:0/2950402720 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1284067ff0 0x7f128406a4b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.620+0000 7f129dc07640 1 --2- 172.21.15.22:0/2950402720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1280005130 0x7f129814a660 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.620+0000 7f129dc07640 1 -- 172.21.15.22:0/2950402720 >> 172.21.15.22:0/2950402720 conn(0x7f12980b6230 msgr2=0x7f12980d2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.620+0000 7f129dc07640 1 -- 172.21.15.22:0/2950402720 shutdown_connections 2024-03-29T04:18:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.620+0000 7f129dc07640 1 -- 172.21.15.22:0/2950402720 wait complete. 2024-03-29T04:18:19.618 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:17:34.034142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:17:34.886659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:17:38.628632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:18:19.618 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:18:19.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.760+0000 7f8596f6b640 1 Processor -- start 2024-03-29T04:18:19.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.760+0000 7f8596f6b640 1 -- start start 2024-03-29T04:18:19.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.760+0000 7f8596f6b640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f859015baa0 0x7f85900b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.760+0000 7f8596f6b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85900b3c40 con 0x7f85900cf870 2024-03-29T04:18:19.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.760+0000 7f8596f6b640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8590071910 con 0x7f85900b4090 2024-03-29T04:18:19.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.760+0000 7f8596f6b640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f8590071bf0 con 0x7f859015baa0 2024-03-29T04:18:19.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.760+0000 7f8594d07640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f85900cf870 0x7f85900726d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57970/0 (socket says 172.21.15.22:57970) 2024-03-29T04:18:19.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.760+0000 7f8594d07640 1 -- 172.21.15.22:0/1922433932 learned_addr learned my addr 172.21.15.22:0/1922433932 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:19.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.760+0000 7f858ffff640 1 --2- 172.21.15.22:0/1922433932 >> v2:172.21.15.151:3300/0 conn(0x7f859015baa0 0x7f85900b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858f7fe640 1 -- 172.21.15.22:0/1922433932 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 223210017 0 0) 0x7f85900b3c40 con 0x7f85900cf870 2024-03-29T04:18:19.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858f7fe640 1 -- 172.21.15.22:0/1922433932 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8578003530 con 0x7f85900cf870 2024-03-29T04:18:19.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858f7fe640 1 -- 172.21.15.22:0/1922433932 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3659505333 0 0) 0x7f8590071910 con 0x7f85900b4090 2024-03-29T04:18:19.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858f7fe640 1 -- 172.21.15.22:0/1922433932 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85900b3c40 con 0x7f85900b4090 2024-03-29T04:18:19.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858f7fe640 1 -- 172.21.15.22:0/1922433932 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 470678659 0 0) 0x7f8578003530 con 0x7f85900cf870 2024-03-29T04:18:19.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858f7fe640 1 -- 172.21.15.22:0/1922433932 >> v2:172.21.15.151:3300/0 conn(0x7f859015baa0 msgr2=0x7f85900b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:19.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858f7fe640 1 --2- 172.21.15.22:0/1922433932 >> v2:172.21.15.151:3300/0 conn(0x7f859015baa0 0x7f85900b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858f7fe640 1 -- 172.21.15.22:0/1922433932 >> v1:172.21.15.151:6789/0 conn(0x7f85900b4090 legacy=0x7f85900b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:19.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858f7fe640 1 -- 172.21.15.22:0/1922433932 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8590071ed0 con 0x7f85900cf870 2024-03-29T04:18:19.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858f7fe640 1 -- 172.21.15.22:0/1922433932 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f8580001680 con 0x7f85900cf870 2024-03-29T04:18:19.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858f7fe640 1 -- 172.21.15.22:0/1922433932 >> v1:172.21.15.22:6789/0 conn(0x7f85900cf870 legacy=0x7f85900726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:19.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858f7fe640 1 --2- 172.21.15.22:0/1922433932 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8578003b50 0x7f8578003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858f7fe640 1 --2- 172.21.15.22:0/1922433932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8578004810 0x7f8578004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858f7fe640 1 --2- 172.21.15.22:0/1922433932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85780051d0 0x7f857800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858f7fe640 1 -- 172.21.15.22:0/1922433932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8590071bf0 con 0x7f8578004810 2024-03-29T04:18:19.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858f7fe640 1 -- 172.21.15.22:0/1922433932 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f857800bff0 con 0x7f85780051d0 2024-03-29T04:18:19.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858f7fe640 1 -- 172.21.15.22:0/1922433932 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f857800c1f0 con 0x7f8578003b50 2024-03-29T04:18:19.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858ffff640 1 --2- 172.21.15.22:0/1922433932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8578004810 0x7f8578004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f8594d07640 1 --2- 172.21.15.22:0/1922433932 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8578003b50 0x7f8578003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858ffff640 1 -- 172.21.15.22:0/1922433932 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8578003b50 msgr2=0x7f8578003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:19.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858ffff640 1 --2- 172.21.15.22:0/1922433932 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8578003b50 0x7f8578003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858ffff640 1 -- 172.21.15.22:0/1922433932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85780051d0 msgr2=0x7f857800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:19.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f8595508640 1 --2- 172.21.15.22:0/1922433932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85780051d0 0x7f857800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858ffff640 1 --2- 172.21.15.22:0/1922433932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85780051d0 0x7f857800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858ffff640 1 -- 172.21.15.22:0/1922433932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8590071ed0 con 0x7f8578004810 2024-03-29T04:18:19.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f8595508640 1 --2- 172.21.15.22:0/1922433932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85780051d0 0x7f857800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:19.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f8595508640 1 -- 172.21.15.22:0/1922433932 reap_dead start 2024-03-29T04:18:19.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858ffff640 1 --2- 172.21.15.22:0/1922433932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8578004810 0x7f8578004c90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8584009ae0 tx=0x7f858401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec1f650e51d6a16 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:19.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858f7fe640 1 -- 172.21.15.22:0/1922433932 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85840b2070 con 0x7f8578004810 2024-03-29T04:18:19.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858f7fe640 1 -- 172.21.15.22:0/1922433932 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85840a5550 con 0x7f8578004810 2024-03-29T04:18:19.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.764+0000 7f858f7fe640 1 -- 172.21.15.22:0/1922433932 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85840096b0 con 0x7f8578004810 2024-03-29T04:18:19.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8596f6b640 1 -- 172.21.15.22:0/1922433932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8578004810 msgr2=0x7f8578004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:19.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8596f6b640 1 --2- 172.21.15.22:0/1922433932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8578004810 0x7f8578004c90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8584009ae0 tx=0x7f858401ca70 comp rx=0 tx=0).stop 2024-03-29T04:18:19.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8596f6b640 1 -- 172.21.15.22:0/1922433932 shutdown_connections 2024-03-29T04:18:19.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8596f6b640 1 --2- 172.21.15.22:0/1922433932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8578004810 0x7f8578004c90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8596f6b640 1 -- 172.21.15.22:0/1922433932 >> 172.21.15.22:0/1922433932 conn(0x7f85900d00a0 msgr2=0x7f85900d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:19.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8596f6b640 1 -- 172.21.15.22:0/1922433932 shutdown_connections 2024-03-29T04:18:19.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8596f6b640 1 -- 172.21.15.22:0/1922433932 wait complete. 2024-03-29T04:18:19.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8596f6b640 1 Processor -- start 2024-03-29T04:18:19.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8596f6b640 1 -- start start 2024-03-29T04:18:19.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8596f6b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8578004810 0x7f85901f8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8596f6b640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8590151500 0x7f8590149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8596f6b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8590072520 0x7f85900729a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8596f6b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f85900b3700 con 0x7f8578004810 2024-03-29T04:18:19.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8596f6b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f85900cfc50 con 0x7f8590072520 2024-03-29T04:18:19.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8596f6b640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f85900b3880 con 0x7f8590151500 2024-03-29T04:18:19.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8594d07640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8578004810 0x7f85901f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8594d07640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8578004810 0x7f85901f8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59424/0 (socket says 172.21.15.22:59424) 2024-03-29T04:18:19.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8594d07640 1 -- 172.21.15.22:0/1289161613 learned_addr learned my addr 172.21.15.22:0/1289161613 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:19.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8595508640 1 --2- 172.21.15.22:0/1289161613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8590072520 0x7f85900729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f858ffff640 1 --2- 172.21.15.22:0/1289161613 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8590151500 0x7f8590149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8594d07640 1 -- 172.21.15.22:0/1289161613 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8590151500 msgr2=0x7f8590149e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:19.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8594d07640 1 --2- 172.21.15.22:0/1289161613 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8590151500 0x7f8590149e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8594d07640 1 -- 172.21.15.22:0/1289161613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8590072520 msgr2=0x7f85900729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:19.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8594d07640 1 --2- 172.21.15.22:0/1289161613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8590072520 0x7f85900729a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8594d07640 1 -- 172.21.15.22:0/1289161613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859014df40 con 0x7f8578004810 2024-03-29T04:18:19.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8595508640 1 --2- 172.21.15.22:0/1289161613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8590072520 0x7f85900729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:19.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8594d07640 1 --2- 172.21.15.22:0/1289161613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8578004810 0x7f85901f8340 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f858000e900 tx=0x7f858000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:19.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f858d7fa640 1 -- 172.21.15.22:0/1289161613 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8580019070 con 0x7f8578004810 2024-03-29T04:18:19.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f858d7fa640 1 -- 172.21.15.22:0/1289161613 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f858000c870 con 0x7f8578004810 2024-03-29T04:18:19.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f858d7fa640 1 -- 172.21.15.22:0/1289161613 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858000cc60 con 0x7f8578004810 2024-03-29T04:18:19.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8596f6b640 1 -- 172.21.15.22:0/1289161613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f859014e220 con 0x7f8578004810 2024-03-29T04:18:19.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.768+0000 7f8596f6b640 1 -- 172.21.15.22:0/1289161613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f859014e7e0 con 0x7f8578004810 2024-03-29T04:18:19.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.772+0000 7f8596f6b640 1 -- 172.21.15.22:0/1289161613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8590072ee0 con 0x7f8578004810 2024-03-29T04:18:19.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.772+0000 7f858d7fa640 1 -- 172.21.15.22:0/1289161613 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f858001c070 con 0x7f8578004810 2024-03-29T04:18:19.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.772+0000 7f858d7fa640 1 --2- 172.21.15.22:0/1289161613 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f857c067ff0 0x7f857c06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:19.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.772+0000 7f858d7fa640 1 -- 172.21.15.22:0/1289161613 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f8580086120 con 0x7f8578004810 2024-03-29T04:18:19.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.772+0000 7f858ffff640 1 --2- 172.21.15.22:0/1289161613 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f857c067ff0 0x7f857c06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:19.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.772+0000 7f858ffff640 1 --2- 172.21.15.22:0/1289161613 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f857c067ff0 0x7f857c06a4b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f85840af000 tx=0x7f8584008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:19.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.776+0000 7f858d7fa640 1 -- 172.21.15.22:0/1289161613 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f8580056560 con 0x7f8578004810 2024-03-29T04:18:19.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.936+0000 7f8596f6b640 1 -- 172.21.15.22:0/1289161613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8590068900 con 0x7f8578004810 2024-03-29T04:18:19.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.936+0000 7f858d7fa640 1 -- 172.21.15.22:0/1289161613 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f8580055e60 con 0x7f8578004810 2024-03-29T04:18:19.916 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:18:19.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.936+0000 7f8596f6b640 1 -- 172.21.15.22:0/1289161613 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f857c067ff0 msgr2=0x7f857c06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:19.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.936+0000 7f8596f6b640 1 --2- 172.21.15.22:0/1289161613 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f857c067ff0 0x7f857c06a4b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f85840af000 tx=0x7f8584008040 comp rx=0 tx=0).stop 2024-03-29T04:18:19.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.936+0000 7f8596f6b640 1 -- 172.21.15.22:0/1289161613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8578004810 msgr2=0x7f85901f8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:19.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.936+0000 7f8596f6b640 1 --2- 172.21.15.22:0/1289161613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8578004810 0x7f85901f8340 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f858000e900 tx=0x7f858000edc0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.936+0000 7f8596f6b640 1 -- 172.21.15.22:0/1289161613 shutdown_connections 2024-03-29T04:18:19.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.936+0000 7f8596f6b640 1 --2- 172.21.15.22:0/1289161613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8590072520 0x7f85900729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.936+0000 7f8596f6b640 1 --2- 172.21.15.22:0/1289161613 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8590151500 0x7f8590149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.936+0000 7f8596f6b640 1 --2- 172.21.15.22:0/1289161613 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f857c067ff0 0x7f857c06a4b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.936+0000 7f8596f6b640 1 --2- 172.21.15.22:0/1289161613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8578004810 0x7f85901f8340 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:19.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.936+0000 7f8596f6b640 1 -- 172.21.15.22:0/1289161613 >> 172.21.15.22:0/1289161613 conn(0x7f85900d00a0 msgr2=0x7f85900ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:19.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.936+0000 7f8596f6b640 1 -- 172.21.15.22:0/1289161613 shutdown_connections 2024-03-29T04:18:19.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:19.936+0000 7f8596f6b640 1 -- 172.21.15.22:0/1289161613 wait complete. 2024-03-29T04:18:19.935 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:18:20.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.076+0000 7fb768fc0640 1 Processor -- start 2024-03-29T04:18:20.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.076+0000 7fb768fc0640 1 -- start start 2024-03-29T04:18:20.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.080+0000 7fb768fc0640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb764144930 0x7fb7640b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:20.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.080+0000 7fb768fc0640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fb7640b4870 0x7fb76406c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:20.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.080+0000 7fb768fc0640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb76406cbb0 0x7fb7640713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:20.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.080+0000 7fb768fc0640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fb7640cdb90 con 0x7fb764144930 2024-03-29T04:18:20.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.080+0000 7fb768fc0640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fb7640b33c0 con 0x7fb7640b4870 2024-03-29T04:18:20.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.080+0000 7fb768fc0640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb7640b3660 con 0x7fb76406cbb0 2024-03-29T04:18:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.080+0000 7fb76259c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb764144930 0x7fb7640b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.080+0000 7fb76259c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb764144930 0x7fb7640b2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59428/0 (socket says 172.21.15.22:59428) 2024-03-29T04:18:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.080+0000 7fb76259c640 1 -- 172.21.15.22:0/1837887387 learned_addr learned my addr 172.21.15.22:0/1837887387 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.080+0000 7fb761d9b640 1 --2- 172.21.15.22:0/1837887387 >> v2:172.21.15.151:3300/0 conn(0x7fb7640b4870 0x7fb76406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.080+0000 7fb762d9d640 1 --2- 172.21.15.22:0/1837887387 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb76406cbb0 0x7fb7640713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.080+0000 7fb76259c640 1 -- 172.21.15.22:0/1837887387 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb76406cbb0 msgr2=0x7fb7640713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.080+0000 7fb76259c640 1 --2- 172.21.15.22:0/1837887387 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb76406cbb0 0x7fb7640713f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.080+0000 7fb76259c640 1 -- 172.21.15.22:0/1837887387 >> v2:172.21.15.151:3300/0 conn(0x7fb7640b4870 msgr2=0x7fb76406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.080+0000 7fb76259c640 1 --2- 172.21.15.22:0/1837887387 >> v2:172.21.15.151:3300/0 conn(0x7fb7640b4870 0x7fb76406c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.080+0000 7fb76259c640 1 -- 172.21.15.22:0/1837887387 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7640b3940 con 0x7fb764144930 2024-03-29T04:18:20.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.080+0000 7fb76259c640 1 --2- 172.21.15.22:0/1837887387 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb764144930 0x7fb7640b2e80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb7640b4d70 tx=0x7fb75401c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7aae1bce8eb2d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:20.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.080+0000 7fb76159a640 1 -- 172.21.15.22:0/1837887387 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7540b0070 con 0x7fb764144930 2024-03-29T04:18:20.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.080+0000 7fb76159a640 1 -- 172.21.15.22:0/1837887387 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7540ab600 con 0x7fb764144930 2024-03-29T04:18:20.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.080+0000 7fb76159a640 1 -- 172.21.15.22:0/1837887387 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7540ab9d0 con 0x7fb764144930 2024-03-29T04:18:20.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb768fc0640 1 -- 172.21.15.22:0/1837887387 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb764144930 msgr2=0x7fb7640b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:20.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb768fc0640 1 --2- 172.21.15.22:0/1837887387 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb764144930 0x7fb7640b2e80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb7640b4d70 tx=0x7fb75401c1d0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb768fc0640 1 -- 172.21.15.22:0/1837887387 shutdown_connections 2024-03-29T04:18:20.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb768fc0640 1 --2- 172.21.15.22:0/1837887387 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb76406cbb0 0x7fb7640713f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb768fc0640 1 --2- 172.21.15.22:0/1837887387 >> v2:172.21.15.151:3300/0 conn(0x7fb7640b4870 0x7fb76406c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb768fc0640 1 --2- 172.21.15.22:0/1837887387 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb764144930 0x7fb7640b2e80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb768fc0640 1 -- 172.21.15.22:0/1837887387 >> 172.21.15.22:0/1837887387 conn(0x7fb7640d00a0 msgr2=0x7fb7640d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:20.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb768fc0640 1 -- 172.21.15.22:0/1837887387 shutdown_connections 2024-03-29T04:18:20.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb768fc0640 1 -- 172.21.15.22:0/1837887387 wait complete. 2024-03-29T04:18:20.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb768fc0640 1 Processor -- start 2024-03-29T04:18:20.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb768fc0640 1 -- start start 2024-03-29T04:18:20.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb768fc0640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb76406cbb0 0x7fb7641f8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:20.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb768fc0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7640b4870 0x7fb7641355d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:20.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb768fc0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb764144930 0x7fb76415d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:20.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb768fc0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb7640ce1a0 con 0x7fb7640b4870 2024-03-29T04:18:20.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb768fc0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7640ce020 con 0x7fb764144930 2024-03-29T04:18:20.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb768fc0640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb7640ce320 con 0x7fb76406cbb0 2024-03-29T04:18:20.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb761d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7640b4870 0x7fb7641355d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:20.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb761d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7640b4870 0x7fb7641355d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59444/0 (socket says 172.21.15.22:59444) 2024-03-29T04:18:20.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb761d9b640 1 -- 172.21.15.22:0/1409328023 learned_addr learned my addr 172.21.15.22:0/1409328023 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:20.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb762d9d640 1 --2- 172.21.15.22:0/1409328023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb764144930 0x7fb76415d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:20.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb76259c640 1 --2- 172.21.15.22:0/1409328023 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb76406cbb0 0x7fb7641f8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:20.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb761d9b640 1 -- 172.21.15.22:0/1409328023 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb76406cbb0 msgr2=0x7fb7641f8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:20.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb761d9b640 1 --2- 172.21.15.22:0/1409328023 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb76406cbb0 0x7fb7641f8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb761d9b640 1 -- 172.21.15.22:0/1409328023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb764144930 msgr2=0x7fb76415d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:20.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb761d9b640 1 --2- 172.21.15.22:0/1409328023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb764144930 0x7fb76415d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb761d9b640 1 -- 172.21.15.22:0/1409328023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7641f5300 con 0x7fb7640b4870 2024-03-29T04:18:20.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb76259c640 1 --2- 172.21.15.22:0/1409328023 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb76406cbb0 0x7fb7641f8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:20.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb761d9b640 1 --2- 172.21.15.22:0/1409328023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7640b4870 0x7fb7641355d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00efc0 tx=0x7fb74c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:20.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb75affd640 1 -- 172.21.15.22:0/1409328023 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c018020 con 0x7fb7640b4870 2024-03-29T04:18:20.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb75affd640 1 -- 172.21.15.22:0/1409328023 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb74c014480 con 0x7fb7640b4870 2024-03-29T04:18:20.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb768fc0640 1 -- 172.21.15.22:0/1409328023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7641f99b0 con 0x7fb7640b4870 2024-03-29T04:18:20.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb75affd640 1 -- 172.21.15.22:0/1409328023 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c014850 con 0x7fb7640b4870 2024-03-29T04:18:20.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.084+0000 7fb768fc0640 1 -- 172.21.15.22:0/1409328023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7641f93f0 con 0x7fb7640b4870 2024-03-29T04:18:20.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.088+0000 7fb768fc0640 1 -- 172.21.15.22:0/1409328023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb764068900 con 0x7fb7640b4870 2024-03-29T04:18:20.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.088+0000 7fb75affd640 1 -- 172.21.15.22:0/1409328023 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb74c01b020 con 0x7fb7640b4870 2024-03-29T04:18:20.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.088+0000 7fb75affd640 1 --2- 172.21.15.22:0/1409328023 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb744067fc0 0x7fb74406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:20.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.088+0000 7fb75affd640 1 -- 172.21.15.22:0/1409328023 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fb74c0873e0 con 0x7fb7640b4870 2024-03-29T04:18:20.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.088+0000 7fb76259c640 1 --2- 172.21.15.22:0/1409328023 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb744067fc0 0x7fb74406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:20.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.088+0000 7fb76259c640 1 --2- 172.21.15.22:0/1409328023 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb744067fc0 0x7fb74406a480 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb7540ad000 tx=0x7fb754008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:20.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.092+0000 7fb75affd640 1 -- 172.21.15.22:0/1409328023 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fb74c022070 con 0x7fb7640b4870 2024-03-29T04:18:20.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.256+0000 7fb768fc0640 1 -- 172.21.15.22:0/1409328023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb76413e0e0 con 0x7fb7640b4870 2024-03-29T04:18:20.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.256+0000 7fb75affd640 1 -- 172.21.15.22:0/1409328023 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7fb74c010040 con 0x7fb7640b4870 2024-03-29T04:18:20.237 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:20.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.256+0000 7fb768fc0640 1 -- 172.21.15.22:0/1409328023 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb744067fc0 msgr2=0x7fb74406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:20.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.256+0000 7fb768fc0640 1 --2- 172.21.15.22:0/1409328023 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb744067fc0 0x7fb74406a480 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb7540ad000 tx=0x7fb754008040 comp rx=0 tx=0).stop 2024-03-29T04:18:20.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.256+0000 7fb768fc0640 1 -- 172.21.15.22:0/1409328023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7640b4870 msgr2=0x7fb7641355d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:20.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.256+0000 7fb768fc0640 1 --2- 172.21.15.22:0/1409328023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7640b4870 0x7fb7641355d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00efc0 tx=0x7fb74c00c490 comp rx=0 tx=0).stop 2024-03-29T04:18:20.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.256+0000 7fb768fc0640 1 -- 172.21.15.22:0/1409328023 shutdown_connections 2024-03-29T04:18:20.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.256+0000 7fb768fc0640 1 --2- 172.21.15.22:0/1409328023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb764144930 0x7fb76415d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.256+0000 7fb768fc0640 1 --2- 172.21.15.22:0/1409328023 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb744067fc0 0x7fb74406a480 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.256+0000 7fb768fc0640 1 --2- 172.21.15.22:0/1409328023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7640b4870 0x7fb7641355d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.256+0000 7fb768fc0640 1 --2- 172.21.15.22:0/1409328023 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb76406cbb0 0x7fb7641f8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.260+0000 7fb768fc0640 1 -- 172.21.15.22:0/1409328023 >> 172.21.15.22:0/1409328023 conn(0x7fb7640d00a0 msgr2=0x7fb76415be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:20.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.260+0000 7fb768fc0640 1 -- 172.21.15.22:0/1409328023 shutdown_connections 2024-03-29T04:18:20.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.260+0000 7fb768fc0640 1 -- 172.21.15.22:0/1409328023 wait complete. 2024-03-29T04:18:20.258 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:18:20.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.412+0000 7f8e0fb06640 1 Processor -- start 2024-03-29T04:18:20.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.412+0000 7f8e0fb06640 1 -- start start 2024-03-29T04:18:20.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.412+0000 7f8e0fb06640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e081459b0 0x7f8e080b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:20.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.412+0000 7f8e0fb06640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8e08160320 0x7f8e080cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:20.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.412+0000 7f8e0fb06640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e0806e040 con 0x7f8e0815fec0 2024-03-29T04:18:20.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.412+0000 7f8e0fb06640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f8e080cd1f0 con 0x7f8e08160320 2024-03-29T04:18:20.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.412+0000 7f8e0fb06640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8e0806e990 con 0x7f8e081459b0 2024-03-29T04:18:20.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.412+0000 7f8e0e0a3640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8e08160320 0x7f8e080cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:20.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.412+0000 7f8e0e0a3640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8e08160320 0x7f8e080cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59456/0 (socket says 172.21.15.22:59456) 2024-03-29T04:18:20.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.412+0000 7f8e0e0a3640 1 -- 172.21.15.22:0/3320489571 learned_addr learned my addr 172.21.15.22:0/3320489571 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:20.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.412+0000 7f8e0d8a2640 1 --2- 172.21.15.22:0/3320489571 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e081459b0 0x7f8e080b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:20.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.412+0000 7f8e0e0a3640 1 -- 172.21.15.22:0/3320489571 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e081459b0 msgr2=0x7f8e080b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:20.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.412+0000 7f8e0e0a3640 1 --2- 172.21.15.22:0/3320489571 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e081459b0 0x7f8e080b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.412+0000 7f8e0e0a3640 1 -- 172.21.15.22:0/3320489571 >> v1:172.21.15.151:6789/0 conn(0x7f8e0815fec0 legacy=0x7f8e0806f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:20.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.412+0000 7f8e0c8a0640 1 -- 172.21.15.22:0/3320489571 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2914194836 0 0) 0x7f8e0806e040 con 0x7f8e0815fec0 2024-03-29T04:18:20.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.412+0000 7f8e0e0a3640 1 -- 172.21.15.22:0/3320489571 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0806ec70 con 0x7f8e08160320 2024-03-29T04:18:20.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0e0a3640 1 --2- 172.21.15.22:0/3320489571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e08160320 0x7f8e080cccb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009e50 tx=0x7f8e0001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2084291b073d5775 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:20.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0c8a0640 1 -- 172.21.15.22:0/3320489571 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e000a57f0 con 0x7f8e08160320 2024-03-29T04:18:20.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0c8a0640 1 -- 172.21.15.22:0/3320489571 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e000a5990 con 0x7f8e08160320 2024-03-29T04:18:20.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0c8a0640 1 -- 172.21.15.22:0/3320489571 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e000a5d60 con 0x7f8e08160320 2024-03-29T04:18:20.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0fb06640 1 -- 172.21.15.22:0/3320489571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e08160320 msgr2=0x7f8e080cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:20.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0fb06640 1 --2- 172.21.15.22:0/3320489571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e08160320 0x7f8e080cccb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009e50 tx=0x7f8e0001ca30 comp rx=0 tx=0).stop 2024-03-29T04:18:20.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0fb06640 1 -- 172.21.15.22:0/3320489571 shutdown_connections 2024-03-29T04:18:20.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0fb06640 1 --2- 172.21.15.22:0/3320489571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e08160320 0x7f8e080cccb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0fb06640 1 --2- 172.21.15.22:0/3320489571 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e081459b0 0x7f8e080b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0fb06640 1 -- 172.21.15.22:0/3320489571 >> 172.21.15.22:0/3320489571 conn(0x7f8e080b2c70 msgr2=0x7f8e080b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:20.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0fb06640 1 -- 172.21.15.22:0/3320489571 shutdown_connections 2024-03-29T04:18:20.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0fb06640 1 -- 172.21.15.22:0/3320489571 wait complete. 2024-03-29T04:18:20.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0fb06640 1 Processor -- start 2024-03-29T04:18:20.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0fb06640 1 -- start start 2024-03-29T04:18:20.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0fb06640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e081459b0 0x7f8e081fb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0fb06640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e0815fec0 0x7f8e0815bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0fb06640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e08160320 0x7f8e080ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0fb06640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8e08160d00 con 0x7f8e081459b0 2024-03-29T04:18:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0fb06640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e08160b80 con 0x7f8e0815fec0 2024-03-29T04:18:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0fb06640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8e08145d90 con 0x7f8e08160320 2024-03-29T04:18:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0d8a2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e081459b0 0x7f8e081fb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0d8a2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e081459b0 0x7f8e081fb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59466/0 (socket says 172.21.15.22:59466) 2024-03-29T04:18:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.416+0000 7f8e0d8a2640 1 -- 172.21.15.22:0/3449031332 learned_addr learned my addr 172.21.15.22:0/3449031332 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.420+0000 7f8e0d0a1640 1 --2- 172.21.15.22:0/3449031332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e0815fec0 0x7f8e0815bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.420+0000 7f8e0e0a3640 1 --2- 172.21.15.22:0/3449031332 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e08160320 0x7f8e080ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.420+0000 7f8e0d8a2640 1 -- 172.21.15.22:0/3449031332 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e08160320 msgr2=0x7f8e080ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.420+0000 7f8e0d8a2640 1 --2- 172.21.15.22:0/3449031332 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e08160320 0x7f8e080ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.420+0000 7f8e0d8a2640 1 -- 172.21.15.22:0/3449031332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e0815fec0 msgr2=0x7f8e0815bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.420+0000 7f8e0d8a2640 1 --2- 172.21.15.22:0/3449031332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e0815fec0 0x7f8e0815bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.420+0000 7f8e0d8a2640 1 -- 172.21.15.22:0/3449031332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e081fb810 con 0x7f8e081459b0 2024-03-29T04:18:20.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.420+0000 7f8e0d8a2640 1 --2- 172.21.15.22:0/3449031332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e081459b0 0x7f8e081fb2d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400e930 tx=0x7f8e0400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:20.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.420+0000 7f8dfe7fc640 1 -- 172.21.15.22:0/3449031332 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04017070 con 0x7f8e081459b0 2024-03-29T04:18:20.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.420+0000 7f8dfe7fc640 1 -- 172.21.15.22:0/3449031332 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e0400ca00 con 0x7f8e081459b0 2024-03-29T04:18:20.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.420+0000 7f8dfe7fc640 1 -- 172.21.15.22:0/3449031332 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0400cdb0 con 0x7f8e081459b0 2024-03-29T04:18:20.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.420+0000 7f8e0fb06640 1 -- 172.21.15.22:0/3449031332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e080cf4c0 con 0x7f8e081459b0 2024-03-29T04:18:20.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.420+0000 7f8e0fb06640 1 -- 172.21.15.22:0/3449031332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e080cfa80 con 0x7f8e081459b0 2024-03-29T04:18:20.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.420+0000 7f8e0fb06640 1 -- 172.21.15.22:0/3449031332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e08068900 con 0x7f8e081459b0 2024-03-29T04:18:20.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.420+0000 7f8dfe7fc640 1 -- 172.21.15.22:0/3449031332 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f8e0401a070 con 0x7f8e081459b0 2024-03-29T04:18:20.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.420+0000 7f8dfe7fc640 1 --2- 172.21.15.22:0/3449031332 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8dec067fc0 0x7f8dec06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:20.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.420+0000 7f8dfe7fc640 1 -- 172.21.15.22:0/3449031332 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f8e04086be0 con 0x7f8e081459b0 2024-03-29T04:18:20.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.420+0000 7f8e0d0a1640 1 --2- 172.21.15.22:0/3449031332 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8dec067fc0 0x7f8dec06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:20.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.424+0000 7f8e0d0a1640 1 --2- 172.21.15.22:0/3449031332 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8dec067fc0 0x7f8dec06a480 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8e08135490 tx=0x7f8df8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:20.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.424+0000 7f8dfe7fc640 1 -- 172.21.15.22:0/3449031332 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f8e081fb810 con 0x7f8e081459b0 2024-03-29T04:18:20.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.584+0000 7f8e0fb06640 1 -- 172.21.15.22:0/3449031332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8e0815fc30 con 0x7f8e081459b0 2024-03-29T04:18:20.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.588+0000 7f8dfe7fc640 1 -- 172.21.15.22:0/3449031332 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f8e04056a90 con 0x7f8e081459b0 2024-03-29T04:18:20.568 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-29T04:18:20.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.588+0000 7f8e0fb06640 1 -- 172.21.15.22:0/3449031332 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8dec067fc0 msgr2=0x7f8dec06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:20.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.588+0000 7f8e0fb06640 1 --2- 172.21.15.22:0/3449031332 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8dec067fc0 0x7f8dec06a480 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8e08135490 tx=0x7f8df8009040 comp rx=0 tx=0).stop 2024-03-29T04:18:20.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.588+0000 7f8e0fb06640 1 -- 172.21.15.22:0/3449031332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e081459b0 msgr2=0x7f8e081fb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:20.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.588+0000 7f8e0fb06640 1 --2- 172.21.15.22:0/3449031332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e081459b0 0x7f8e081fb2d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400e930 tx=0x7f8e0400edf0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.588+0000 7f8e0fb06640 1 -- 172.21.15.22:0/3449031332 shutdown_connections 2024-03-29T04:18:20.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.588+0000 7f8e0fb06640 1 --2- 172.21.15.22:0/3449031332 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e08160320 0x7f8e080ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.588+0000 7f8e0fb06640 1 --2- 172.21.15.22:0/3449031332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e0815fec0 0x7f8e0815bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.588+0000 7f8e0fb06640 1 --2- 172.21.15.22:0/3449031332 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8dec067fc0 0x7f8dec06a480 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.588+0000 7f8e0fb06640 1 --2- 172.21.15.22:0/3449031332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e081459b0 0x7f8e081fb2d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.588+0000 7f8e0fb06640 1 -- 172.21.15.22:0/3449031332 >> 172.21.15.22:0/3449031332 conn(0x7f8e080b2c70 msgr2=0x7f8e0813ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:20.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.588+0000 7f8e0fb06640 1 -- 172.21.15.22:0/3449031332 shutdown_connections 2024-03-29T04:18:20.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.588+0000 7f8e0fb06640 1 -- 172.21.15.22:0/3449031332 wait complete. 2024-03-29T04:18:20.588 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:18:20.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.744+0000 7f1a10434640 1 Processor -- start 2024-03-29T04:18:20.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.744+0000 7f1a10434640 1 -- start start 2024-03-29T04:18:20.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.744+0000 7f1a10434640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1a0806c4d0 0x7f1a080ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:20.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.744+0000 7f1a10434640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a080b5470 con 0x7f1a08147e00 2024-03-29T04:18:20.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.744+0000 7f1a10434640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a080b5750 con 0x7f1a0806c070 2024-03-29T04:18:20.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.744+0000 7f1a10434640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f1a080b5a30 con 0x7f1a0806c4d0 2024-03-29T04:18:20.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.744+0000 7f1a0e9d1640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1a0806c4d0 0x7f1a080ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:20.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.744+0000 7f1a0e1d0640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1a08147e00 0x7f1a080b49f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57972/0 (socket says 172.21.15.22:57972) 2024-03-29T04:18:20.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.744+0000 7f1a0e1d0640 1 -- 172.21.15.22:0/1968378762 learned_addr learned my addr 172.21.15.22:0/1968378762 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:20.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.744+0000 7f1a0e9d1640 1 -- 172.21.15.22:0/1968378762 >> v1:172.21.15.151:6789/0 conn(0x7f1a0806c070 legacy=0x7f1a080adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:20.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.744+0000 7f1a0e9d1640 1 -- 172.21.15.22:0/1968378762 >> v1:172.21.15.22:6789/0 conn(0x7f1a08147e00 legacy=0x7f1a080b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:20.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.744+0000 7f1a0e9d1640 1 -- 172.21.15.22:0/1968378762 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a080b5d10 con 0x7f1a0806c4d0 2024-03-29T04:18:20.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a0e9d1640 1 --2- 172.21.15.22:0/1968378762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0806c4d0 0x7f1a080ae2d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009d90 tx=0x7f1a0001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17b625030fcc27d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:20.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a0d1ce640 1 -- 172.21.15.22:0/1968378762 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a000b2070 con 0x7f1a0806c4d0 2024-03-29T04:18:20.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a0d1ce640 1 -- 172.21.15.22:0/1968378762 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a000a5830 con 0x7f1a0806c4d0 2024-03-29T04:18:20.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a0d1ce640 1 -- 172.21.15.22:0/1968378762 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a000a5c00 con 0x7f1a0806c4d0 2024-03-29T04:18:20.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a10434640 1 -- 172.21.15.22:0/1968378762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0806c4d0 msgr2=0x7f1a080ae2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:20.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a10434640 1 --2- 172.21.15.22:0/1968378762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0806c4d0 0x7f1a080ae2d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009d90 tx=0x7f1a0001c970 comp rx=0 tx=0).stop 2024-03-29T04:18:20.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a10434640 1 -- 172.21.15.22:0/1968378762 shutdown_connections 2024-03-29T04:18:20.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a10434640 1 --2- 172.21.15.22:0/1968378762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0806c4d0 0x7f1a080ae2d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a10434640 1 -- 172.21.15.22:0/1968378762 >> 172.21.15.22:0/1968378762 conn(0x7f1a080d1070 msgr2=0x7f1a080d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:20.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a10434640 1 -- 172.21.15.22:0/1968378762 shutdown_connections 2024-03-29T04:18:20.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a10434640 1 -- 172.21.15.22:0/1968378762 wait complete. 2024-03-29T04:18:20.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a10434640 1 Processor -- start 2024-03-29T04:18:20.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a10434640 1 -- start start 2024-03-29T04:18:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a10434640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0806c070 0x7f1a08160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a10434640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a0806c4d0 0x7f1a08142ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a10434640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a08147e00 0x7f1a08143220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a10434640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0816c2b0 con 0x7f1a0806c070 2024-03-29T04:18:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a10434640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0816c130 con 0x7f1a0806c4d0 2024-03-29T04:18:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a10434640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1a0816c430 con 0x7f1a08147e00 2024-03-29T04:18:20.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a0e1d0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0806c070 0x7f1a08160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:20.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a0d9cf640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a0806c4d0 0x7f1a08142ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:20.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a0e1d0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0806c070 0x7f1a08160ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59482/0 (socket says 172.21.15.22:59482) 2024-03-29T04:18:20.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.748+0000 7f1a0e1d0640 1 -- 172.21.15.22:0/277537993 learned_addr learned my addr 172.21.15.22:0/277537993 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:20.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.752+0000 7f1a0e9d1640 1 --2- 172.21.15.22:0/277537993 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a08147e00 0x7f1a08143220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:20.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.752+0000 7f1a0e1d0640 1 -- 172.21.15.22:0/277537993 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a08147e00 msgr2=0x7f1a08143220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:20.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.752+0000 7f1a0e1d0640 1 --2- 172.21.15.22:0/277537993 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a08147e00 0x7f1a08143220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.752+0000 7f1a0e1d0640 1 -- 172.21.15.22:0/277537993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a0806c4d0 msgr2=0x7f1a08142ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:20.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.752+0000 7f1a0e1d0640 1 --2- 172.21.15.22:0/277537993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a0806c4d0 0x7f1a08142ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.752+0000 7f1a0e1d0640 1 -- 172.21.15.22:0/277537993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a081610e0 con 0x7f1a0806c070 2024-03-29T04:18:20.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.752+0000 7f1a0e1d0640 1 --2- 172.21.15.22:0/277537993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0806c070 0x7f1a08160ba0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400c400 tx=0x7f1a0400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:20.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.752+0000 7f19feffd640 1 -- 172.21.15.22:0/277537993 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04017070 con 0x7f1a0806c070 2024-03-29T04:18:20.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.752+0000 7f19feffd640 1 -- 172.21.15.22:0/277537993 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a04005cf0 con 0x7f1a0806c070 2024-03-29T04:18:20.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.752+0000 7f1a10434640 1 -- 172.21.15.22:0/277537993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a08161380 con 0x7f1a0806c070 2024-03-29T04:18:20.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.752+0000 7f19feffd640 1 -- 172.21.15.22:0/277537993 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a040072d0 con 0x7f1a0806c070 2024-03-29T04:18:20.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.752+0000 7f1a10434640 1 -- 172.21.15.22:0/277537993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a08161940 con 0x7f1a0806c070 2024-03-29T04:18:20.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.752+0000 7f1a10434640 1 -- 172.21.15.22:0/277537993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a08068900 con 0x7f1a0806c070 2024-03-29T04:18:20.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.752+0000 7f19feffd640 1 -- 172.21.15.22:0/277537993 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1a0401a070 con 0x7f1a0806c070 2024-03-29T04:18:20.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.752+0000 7f19feffd640 1 --2- 172.21.15.22:0/277537993 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f19ec067fc0 0x7f19ec06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:20.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.752+0000 7f19feffd640 1 -- 172.21.15.22:0/277537993 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f1a04085b20 con 0x7f1a0806c070 2024-03-29T04:18:20.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.752+0000 7f1a0d9cf640 1 --2- 172.21.15.22:0/277537993 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f19ec067fc0 0x7f19ec06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:20.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.752+0000 7f1a0d9cf640 1 --2- 172.21.15.22:0/277537993 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f19ec067fc0 0x7f19ec06a480 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1a08143830 tx=0x7f19f8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.756+0000 7f19feffd640 1 -- 172.21.15.22:0/277537993 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1a04055910 con 0x7f1a0806c070 2024-03-29T04:18:20.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.920+0000 7f1a10434640 1 -- 172.21.15.22:0/277537993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1a080b2970 con 0x7f1a0806c070 2024-03-29T04:18:20.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.920+0000 7f19feffd640 1 -- 172.21.15.22:0/277537993 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f1a04059c10 con 0x7f1a0806c070 2024-03-29T04:18:20.902 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:20.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.924+0000 7f1a10434640 1 -- 172.21.15.22:0/277537993 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f19ec067fc0 msgr2=0x7f19ec06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:20.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.924+0000 7f1a10434640 1 --2- 172.21.15.22:0/277537993 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f19ec067fc0 0x7f19ec06a480 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1a08143830 tx=0x7f19f8009040 comp rx=0 tx=0).stop 2024-03-29T04:18:20.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.924+0000 7f1a10434640 1 -- 172.21.15.22:0/277537993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0806c070 msgr2=0x7f1a08160ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:20.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.924+0000 7f1a10434640 1 --2- 172.21.15.22:0/277537993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0806c070 0x7f1a08160ba0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400c400 tx=0x7f1a0400c8c0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.924+0000 7f1a10434640 1 -- 172.21.15.22:0/277537993 shutdown_connections 2024-03-29T04:18:20.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.924+0000 7f1a10434640 1 --2- 172.21.15.22:0/277537993 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1a08147e00 0x7f1a08143220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.924+0000 7f1a10434640 1 --2- 172.21.15.22:0/277537993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a0806c4d0 0x7f1a08142ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.924+0000 7f1a10434640 1 --2- 172.21.15.22:0/277537993 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f19ec067fc0 0x7f19ec06a480 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.924+0000 7f1a10434640 1 --2- 172.21.15.22:0/277537993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a0806c070 0x7f1a08160ba0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:20.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.924+0000 7f1a10434640 1 -- 172.21.15.22:0/277537993 >> 172.21.15.22:0/277537993 conn(0x7f1a080d1070 msgr2=0x7f1a080af2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:20.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.924+0000 7f1a10434640 1 -- 172.21.15.22:0/277537993 shutdown_connections 2024-03-29T04:18:20.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:20.924+0000 7f1a10434640 1 -- 172.21.15.22:0/277537993 wait complete. 2024-03-29T04:18:20.922 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.060+0000 7f614809d640 1 Processor -- start 2024-03-29T04:18:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.060+0000 7f614809d640 1 -- start start 2024-03-29T04:18:21.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.064+0000 7f614809d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f614015a840 0x7f614015e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.064+0000 7f614809d640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f614015e940 0x7f614006da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.064+0000 7f614809d640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6140072cb0 con 0x7f614006df90 2024-03-29T04:18:21.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.064+0000 7f614809d640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f614015eda0 con 0x7f614015a840 2024-03-29T04:18:21.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.064+0000 7f614809d640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f6140072f90 con 0x7f614015e940 2024-03-29T04:18:21.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.064+0000 7f6145e39640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f614015a840 0x7f614015e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.064+0000 7f614663a640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f614006df90 0x7f614006e420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57974/0 (socket says 172.21.15.22:57974) 2024-03-29T04:18:21.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.064+0000 7f614663a640 1 -- 172.21.15.22:0/82616732 learned_addr learned my addr 172.21.15.22:0/82616732 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:21.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.064+0000 7f6145638640 1 --2- 172.21.15.22:0/82616732 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f614015e940 0x7f614006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.064+0000 7f6144e37640 1 -- 172.21.15.22:0/82616732 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3475265699 0 0) 0x7f6140072cb0 con 0x7f614006df90 2024-03-29T04:18:21.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.064+0000 7f6144e37640 1 -- 172.21.15.22:0/82616732 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f612c003530 con 0x7f614006df90 2024-03-29T04:18:21.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.064+0000 7f6145e39640 1 -- 172.21.15.22:0/82616732 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f614015e940 msgr2=0x7f614006da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:21.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.064+0000 7f6145e39640 1 --2- 172.21.15.22:0/82616732 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f614015e940 0x7f614006da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.064+0000 7f6145e39640 1 -- 172.21.15.22:0/82616732 >> v1:172.21.15.22:6789/0 conn(0x7f614006df90 legacy=0x7f614006e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:21.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.064+0000 7f6145e39640 1 -- 172.21.15.22:0/82616732 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6140073270 con 0x7f614015a840 2024-03-29T04:18:21.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.064+0000 7f6145e39640 1 --2- 172.21.15.22:0/82616732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f614015a840 0x7f614015e400 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f613c0097c0 tx=0x7f613c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd08e886339bd33 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:21.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.064+0000 7f6144e37640 1 -- 172.21.15.22:0/82616732 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c0ab600 con 0x7f614015a840 2024-03-29T04:18:21.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.064+0000 7f6144e37640 1 -- 172.21.15.22:0/82616732 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f613c0b0070 con 0x7f614015a840 2024-03-29T04:18:21.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.064+0000 7f6144e37640 1 -- 172.21.15.22:0/82616732 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c0ab9c0 con 0x7f614015a840 2024-03-29T04:18:21.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614809d640 1 -- 172.21.15.22:0/82616732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f614015a840 msgr2=0x7f614015e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:21.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614809d640 1 --2- 172.21.15.22:0/82616732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f614015a840 0x7f614015e400 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f613c0097c0 tx=0x7f613c01c3e0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614809d640 1 -- 172.21.15.22:0/82616732 shutdown_connections 2024-03-29T04:18:21.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614809d640 1 --2- 172.21.15.22:0/82616732 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f614015e940 0x7f614006da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614809d640 1 --2- 172.21.15.22:0/82616732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f614015a840 0x7f614015e400 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614809d640 1 -- 172.21.15.22:0/82616732 >> 172.21.15.22:0/82616732 conn(0x7f61400cfee0 msgr2=0x7f61400d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:21.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614809d640 1 -- 172.21.15.22:0/82616732 shutdown_connections 2024-03-29T04:18:21.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614809d640 1 -- 172.21.15.22:0/82616732 wait complete. 2024-03-29T04:18:21.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614809d640 1 Processor -- start 2024-03-29T04:18:21.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614809d640 1 -- start start 2024-03-29T04:18:21.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614809d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f614006df90 0x7f61400b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614809d640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f614015a840 0x7f614013ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614809d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f614015e940 0x7f6140156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614809d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6140072a50 con 0x7f614015e940 2024-03-29T04:18:21.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614809d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f61400728d0 con 0x7f614006df90 2024-03-29T04:18:21.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614809d640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f61400b4310 con 0x7f614015a840 2024-03-29T04:18:21.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614663a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f614015e940 0x7f6140156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614663a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f614015e940 0x7f6140156550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59510/0 (socket says 172.21.15.22:59510) 2024-03-29T04:18:21.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614663a640 1 -- 172.21.15.22:0/312861464 learned_addr learned my addr 172.21.15.22:0/312861464 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:21.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f6145638640 1 --2- 172.21.15.22:0/312861464 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f614015a840 0x7f614013ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f6145e39640 1 --2- 172.21.15.22:0/312861464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f614006df90 0x7f61400b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614663a640 1 -- 172.21.15.22:0/312861464 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f614015a840 msgr2=0x7f614013ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:21.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614663a640 1 --2- 172.21.15.22:0/312861464 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f614015a840 0x7f614013ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614663a640 1 -- 172.21.15.22:0/312861464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f614006df90 msgr2=0x7f61400b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:21.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614663a640 1 --2- 172.21.15.22:0/312861464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f614006df90 0x7f61400b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614663a640 1 -- 172.21.15.22:0/312861464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61400b0cd0 con 0x7f614015e940 2024-03-29T04:18:21.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614663a640 1 --2- 172.21.15.22:0/312861464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f614015e940 0x7f6140156550 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f613800cad0 tx=0x7f613800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:21.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f61367fc640 1 -- 172.21.15.22:0/312861464 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138017070 con 0x7f614015e940 2024-03-29T04:18:21.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f61367fc640 1 -- 172.21.15.22:0/312861464 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f613800ac30 con 0x7f614015e940 2024-03-29T04:18:21.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f61367fc640 1 -- 172.21.15.22:0/312861464 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138012400 con 0x7f614015e940 2024-03-29T04:18:21.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614809d640 1 -- 172.21.15.22:0/312861464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61400b0fb0 con 0x7f614015e940 2024-03-29T04:18:21.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.068+0000 7f614809d640 1 -- 172.21.15.22:0/312861464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61400b1570 con 0x7f614015e940 2024-03-29T04:18:21.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.072+0000 7f614809d640 1 -- 172.21.15.22:0/312861464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61400719d0 con 0x7f614015e940 2024-03-29T04:18:21.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.072+0000 7f61367fc640 1 -- 172.21.15.22:0/312861464 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f613801a070 con 0x7f614015e940 2024-03-29T04:18:21.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.072+0000 7f61367fc640 1 --2- 172.21.15.22:0/312861464 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6124067fc0 0x7f612406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.072+0000 7f61367fc640 1 -- 172.21.15.22:0/312861464 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f6138086a30 con 0x7f614015e940 2024-03-29T04:18:21.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.072+0000 7f6145e39640 1 --2- 172.21.15.22:0/312861464 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6124067fc0 0x7f612406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.072+0000 7f6145e39640 1 --2- 172.21.15.22:0/312861464 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6124067fc0 0x7f612406a480 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f613c0af000 tx=0x7f613c0a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:21.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.076+0000 7f61367fc640 1 -- 172.21.15.22:0/312861464 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f6138021020 con 0x7f614015e940 2024-03-29T04:18:21.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.240+0000 7f614809d640 1 -- 172.21.15.22:0/312861464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6140068870 con 0x7f614015e940 2024-03-29T04:18:21.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.244+0000 7f61367fc640 1 -- 172.21.15.22:0/312861464 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18847 (secure 0 0 0) 0x7f61380567e0 con 0x7f614015e940 2024-03-29T04:18:21.225 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:21.225 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:18:16.893439+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:21.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.244+0000 7f614809d640 1 -- 172.21.15.22:0/312861464 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6124067fc0 msgr2=0x7f612406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:21.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.244+0000 7f614809d640 1 --2- 172.21.15.22:0/312861464 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6124067fc0 0x7f612406a480 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f613c0af000 tx=0x7f613c0a7000 comp rx=0 tx=0).stop 2024-03-29T04:18:21.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.244+0000 7f614809d640 1 -- 172.21.15.22:0/312861464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f614015e940 msgr2=0x7f6140156550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:21.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.244+0000 7f614809d640 1 --2- 172.21.15.22:0/312861464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f614015e940 0x7f6140156550 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f613800cad0 tx=0x7f613800cf90 comp rx=0 tx=0).stop 2024-03-29T04:18:21.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.244+0000 7f614809d640 1 -- 172.21.15.22:0/312861464 shutdown_connections 2024-03-29T04:18:21.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.244+0000 7f614809d640 1 --2- 172.21.15.22:0/312861464 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6124067fc0 0x7f612406a480 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.244+0000 7f614809d640 1 --2- 172.21.15.22:0/312861464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f614015e940 0x7f6140156550 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.244+0000 7f614809d640 1 --2- 172.21.15.22:0/312861464 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f614015a840 0x7f614013ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.244+0000 7f614809d640 1 --2- 172.21.15.22:0/312861464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f614006df90 0x7f61400b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.244+0000 7f614809d640 1 -- 172.21.15.22:0/312861464 >> 172.21.15.22:0/312861464 conn(0x7f61400cfee0 msgr2=0x7f614006ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:21.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.248+0000 7f614809d640 1 -- 172.21.15.22:0/312861464 shutdown_connections 2024-03-29T04:18:21.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.248+0000 7f614809d640 1 -- 172.21.15.22:0/312861464 wait complete. 2024-03-29T04:18:21.245 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:17:34.034142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:17:34.886659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:17:38.628632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:18:21.245 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:18:21.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1317819640 1 Processor -- start 2024-03-29T04:18:21.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1317819640 1 -- start start 2024-03-29T04:18:21.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1317819640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f131015e4c0 0x7f13100b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1317819640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f131006c850 con 0x7f131006c070 2024-03-29T04:18:21.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1317819640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f131006cb30 con 0x7f13100cf6c0 2024-03-29T04:18:21.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1317819640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f131015e8a0 con 0x7f131015e4c0 2024-03-29T04:18:21.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1314db4640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f131006c070 0x7f13100cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57978/0 (socket says 172.21.15.22:57978) 2024-03-29T04:18:21.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1314db4640 1 -- 172.21.15.22:0/2596202805 learned_addr learned my addr 172.21.15.22:0/2596202805 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:21.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f13155b5640 1 --2- 172.21.15.22:0/2596202805 >> v2:172.21.15.151:3300/0 conn(0x7f131015e4c0 0x7f13100b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1307fff640 1 -- 172.21.15.22:0/2596202805 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 838255583 0 0) 0x7f131006c850 con 0x7f131006c070 2024-03-29T04:18:21.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1307fff640 1 -- 172.21.15.22:0/2596202805 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12fc003530 con 0x7f131006c070 2024-03-29T04:18:21.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1307fff640 1 -- 172.21.15.22:0/2596202805 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 190552104 0 0) 0x7f12fc003530 con 0x7f131006c070 2024-03-29T04:18:21.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1307fff640 1 -- 172.21.15.22:0/2596202805 >> v2:172.21.15.151:3300/0 conn(0x7f131015e4c0 msgr2=0x7f13100b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:21.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1307fff640 1 --2- 172.21.15.22:0/2596202805 >> v2:172.21.15.151:3300/0 conn(0x7f131015e4c0 0x7f13100b9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1307fff640 1 -- 172.21.15.22:0/2596202805 >> v1:172.21.15.151:6789/0 conn(0x7f13100cf6c0 legacy=0x7f13100d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:21.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1307fff640 1 -- 172.21.15.22:0/2596202805 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131006ce10 con 0x7f131006c070 2024-03-29T04:18:21.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1307fff640 1 -- 172.21.15.22:0/2596202805 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f1300001680 con 0x7f131006c070 2024-03-29T04:18:21.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1307fff640 1 -- 172.21.15.22:0/2596202805 >> v1:172.21.15.22:6789/0 conn(0x7f131006c070 legacy=0x7f13100cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:21.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1307fff640 1 --2- 172.21.15.22:0/2596202805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12fc003a10 0x7f12fc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1307fff640 1 --2- 172.21.15.22:0/2596202805 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12fc004810 0x7f12fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1307fff640 1 --2- 172.21.15.22:0/2596202805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12fc0051d0 0x7f12fc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1307fff640 1 -- 172.21.15.22:0/2596202805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f131015e8a0 con 0x7f12fc003a10 2024-03-29T04:18:21.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1307fff640 1 -- 172.21.15.22:0/2596202805 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc00c0d0 con 0x7f12fc0051d0 2024-03-29T04:18:21.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1307fff640 1 -- 172.21.15.22:0/2596202805 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f12fc00c2d0 con 0x7f12fc004810 2024-03-29T04:18:21.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f13155b5640 1 --2- 172.21.15.22:0/2596202805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12fc003a10 0x7f12fc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1314db4640 1 --2- 172.21.15.22:0/2596202805 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12fc004810 0x7f12fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1315db6640 1 --2- 172.21.15.22:0/2596202805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12fc0051d0 0x7f12fc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f13155b5640 1 -- 172.21.15.22:0/2596202805 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12fc004810 msgr2=0x7f12fc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:21.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f13155b5640 1 --2- 172.21.15.22:0/2596202805 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12fc004810 0x7f12fc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f13155b5640 1 -- 172.21.15.22:0/2596202805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12fc0051d0 msgr2=0x7f12fc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:21.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f13155b5640 1 --2- 172.21.15.22:0/2596202805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12fc0051d0 0x7f12fc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f13155b5640 1 -- 172.21.15.22:0/2596202805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12fc003530 con 0x7f12fc003a10 2024-03-29T04:18:21.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.388+0000 7f1315db6640 1 -- 172.21.15.22:0/2596202805 reap_dead start 2024-03-29T04:18:21.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.392+0000 7f13155b5640 1 --2- 172.21.15.22:0/2596202805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12fc003a10 0x7f12fc003e30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f130c009ae0 tx=0x7f130c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f1cb3bbae4cf9c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:21.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.392+0000 7f1307fff640 1 -- 172.21.15.22:0/2596202805 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c0096b0 con 0x7f12fc003a10 2024-03-29T04:18:21.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.392+0000 7f1307fff640 1 -- 172.21.15.22:0/2596202805 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f130c0a6450 con 0x7f12fc003a10 2024-03-29T04:18:21.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.392+0000 7f1307fff640 1 -- 172.21.15.22:0/2596202805 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c0a6840 con 0x7f12fc003a10 2024-03-29T04:18:21.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.392+0000 7f1317819640 1 -- 172.21.15.22:0/2596202805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12fc003a10 msgr2=0x7f12fc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:21.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.392+0000 7f1317819640 1 --2- 172.21.15.22:0/2596202805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12fc003a10 0x7f12fc003e30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f130c009ae0 tx=0x7f130c01ca70 comp rx=0 tx=0).stop 2024-03-29T04:18:21.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.392+0000 7f1317819640 1 -- 172.21.15.22:0/2596202805 shutdown_connections 2024-03-29T04:18:21.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.392+0000 7f1317819640 1 --2- 172.21.15.22:0/2596202805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12fc003a10 0x7f12fc003e30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.392+0000 7f1317819640 1 -- 172.21.15.22:0/2596202805 >> 172.21.15.22:0/2596202805 conn(0x7f13100b6410 msgr2=0x7f13100d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:21.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.392+0000 7f1317819640 1 -- 172.21.15.22:0/2596202805 shutdown_connections 2024-03-29T04:18:21.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.392+0000 7f1317819640 1 -- 172.21.15.22:0/2596202805 wait complete. 2024-03-29T04:18:21.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1317819640 1 Processor -- start 2024-03-29T04:18:21.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1317819640 1 -- start start 2024-03-29T04:18:21.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1317819640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12fc003a10 0x7f1310154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1317819640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13101faba0 0x7f13101465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1317819640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13101fb000 0x7f13100cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1317819640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1310162b20 con 0x7f13101faba0 2024-03-29T04:18:21.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1317819640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13101629a0 con 0x7f13101fb000 2024-03-29T04:18:21.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1317819640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1310162ca0 con 0x7f12fc003a10 2024-03-29T04:18:21.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1314db4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13101faba0 0x7f13101465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1314db4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13101faba0 0x7f13101465c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59520/0 (socket says 172.21.15.22:59520) 2024-03-29T04:18:21.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1314db4640 1 -- 172.21.15.22:0/954477848 learned_addr learned my addr 172.21.15.22:0/954477848 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:21.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f13155b5640 1 --2- 172.21.15.22:0/954477848 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12fc003a10 0x7f1310154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1315db6640 1 --2- 172.21.15.22:0/954477848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13101fb000 0x7f13100cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1314db4640 1 -- 172.21.15.22:0/954477848 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12fc003a10 msgr2=0x7f1310154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:21.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1314db4640 1 --2- 172.21.15.22:0/954477848 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12fc003a10 0x7f1310154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1314db4640 1 -- 172.21.15.22:0/954477848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13101fb000 msgr2=0x7f13100cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:21.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1314db4640 1 --2- 172.21.15.22:0/954477848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13101fb000 0x7f13100cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1314db4640 1 -- 172.21.15.22:0/954477848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13100cb550 con 0x7f13101faba0 2024-03-29T04:18:21.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1314db4640 1 --2- 172.21.15.22:0/954477848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13101faba0 0x7f13101465c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1300001c00 tx=0x7f130000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:21.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1305ffb640 1 -- 172.21.15.22:0/954477848 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130000c6c0 con 0x7f13101faba0 2024-03-29T04:18:21.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1305ffb640 1 -- 172.21.15.22:0/954477848 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1300018070 con 0x7f13101faba0 2024-03-29T04:18:21.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1317819640 1 -- 172.21.15.22:0/954477848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13100cb830 con 0x7f13101faba0 2024-03-29T04:18:21.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1305ffb640 1 -- 172.21.15.22:0/954477848 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130000cac0 con 0x7f13101faba0 2024-03-29T04:18:21.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1317819640 1 -- 172.21.15.22:0/954477848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13100cbdf0 con 0x7f13101faba0 2024-03-29T04:18:21.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.396+0000 7f1317819640 1 -- 172.21.15.22:0/954477848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1310068900 con 0x7f13101faba0 2024-03-29T04:18:21.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.400+0000 7f1305ffb640 1 -- 172.21.15.22:0/954477848 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f130000cc60 con 0x7f13101faba0 2024-03-29T04:18:21.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.400+0000 7f1305ffb640 1 --2- 172.21.15.22:0/954477848 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f12f4067fc0 0x7f12f406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.400+0000 7f1305ffb640 1 -- 172.21.15.22:0/954477848 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f13000867d0 con 0x7f13101faba0 2024-03-29T04:18:21.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.400+0000 7f13155b5640 1 --2- 172.21.15.22:0/954477848 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f12f4067fc0 0x7f12f406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.400+0000 7f13155b5640 1 --2- 172.21.15.22:0/954477848 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f12f4067fc0 0x7f12f406a480 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f130c0ad000 tx=0x7f130c0b2040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:21.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.404+0000 7f1305ffb640 1 -- 172.21.15.22:0/954477848 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1300056b80 con 0x7f13101faba0 2024-03-29T04:18:21.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.568+0000 7f1317819640 1 -- 172.21.15.22:0/954477848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f13100d43e0 con 0x7f13101faba0 2024-03-29T04:18:21.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.568+0000 7f1305ffb640 1 -- 172.21.15.22:0/954477848 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f1300056480 con 0x7f13101faba0 2024-03-29T04:18:21.549 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:18:21.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.568+0000 7f1317819640 1 -- 172.21.15.22:0/954477848 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f12f4067fc0 msgr2=0x7f12f406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:21.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.568+0000 7f1317819640 1 --2- 172.21.15.22:0/954477848 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f12f4067fc0 0x7f12f406a480 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f130c0ad000 tx=0x7f130c0b2040 comp rx=0 tx=0).stop 2024-03-29T04:18:21.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.568+0000 7f1317819640 1 -- 172.21.15.22:0/954477848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13101faba0 msgr2=0x7f13101465c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:21.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.568+0000 7f1317819640 1 --2- 172.21.15.22:0/954477848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13101faba0 0x7f13101465c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1300001c00 tx=0x7f130000eb90 comp rx=0 tx=0).stop 2024-03-29T04:18:21.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.568+0000 7f1317819640 1 -- 172.21.15.22:0/954477848 shutdown_connections 2024-03-29T04:18:21.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.568+0000 7f1317819640 1 --2- 172.21.15.22:0/954477848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13101fb000 0x7f13100cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.568+0000 7f1317819640 1 --2- 172.21.15.22:0/954477848 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f12f4067fc0 0x7f12f406a480 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.568+0000 7f1317819640 1 --2- 172.21.15.22:0/954477848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13101faba0 0x7f13101465c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.568+0000 7f1317819640 1 --2- 172.21.15.22:0/954477848 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f12fc003a10 0x7f1310154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.568+0000 7f1317819640 1 -- 172.21.15.22:0/954477848 >> 172.21.15.22:0/954477848 conn(0x7f13100b6410 msgr2=0x7f13100b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:21.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.568+0000 7f1317819640 1 -- 172.21.15.22:0/954477848 shutdown_connections 2024-03-29T04:18:21.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.568+0000 7f1317819640 1 -- 172.21.15.22:0/954477848 wait complete. 2024-03-29T04:18:21.567 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:18:21.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.712+0000 7f69aae52640 1 Processor -- start 2024-03-29T04:18:21.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.712+0000 7f69aae52640 1 -- start start 2024-03-29T04:18:21.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.712+0000 7f69aae52640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f69a4161c50 0x7f69a40ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.712+0000 7f69aae52640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f69a40b7d40 0x7f69a40cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.712+0000 7f69aae52640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69a40ce660 con 0x7f69a40cf720 2024-03-29T04:18:21.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.712+0000 7f69aae52640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f69a40b78b0 con 0x7f69a4161c50 2024-03-29T04:18:21.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.712+0000 7f69aae52640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f69a40b50f0 con 0x7f69a40b7d40 2024-03-29T04:18:21.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.712+0000 7f69a93ef640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f69a40cf720 0x7f69a4162210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57986/0 (socket says 172.21.15.22:57986) 2024-03-29T04:18:21.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.712+0000 7f69a93ef640 1 -- 172.21.15.22:0/3443954363 learned_addr learned my addr 172.21.15.22:0/3443954363 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:21.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.712+0000 7f69a8bee640 1 --2- 172.21.15.22:0/3443954363 >> v2:172.21.15.22:3300/0 conn(0x7f69a4161c50 0x7f69a40ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.712+0000 7f69a3fff640 1 --2- 172.21.15.22:0/3443954363 >> v2:172.21.15.151:3300/0 conn(0x7f69a40b7d40 0x7f69a40cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.712+0000 7f69a37fe640 1 -- 172.21.15.22:0/3443954363 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3180585151 0 0) 0x7f69a40ce660 con 0x7f69a40cf720 2024-03-29T04:18:21.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.712+0000 7f69a37fe640 1 -- 172.21.15.22:0/3443954363 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f698c003530 con 0x7f69a40cf720 2024-03-29T04:18:21.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.712+0000 7f69a8bee640 1 -- 172.21.15.22:0/3443954363 >> v2:172.21.15.151:3300/0 conn(0x7f69a40b7d40 msgr2=0x7f69a40cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:21.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.712+0000 7f69a8bee640 1 --2- 172.21.15.22:0/3443954363 >> v2:172.21.15.151:3300/0 conn(0x7f69a40b7d40 0x7f69a40cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.712+0000 7f69a8bee640 1 -- 172.21.15.22:0/3443954363 >> v1:172.21.15.22:6789/0 conn(0x7f69a40cf720 legacy=0x7f69a4162210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:21.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.712+0000 7f69a8bee640 1 -- 172.21.15.22:0/3443954363 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a40b53d0 con 0x7f69a4161c50 2024-03-29T04:18:21.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.712+0000 7f69a3fff640 1 --2- 172.21.15.22:0/3443954363 >> v2:172.21.15.151:3300/0 conn(0x7f69a40b7d40 0x7f69a40cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:21.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69a8bee640 1 --2- 172.21.15.22:0/3443954363 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69a4161c50 0x7f69a40ce120 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f69a41632b0 tx=0x7f699401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcf780a3467774f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:21.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69a37fe640 1 -- 172.21.15.22:0/3443954363 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69940b0070 con 0x7f69a4161c50 2024-03-29T04:18:21.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69a37fe640 1 -- 172.21.15.22:0/3443954363 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69940ab600 con 0x7f69a4161c50 2024-03-29T04:18:21.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69a37fe640 1 -- 172.21.15.22:0/3443954363 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69940ab9d0 con 0x7f69a4161c50 2024-03-29T04:18:21.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69aae52640 1 -- 172.21.15.22:0/3443954363 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69a4161c50 msgr2=0x7f69a40ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:21.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69aae52640 1 --2- 172.21.15.22:0/3443954363 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69a4161c50 0x7f69a40ce120 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f69a41632b0 tx=0x7f699401c360 comp rx=0 tx=0).stop 2024-03-29T04:18:21.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69aae52640 1 -- 172.21.15.22:0/3443954363 shutdown_connections 2024-03-29T04:18:21.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69aae52640 1 --2- 172.21.15.22:0/3443954363 >> v2:172.21.15.151:3300/0 conn(0x7f69a40b7d40 0x7f69a40cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69aae52640 1 --2- 172.21.15.22:0/3443954363 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69a4161c50 0x7f69a40ce120 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69aae52640 1 -- 172.21.15.22:0/3443954363 >> 172.21.15.22:0/3443954363 conn(0x7f69a40ceae0 msgr2=0x7f69a40ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:21.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69aae52640 1 -- 172.21.15.22:0/3443954363 shutdown_connections 2024-03-29T04:18:21.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69aae52640 1 -- 172.21.15.22:0/3443954363 wait complete. 2024-03-29T04:18:21.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69aae52640 1 Processor -- start 2024-03-29T04:18:21.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69aae52640 1 -- start start 2024-03-29T04:18:21.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69aae52640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f69a40b7d40 0x7f69a4156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69aae52640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69a40cf720 0x7f69a40c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69aae52640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69a4161c50 0x7f69a40ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69aae52640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f69a416b7c0 con 0x7f69a4161c50 2024-03-29T04:18:21.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69aae52640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f69a40cd000 con 0x7f69a40cf720 2024-03-29T04:18:21.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69aae52640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f69a416b940 con 0x7f69a40b7d40 2024-03-29T04:18:21.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69a93ef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69a4161c50 0x7f69a40ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.716+0000 7f69a93ef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69a4161c50 0x7f69a40ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59540/0 (socket says 172.21.15.22:59540) 2024-03-29T04:18:21.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.720+0000 7f69a93ef640 1 -- 172.21.15.22:0/2471314103 learned_addr learned my addr 172.21.15.22:0/2471314103 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:21.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.720+0000 7f69a8bee640 1 --2- 172.21.15.22:0/2471314103 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f69a40b7d40 0x7f69a4156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.720+0000 7f69a3fff640 1 --2- 172.21.15.22:0/2471314103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69a40cf720 0x7f69a40c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.720+0000 7f69a93ef640 1 -- 172.21.15.22:0/2471314103 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f69a40b7d40 msgr2=0x7f69a4156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:21.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.720+0000 7f69a93ef640 1 --2- 172.21.15.22:0/2471314103 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f69a40b7d40 0x7f69a4156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.720+0000 7f69a93ef640 1 -- 172.21.15.22:0/2471314103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69a40cf720 msgr2=0x7f69a40c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:21.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.720+0000 7f69a93ef640 1 --2- 172.21.15.22:0/2471314103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69a40cf720 0x7f69a40c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.720+0000 7f69a93ef640 1 -- 172.21.15.22:0/2471314103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a4152910 con 0x7f69a4161c50 2024-03-29T04:18:21.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.720+0000 7f69a93ef640 1 --2- 172.21.15.22:0/2471314103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69a4161c50 0x7f69a40ca3c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f699c00caa0 tx=0x7f699c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:21.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.720+0000 7f69a17fa640 1 -- 172.21.15.22:0/2471314103 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c017070 con 0x7f69a4161c50 2024-03-29T04:18:21.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.720+0000 7f69a17fa640 1 -- 172.21.15.22:0/2471314103 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f699c00ac00 con 0x7f69a4161c50 2024-03-29T04:18:21.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.720+0000 7f69aae52640 1 -- 172.21.15.22:0/2471314103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a4152bf0 con 0x7f69a4161c50 2024-03-29T04:18:21.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.720+0000 7f69a17fa640 1 -- 172.21.15.22:0/2471314103 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c012400 con 0x7f69a4161c50 2024-03-29T04:18:21.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.720+0000 7f69aae52640 1 -- 172.21.15.22:0/2471314103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a41531b0 con 0x7f69a4161c50 2024-03-29T04:18:21.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.720+0000 7f69aae52640 1 -- 172.21.15.22:0/2471314103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a4068900 con 0x7f69a4161c50 2024-03-29T04:18:21.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.720+0000 7f69a17fa640 1 -- 172.21.15.22:0/2471314103 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f699c01a070 con 0x7f69a4161c50 2024-03-29T04:18:21.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.720+0000 7f69a17fa640 1 --2- 172.21.15.22:0/2471314103 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6990067fc0 0x7f699006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.720+0000 7f69a17fa640 1 -- 172.21.15.22:0/2471314103 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f699c0867c0 con 0x7f69a4161c50 2024-03-29T04:18:21.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.720+0000 7f69a8bee640 1 --2- 172.21.15.22:0/2471314103 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6990067fc0 0x7f699006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.720+0000 7f69a8bee640 1 --2- 172.21.15.22:0/2471314103 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6990067fc0 0x7f699006a480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f69940ad000 tx=0x7f6994008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:21.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.724+0000 7f69a17fa640 1 -- 172.21.15.22:0/2471314103 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f699c056ad0 con 0x7f69a4161c50 2024-03-29T04:18:21.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.880+0000 7f69aae52640 1 -- 172.21.15.22:0/2471314103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f69a40d1c50 con 0x7f69a4161c50 2024-03-29T04:18:21.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.880+0000 7f69a17fa640 1 -- 172.21.15.22:0/2471314103 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f699c0563d0 con 0x7f69a4161c50 2024-03-29T04:18:21.862 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:21.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.884+0000 7f69aae52640 1 -- 172.21.15.22:0/2471314103 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6990067fc0 msgr2=0x7f699006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:21.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.884+0000 7f69aae52640 1 --2- 172.21.15.22:0/2471314103 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6990067fc0 0x7f699006a480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f69940ad000 tx=0x7f6994008040 comp rx=0 tx=0).stop 2024-03-29T04:18:21.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.884+0000 7f69aae52640 1 -- 172.21.15.22:0/2471314103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69a4161c50 msgr2=0x7f69a40ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:21.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.884+0000 7f69aae52640 1 --2- 172.21.15.22:0/2471314103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69a4161c50 0x7f69a40ca3c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f699c00caa0 tx=0x7f699c00cf60 comp rx=0 tx=0).stop 2024-03-29T04:18:21.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.884+0000 7f69aae52640 1 -- 172.21.15.22:0/2471314103 shutdown_connections 2024-03-29T04:18:21.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.884+0000 7f69aae52640 1 --2- 172.21.15.22:0/2471314103 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6990067fc0 0x7f699006a480 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.884+0000 7f69aae52640 1 --2- 172.21.15.22:0/2471314103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69a4161c50 0x7f69a40ca3c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.884+0000 7f69aae52640 1 --2- 172.21.15.22:0/2471314103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69a40cf720 0x7f69a40c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.884+0000 7f69aae52640 1 --2- 172.21.15.22:0/2471314103 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f69a40b7d40 0x7f69a4156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.884+0000 7f69aae52640 1 -- 172.21.15.22:0/2471314103 >> 172.21.15.22:0/2471314103 conn(0x7f69a40ceae0 msgr2=0x7f69a40b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:21.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.884+0000 7f69aae52640 1 -- 172.21.15.22:0/2471314103 shutdown_connections 2024-03-29T04:18:21.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:21.884+0000 7f69aae52640 1 -- 172.21.15.22:0/2471314103 wait complete. 2024-03-29T04:18:21.880 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:18:21.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.016+0000 7fbac96f6640 1 Processor -- start 2024-03-29T04:18:21.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.016+0000 7fbac96f6640 1 -- start start 2024-03-29T04:18:21.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.016+0000 7fbac96f6640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbac413f140 0x7fbac41606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.016+0000 7fbac96f6640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fbac415ff40 0x7fbac406c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:21.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.016+0000 7fbac96f6640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbac406d710 con 0x7fbac406cbb0 2024-03-29T04:18:21.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.016+0000 7fbac96f6640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fbac406d000 con 0x7fbac415ff40 2024-03-29T04:18:21.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.016+0000 7fbac96f6640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fbac406e320 con 0x7fbac413f140 2024-03-29T04:18:21.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.016+0000 7fbac37fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fbac406cbb0 0x7fbac4141a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57988/0 (socket says 172.21.15.22:57988) 2024-03-29T04:18:21.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.016+0000 7fbac37fe640 1 -- 172.21.15.22:0/1625598879 learned_addr learned my addr 172.21.15.22:0/1625598879 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:21.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.016+0000 7fbac27fc640 1 --2- 172.21.15.22:0/1625598879 >> v2:172.21.15.151:3300/0 conn(0x7fbac415ff40 0x7fbac406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.016+0000 7fbac2ffd640 1 --2- 172.21.15.22:0/1625598879 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbac413f140 0x7fbac41606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.016+0000 7fbac1ffb640 1 -- 172.21.15.22:0/1625598879 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3330201365 0 0) 0x7fbac406d710 con 0x7fbac406cbb0 2024-03-29T04:18:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.016+0000 7fbac1ffb640 1 -- 172.21.15.22:0/1625598879 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbaac003530 con 0x7fbac406cbb0 2024-03-29T04:18:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.016+0000 7fbac27fc640 1 -- 172.21.15.22:0/1625598879 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbac413f140 msgr2=0x7fbac41606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.016+0000 7fbac27fc640 1 --2- 172.21.15.22:0/1625598879 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbac413f140 0x7fbac41606d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.016+0000 7fbac27fc640 1 -- 172.21.15.22:0/1625598879 >> v1:172.21.15.22:6789/0 conn(0x7fbac406cbb0 legacy=0x7fbac4141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.016+0000 7fbac1ffb640 1 -- 172.21.15.22:0/1625598879 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3560432921 0 0) 0x7fbaac003530 con 0x7fbac406cbb0 2024-03-29T04:18:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.016+0000 7fbac27fc640 1 -- 172.21.15.22:0/1625598879 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac406d9f0 con 0x7fbac415ff40 2024-03-29T04:18:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.016+0000 7fbac2ffd640 1 --2- 172.21.15.22:0/1625598879 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbac413f140 0x7fbac41606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:18:22.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.020+0000 7fbac27fc640 1 --2- 172.21.15.22:0/1625598879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbac415ff40 0x7fbac406c670 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbab80097c0 tx=0x7fbab801c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c71f7576a839436 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:22.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.020+0000 7fbac1ffb640 1 -- 172.21.15.22:0/1625598879 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab80b0070 con 0x7fbac415ff40 2024-03-29T04:18:22.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.020+0000 7fbac1ffb640 1 -- 172.21.15.22:0/1625598879 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab80ab400 con 0x7fbac415ff40 2024-03-29T04:18:22.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.020+0000 7fbac1ffb640 1 -- 172.21.15.22:0/1625598879 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc001a00 con 0x7fbac415ff40 2024-03-29T04:18:22.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.020+0000 7fbac96f6640 1 -- 172.21.15.22:0/1625598879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbac415ff40 msgr2=0x7fbac406c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:22.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.020+0000 7fbac96f6640 1 --2- 172.21.15.22:0/1625598879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbac415ff40 0x7fbac406c670 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbab80097c0 tx=0x7fbab801c390 comp rx=0 tx=0).stop 2024-03-29T04:18:22.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.020+0000 7fbac96f6640 1 -- 172.21.15.22:0/1625598879 shutdown_connections 2024-03-29T04:18:22.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.020+0000 7fbac96f6640 1 --2- 172.21.15.22:0/1625598879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbac415ff40 0x7fbac406c670 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.020+0000 7fbac96f6640 1 --2- 172.21.15.22:0/1625598879 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbac413f140 0x7fbac41606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.020+0000 7fbac96f6640 1 -- 172.21.15.22:0/1625598879 >> 172.21.15.22:0/1625598879 conn(0x7fbac40d00a0 msgr2=0x7fbac40d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:22.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.020+0000 7fbac96f6640 1 -- 172.21.15.22:0/1625598879 shutdown_connections 2024-03-29T04:18:22.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.020+0000 7fbac96f6640 1 -- 172.21.15.22:0/1625598879 wait complete. 2024-03-29T04:18:22.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.020+0000 7fbac96f6640 1 Processor -- start 2024-03-29T04:18:22.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.020+0000 7fbac96f6640 1 -- start start 2024-03-29T04:18:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbac96f6640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbac406cbb0 0x7fbac40b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbac96f6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbac413f140 0x7fbac40b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbac96f6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbac415ff40 0x7fbac4152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbac96f6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbac4160d90 con 0x7fbac413f140 2024-03-29T04:18:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbac96f6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbac4160c10 con 0x7fbac415ff40 2024-03-29T04:18:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbac96f6640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fbac4168ff0 con 0x7fbac406cbb0 2024-03-29T04:18:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbac27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbac413f140 0x7fbac40b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbac27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbac413f140 0x7fbac40b2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59552/0 (socket says 172.21.15.22:59552) 2024-03-29T04:18:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbac27fc640 1 -- 172.21.15.22:0/2610200210 learned_addr learned my addr 172.21.15.22:0/2610200210 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbac37fe640 1 --2- 172.21.15.22:0/2610200210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbac415ff40 0x7fbac4152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbac2ffd640 1 --2- 172.21.15.22:0/2610200210 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbac406cbb0 0x7fbac40b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbac27fc640 1 -- 172.21.15.22:0/2610200210 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbac406cbb0 msgr2=0x7fbac40b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbac27fc640 1 --2- 172.21.15.22:0/2610200210 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbac406cbb0 0x7fbac40b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbac27fc640 1 -- 172.21.15.22:0/2610200210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbac415ff40 msgr2=0x7fbac4152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbac27fc640 1 --2- 172.21.15.22:0/2610200210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbac415ff40 0x7fbac4152480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbac27fc640 1 -- 172.21.15.22:0/2610200210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac40b4d00 con 0x7fbac413f140 2024-03-29T04:18:22.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbac27fc640 1 --2- 172.21.15.22:0/2610200210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbac413f140 0x7fbac40b2f80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbab801aa40 tx=0x7fbab8019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:22.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbaab7fe640 1 -- 172.21.15.22:0/2610200210 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab80b0040 con 0x7fbac413f140 2024-03-29T04:18:22.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbaab7fe640 1 -- 172.21.15.22:0/2610200210 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab8003620 con 0x7fbac413f140 2024-03-29T04:18:22.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbac96f6640 1 -- 172.21.15.22:0/2610200210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac40b36a0 con 0x7fbac413f140 2024-03-29T04:18:22.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbaab7fe640 1 -- 172.21.15.22:0/2610200210 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab80039d0 con 0x7fbac413f140 2024-03-29T04:18:22.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbac96f6640 1 -- 172.21.15.22:0/2610200210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac40b3c60 con 0x7fbac413f140 2024-03-29T04:18:22.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbac96f6640 1 -- 172.21.15.22:0/2610200210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac4068900 con 0x7fbac413f140 2024-03-29T04:18:22.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbaab7fe640 1 -- 172.21.15.22:0/2610200210 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fbab80bb030 con 0x7fbac413f140 2024-03-29T04:18:22.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbaab7fe640 1 --2- 172.21.15.22:0/2610200210 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbab0067ff0 0x7fbab006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbaab7fe640 1 -- 172.21.15.22:0/2610200210 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fbab811fbb0 con 0x7fbac413f140 2024-03-29T04:18:22.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.024+0000 7fbac2ffd640 1 --2- 172.21.15.22:0/2610200210 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbab0067ff0 0x7fbab006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:22.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.028+0000 7fbac2ffd640 1 --2- 172.21.15.22:0/2610200210 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbab0067ff0 0x7fbab006a4b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbac41417a0 tx=0x7fbab4009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:22.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.032+0000 7fbaab7fe640 1 -- 172.21.15.22:0/2610200210 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fbab80c2020 con 0x7fbac413f140 2024-03-29T04:18:22.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.192+0000 7fbac96f6640 1 -- 172.21.15.22:0/2610200210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbac406c990 con 0x7fbac413f140 2024-03-29T04:18:22.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.196+0000 7fbaab7fe640 1 -- 172.21.15.22:0/2610200210 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fbab8015fb0 con 0x7fbac413f140 2024-03-29T04:18:22.175 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-29T04:18:22.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.196+0000 7fbac96f6640 1 -- 172.21.15.22:0/2610200210 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbab0067ff0 msgr2=0x7fbab006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:22.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.196+0000 7fbac96f6640 1 --2- 172.21.15.22:0/2610200210 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbab0067ff0 0x7fbab006a4b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbac41417a0 tx=0x7fbab4009190 comp rx=0 tx=0).stop 2024-03-29T04:18:22.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.196+0000 7fbac96f6640 1 -- 172.21.15.22:0/2610200210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbac413f140 msgr2=0x7fbac40b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:22.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.196+0000 7fbac96f6640 1 --2- 172.21.15.22:0/2610200210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbac413f140 0x7fbac40b2f80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbab801aa40 tx=0x7fbab8019130 comp rx=0 tx=0).stop 2024-03-29T04:18:22.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.196+0000 7fbac96f6640 1 -- 172.21.15.22:0/2610200210 shutdown_connections 2024-03-29T04:18:22.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.196+0000 7fbac96f6640 1 --2- 172.21.15.22:0/2610200210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbac415ff40 0x7fbac4152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.196+0000 7fbac96f6640 1 --2- 172.21.15.22:0/2610200210 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbab0067ff0 0x7fbab006a4b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.196+0000 7fbac96f6640 1 --2- 172.21.15.22:0/2610200210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbac413f140 0x7fbac40b2f80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.196+0000 7fbac96f6640 1 --2- 172.21.15.22:0/2610200210 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbac406cbb0 0x7fbac40b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.196+0000 7fbac96f6640 1 -- 172.21.15.22:0/2610200210 >> 172.21.15.22:0/2610200210 conn(0x7fbac40d00a0 msgr2=0x7fbac40b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:22.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.196+0000 7fbac96f6640 1 -- 172.21.15.22:0/2610200210 shutdown_connections 2024-03-29T04:18:22.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.196+0000 7fbac96f6640 1 -- 172.21.15.22:0/2610200210 wait complete. 2024-03-29T04:18:22.195 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:18:22.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.340+0000 7f793e058640 1 Processor -- start 2024-03-29T04:18:22.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.340+0000 7f793e058640 1 -- start start 2024-03-29T04:18:22.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.340+0000 7f793e058640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f79380b9e30 0x7f79380cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.340+0000 7f793e058640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f793806c850 con 0x7f793815e4c0 2024-03-29T04:18:22.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.340+0000 7f793e058640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f793806cb30 con 0x7f79380cf4f0 2024-03-29T04:18:22.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.340+0000 7f793e058640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f79380ba290 con 0x7f79380b9e30 2024-03-29T04:18:22.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.340+0000 7f7936ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f79380b9e30 0x7f79380cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:22.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.340+0000 7f7936ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f79380b9e30 0x7f79380cefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59554/0 (socket says 172.21.15.22:59554) 2024-03-29T04:18:22.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.340+0000 7f7936ffd640 1 -- 172.21.15.22:0/2449039344 learned_addr learned my addr 172.21.15.22:0/2449039344 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:22.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.340+0000 7f79377fe640 1 --1- 172.21.15.22:0/2449039344 >> v1:172.21.15.22:6789/0 conn(0x7f793815e4c0 0x7f793806c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57998/0 (socket says 172.21.15.22:57998) 2024-03-29T04:18:22.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.340+0000 7f7936ffd640 1 -- 172.21.15.22:0/2449039344 >> v1:172.21.15.151:6789/0 conn(0x7f79380cf4f0 legacy=0x7f79380d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:22.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.340+0000 7f7936ffd640 1 -- 172.21.15.22:0/2449039344 >> v1:172.21.15.22:6789/0 conn(0x7f793815e4c0 legacy=0x7f793806c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:22.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.340+0000 7f7936ffd640 1 -- 172.21.15.22:0/2449039344 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793806ce10 con 0x7f79380b9e30 2024-03-29T04:18:22.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.340+0000 7f79367fc640 1 -- 172.21.15.22:0/2449039344 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 451741199 0 0) 0x7f793806c850 con 0x7f793815e4c0 2024-03-29T04:18:22.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.344+0000 7f7936ffd640 1 --2- 172.21.15.22:0/2449039344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79380b9e30 0x7f79380cefb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7928009870 tx=0x7f792801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b25ae0481035ea server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:22.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.344+0000 7f79367fc640 1 -- 172.21.15.22:0/2449039344 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79280b2070 con 0x7f79380b9e30 2024-03-29T04:18:22.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.344+0000 7f79367fc640 1 -- 172.21.15.22:0/2449039344 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79280a9070 con 0x7f79380b9e30 2024-03-29T04:18:22.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.344+0000 7f79367fc640 1 -- 172.21.15.22:0/2449039344 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79280ad880 con 0x7f79380b9e30 2024-03-29T04:18:22.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.344+0000 7f793e058640 1 -- 172.21.15.22:0/2449039344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79380b9e30 msgr2=0x7f79380cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:22.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.344+0000 7f793e058640 1 --2- 172.21.15.22:0/2449039344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79380b9e30 0x7f79380cefb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7928009870 tx=0x7f792801c560 comp rx=0 tx=0).stop 2024-03-29T04:18:22.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.344+0000 7f793e058640 1 -- 172.21.15.22:0/2449039344 shutdown_connections 2024-03-29T04:18:22.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.344+0000 7f793e058640 1 --2- 172.21.15.22:0/2449039344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79380b9e30 0x7f79380cefb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.344+0000 7f793e058640 1 -- 172.21.15.22:0/2449039344 >> 172.21.15.22:0/2449039344 conn(0x7f79380b6410 msgr2=0x7f79380d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:22.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.344+0000 7f793e058640 1 -- 172.21.15.22:0/2449039344 shutdown_connections 2024-03-29T04:18:22.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.344+0000 7f793e058640 1 -- 172.21.15.22:0/2449039344 wait complete. 2024-03-29T04:18:22.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.344+0000 7f793e058640 1 Processor -- start 2024-03-29T04:18:22.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.344+0000 7f793e058640 1 -- start start 2024-03-29T04:18:22.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.344+0000 7f793e058640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79380b9e30 0x7f7938154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.344+0000 7f793e058640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79380cf4f0 0x7f79381608d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.344+0000 7f793e058640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f793815e4c0 0x7f793815ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.344+0000 7f793e058640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f793815e8a0 con 0x7f793815e4c0 2024-03-29T04:18:22.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.344+0000 7f793e058640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f79380d3f20 con 0x7f79380cf4f0 2024-03-29T04:18:22.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.344+0000 7f793e058640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f79380d40a0 con 0x7f79380b9e30 2024-03-29T04:18:22.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.348+0000 7f7936ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79380cf4f0 0x7f79381608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:22.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.348+0000 7f7937fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f793815e4c0 0x7f793815ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:22.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.348+0000 7f7936ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79380cf4f0 0x7f79381608d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:50536/0 (socket says 172.21.15.22:50536) 2024-03-29T04:18:22.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.348+0000 7f7936ffd640 1 -- 172.21.15.22:0/600810677 learned_addr learned my addr 172.21.15.22:0/600810677 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:22.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.348+0000 7f79377fe640 1 --2- 172.21.15.22:0/600810677 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79380b9e30 0x7f7938154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:22.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.348+0000 7f7937fff640 1 -- 172.21.15.22:0/600810677 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79380b9e30 msgr2=0x7f7938154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:22.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.348+0000 7f7937fff640 1 --2- 172.21.15.22:0/600810677 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79380b9e30 0x7f7938154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.348+0000 7f7937fff640 1 -- 172.21.15.22:0/600810677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79380cf4f0 msgr2=0x7f79381608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:22.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.348+0000 7f7937fff640 1 --2- 172.21.15.22:0/600810677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79380cf4f0 0x7f79381608d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.348+0000 7f7937fff640 1 -- 172.21.15.22:0/600810677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7938153e80 con 0x7f793815e4c0 2024-03-29T04:18:22.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.348+0000 7f7937fff640 1 --2- 172.21.15.22:0/600810677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f793815e4c0 0x7f793815ff20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f793000c700 tx=0x7f793000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:22.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.348+0000 7f791ffff640 1 -- 172.21.15.22:0/600810677 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930003b80 con 0x7f793815e4c0 2024-03-29T04:18:22.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.348+0000 7f791ffff640 1 -- 172.21.15.22:0/600810677 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7930003d20 con 0x7f793815e4c0 2024-03-29T04:18:22.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.348+0000 7f793e058640 1 -- 172.21.15.22:0/600810677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7938154160 con 0x7f793815e4c0 2024-03-29T04:18:22.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.348+0000 7f791ffff640 1 -- 172.21.15.22:0/600810677 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793000aa30 con 0x7f793815e4c0 2024-03-29T04:18:22.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.348+0000 7f793e058640 1 -- 172.21.15.22:0/600810677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7938154a00 con 0x7f793815e4c0 2024-03-29T04:18:22.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.348+0000 7f793e058640 1 -- 172.21.15.22:0/600810677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7938068900 con 0x7f793815e4c0 2024-03-29T04:18:22.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.348+0000 7f791ffff640 1 -- 172.21.15.22:0/600810677 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f793000abd0 con 0x7f793815e4c0 2024-03-29T04:18:22.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.348+0000 7f791ffff640 1 --2- 172.21.15.22:0/600810677 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7924067ff0 0x7f792406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.348+0000 7f791ffff640 1 -- 172.21.15.22:0/600810677 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f79300868f0 con 0x7f793815e4c0 2024-03-29T04:18:22.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.348+0000 7f79377fe640 1 --2- 172.21.15.22:0/600810677 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7924067ff0 0x7f792406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:22.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.352+0000 7f79377fe640 1 --2- 172.21.15.22:0/600810677 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7924067ff0 0x7f792406a4b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f792c000f70 tx=0x7f792c007240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:22.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.352+0000 7f791ffff640 1 -- 172.21.15.22:0/600810677 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f7930056630 con 0x7f793815e4c0 2024-03-29T04:18:22.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.516+0000 7f793e058640 1 -- 172.21.15.22:0/600810677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7938154bd0 con 0x7f793815e4c0 2024-03-29T04:18:22.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.516+0000 7f791ffff640 1 -- 172.21.15.22:0/600810677 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f793005a930 con 0x7f793815e4c0 2024-03-29T04:18:22.497 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:22.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.516+0000 7f793e058640 1 -- 172.21.15.22:0/600810677 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7924067ff0 msgr2=0x7f792406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:22.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.516+0000 7f793e058640 1 --2- 172.21.15.22:0/600810677 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7924067ff0 0x7f792406a4b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f792c000f70 tx=0x7f792c007240 comp rx=0 tx=0).stop 2024-03-29T04:18:22.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.516+0000 7f793e058640 1 -- 172.21.15.22:0/600810677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f793815e4c0 msgr2=0x7f793815ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:22.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.516+0000 7f793e058640 1 --2- 172.21.15.22:0/600810677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f793815e4c0 0x7f793815ff20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f793000c700 tx=0x7f793000cbc0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.516+0000 7f793e058640 1 -- 172.21.15.22:0/600810677 shutdown_connections 2024-03-29T04:18:22.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.516+0000 7f793e058640 1 --2- 172.21.15.22:0/600810677 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7924067ff0 0x7f792406a4b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.516+0000 7f793e058640 1 --2- 172.21.15.22:0/600810677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f793815e4c0 0x7f793815ff20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.516+0000 7f793e058640 1 --2- 172.21.15.22:0/600810677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79380cf4f0 0x7f79381608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.516+0000 7f793e058640 1 --2- 172.21.15.22:0/600810677 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79380b9e30 0x7f7938154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.516+0000 7f793e058640 1 -- 172.21.15.22:0/600810677 >> 172.21.15.22:0/600810677 conn(0x7f79380b6410 msgr2=0x7f79381523a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:22.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.516+0000 7f793e058640 1 -- 172.21.15.22:0/600810677 shutdown_connections 2024-03-29T04:18:22.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.516+0000 7f793e058640 1 -- 172.21.15.22:0/600810677 wait complete. 2024-03-29T04:18:22.517 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:22.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.656+0000 7f27496c9640 1 Processor -- start 2024-03-29T04:18:22.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.656+0000 7f27496c9640 1 -- start start 2024-03-29T04:18:22.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.656+0000 7f27496c9640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2744144270 0x7f2744144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.656+0000 7f27496c9640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f27440b2ef0 0x7f274415f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.656+0000 7f27496c9640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f27440b4490 0x7f274406c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.656+0000 7f27496c9640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f2744144fa0 con 0x7f2744144270 2024-03-29T04:18:22.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.656+0000 7f27496c9640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f274415fce0 con 0x7f27440b2ef0 2024-03-29T04:18:22.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.656+0000 7f27496c9640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f27440b33f0 con 0x7f27440b4490 2024-03-29T04:18:22.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.656+0000 7f2742ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2744144270 0x7f2744144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:22.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.656+0000 7f2742ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2744144270 0x7f2744144650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59570/0 (socket says 172.21.15.22:59570) 2024-03-29T04:18:22.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.656+0000 7f2742ffd640 1 -- 172.21.15.22:0/1004125430 learned_addr learned my addr 172.21.15.22:0/1004125430 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:22.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.656+0000 7f27427fc640 1 --2- 172.21.15.22:0/1004125430 >> v2:172.21.15.151:3300/0 conn(0x7f27440b2ef0 0x7f274415f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:22.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.656+0000 7f2742ffd640 1 -- 172.21.15.22:0/1004125430 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f27440b4490 msgr2=0x7f274406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:22.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.656+0000 7f2742ffd640 1 --2- 172.21.15.22:0/1004125430 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f27440b4490 0x7f274406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.656+0000 7f2742ffd640 1 -- 172.21.15.22:0/1004125430 >> v2:172.21.15.151:3300/0 conn(0x7f27440b2ef0 msgr2=0x7f274415f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:22.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.656+0000 7f2742ffd640 1 --2- 172.21.15.22:0/1004125430 >> v2:172.21.15.151:3300/0 conn(0x7f27440b2ef0 0x7f274415f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.656+0000 7f2742ffd640 1 -- 172.21.15.22:0/1004125430 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27440b36d0 con 0x7f2744144270 2024-03-29T04:18:22.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.660+0000 7f2742ffd640 1 --2- 172.21.15.22:0/1004125430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2744144270 0x7f2744144650 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f27440b4e70 tx=0x7f273401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a518ab3b5b5c9b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:22.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.660+0000 7f2741ffb640 1 -- 172.21.15.22:0/1004125430 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27340aa610 con 0x7f2744144270 2024-03-29T04:18:22.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.660+0000 7f2741ffb640 1 -- 172.21.15.22:0/1004125430 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27340af070 con 0x7f2744144270 2024-03-29T04:18:22.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.660+0000 7f2741ffb640 1 -- 172.21.15.22:0/1004125430 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27340aa9d0 con 0x7f2744144270 2024-03-29T04:18:22.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.660+0000 7f27496c9640 1 -- 172.21.15.22:0/1004125430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2744144270 msgr2=0x7f2744144650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:22.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.660+0000 7f27496c9640 1 --2- 172.21.15.22:0/1004125430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2744144270 0x7f2744144650 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f27440b4e70 tx=0x7f273401c4f0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.660+0000 7f27496c9640 1 -- 172.21.15.22:0/1004125430 shutdown_connections 2024-03-29T04:18:22.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.660+0000 7f27496c9640 1 --2- 172.21.15.22:0/1004125430 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f27440b4490 0x7f274406c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.660+0000 7f27496c9640 1 --2- 172.21.15.22:0/1004125430 >> v2:172.21.15.151:3300/0 conn(0x7f27440b2ef0 0x7f274415f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.660+0000 7f27496c9640 1 --2- 172.21.15.22:0/1004125430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2744144270 0x7f2744144650 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.660+0000 7f27496c9640 1 -- 172.21.15.22:0/1004125430 >> 172.21.15.22:0/1004125430 conn(0x7f27440cfee0 msgr2=0x7f27440d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:22.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.660+0000 7f27496c9640 1 -- 172.21.15.22:0/1004125430 shutdown_connections 2024-03-29T04:18:22.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.660+0000 7f27496c9640 1 -- 172.21.15.22:0/1004125430 wait complete. 2024-03-29T04:18:22.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.660+0000 7f27496c9640 1 Processor -- start 2024-03-29T04:18:22.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.660+0000 7f27496c9640 1 -- start start 2024-03-29T04:18:22.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.660+0000 7f27496c9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27440b2ef0 0x7f274415a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.660+0000 7f27496c9640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f27440b4490 0x7f2744137ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.660+0000 7f27496c9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2744144270 0x7f2744137ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.660+0000 7f27496c9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f27440cdd10 con 0x7f27440b2ef0 2024-03-29T04:18:22.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.660+0000 7f27496c9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f274406cfc0 con 0x7f2744144270 2024-03-29T04:18:22.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.660+0000 7f27496c9640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f27440cde90 con 0x7f27440b4490 2024-03-29T04:18:22.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f2742ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27440b2ef0 0x7f274415a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:22.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f2742ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27440b2ef0 0x7f274415a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59582/0 (socket says 172.21.15.22:59582) 2024-03-29T04:18:22.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f2742ffd640 1 -- 172.21.15.22:0/1032728698 learned_addr learned my addr 172.21.15.22:0/1032728698 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:22.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f27427fc640 1 --2- 172.21.15.22:0/1032728698 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f27440b4490 0x7f2744137ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:22.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f27437fe640 1 --2- 172.21.15.22:0/1032728698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2744144270 0x7f2744137ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:22.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f2742ffd640 1 -- 172.21.15.22:0/1032728698 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f27440b4490 msgr2=0x7f2744137ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:22.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f2742ffd640 1 --2- 172.21.15.22:0/1032728698 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f27440b4490 0x7f2744137ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f2742ffd640 1 -- 172.21.15.22:0/1032728698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2744144270 msgr2=0x7f2744137ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:22.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f2742ffd640 1 --2- 172.21.15.22:0/1032728698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2744144270 0x7f2744137ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f2742ffd640 1 -- 172.21.15.22:0/1032728698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2744139420 con 0x7f27440b2ef0 2024-03-29T04:18:22.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f27437fe640 1 --2- 172.21.15.22:0/1032728698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2744144270 0x7f2744137ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:22.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f27427fc640 1 --2- 172.21.15.22:0/1032728698 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f27440b4490 0x7f2744137ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:22.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f2742ffd640 1 --2- 172.21.15.22:0/1032728698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27440b2ef0 0x7f274415a900 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f273401cd90 tx=0x7f273401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:22.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f272b7fe640 1 -- 172.21.15.22:0/1032728698 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27340bd5f0 con 0x7f27440b2ef0 2024-03-29T04:18:22.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f272b7fe640 1 -- 172.21.15.22:0/1032728698 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27340bd7b0 con 0x7f27440b2ef0 2024-03-29T04:18:22.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f272b7fe640 1 -- 172.21.15.22:0/1032728698 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27340bdba0 con 0x7f27440b2ef0 2024-03-29T04:18:22.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f27496c9640 1 -- 172.21.15.22:0/1032728698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2744139700 con 0x7f27440b2ef0 2024-03-29T04:18:22.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f27496c9640 1 -- 172.21.15.22:0/1032728698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2744139cc0 con 0x7f27440b2ef0 2024-03-29T04:18:22.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f27496c9640 1 -- 172.21.15.22:0/1032728698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2744068890 con 0x7f27440b2ef0 2024-03-29T04:18:22.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f272b7fe640 1 -- 172.21.15.22:0/1032728698 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f27340b33f0 con 0x7f27440b2ef0 2024-03-29T04:18:22.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f272b7fe640 1 --2- 172.21.15.22:0/1032728698 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2730067fc0 0x7f273006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f272b7fe640 1 -- 172.21.15.22:0/1032728698 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f2734120a60 con 0x7f27440b2ef0 2024-03-29T04:18:22.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f27427fc640 1 --2- 172.21.15.22:0/1032728698 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2730067fc0 0x7f273006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:22.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.664+0000 7f27427fc640 1 --2- 172.21.15.22:0/1032728698 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2730067fc0 0x7f273006a480 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2744135380 tx=0x7f2738008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.668+0000 7f272b7fe640 1 -- 172.21.15.22:0/1032728698 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f27340a7070 con 0x7f27440b2ef0 2024-03-29T04:18:22.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.824+0000 7f27496c9640 1 -- 172.21.15.22:0/1032728698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27440722d0 con 0x7f27440b2ef0 2024-03-29T04:18:22.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.828+0000 7f272b7fe640 1 -- 172.21.15.22:0/1032728698 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18847 (secure 0 0 0) 0x7f2734016110 con 0x7f27440b2ef0 2024-03-29T04:18:22.810 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:22.810 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:18:16.893439+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.832+0000 7f27496c9640 1 -- 172.21.15.22:0/1032728698 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2730067fc0 msgr2=0x7f273006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.832+0000 7f27496c9640 1 --2- 172.21.15.22:0/1032728698 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2730067fc0 0x7f273006a480 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2744135380 tx=0x7f2738008040 comp rx=0 tx=0).stop 2024-03-29T04:18:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.832+0000 7f27496c9640 1 -- 172.21.15.22:0/1032728698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27440b2ef0 msgr2=0x7f274415a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.832+0000 7f27496c9640 1 --2- 172.21.15.22:0/1032728698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27440b2ef0 0x7f274415a900 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f273401cd90 tx=0x7f273401ad30 comp rx=0 tx=0).stop 2024-03-29T04:18:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.832+0000 7f27496c9640 1 -- 172.21.15.22:0/1032728698 shutdown_connections 2024-03-29T04:18:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.832+0000 7f27496c9640 1 --2- 172.21.15.22:0/1032728698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2744144270 0x7f2744137ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.832+0000 7f27496c9640 1 --2- 172.21.15.22:0/1032728698 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f27440b4490 0x7f2744137ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.832+0000 7f27496c9640 1 --2- 172.21.15.22:0/1032728698 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2730067fc0 0x7f273006a480 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.832+0000 7f27496c9640 1 --2- 172.21.15.22:0/1032728698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27440b2ef0 0x7f274415a900 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.832+0000 7f27496c9640 1 -- 172.21.15.22:0/1032728698 >> 172.21.15.22:0/1032728698 conn(0x7f27440cfee0 msgr2=0x7f274413f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.832+0000 7f27496c9640 1 -- 172.21.15.22:0/1032728698 shutdown_connections 2024-03-29T04:18:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.832+0000 7f27496c9640 1 -- 172.21.15.22:0/1032728698 wait complete. 2024-03-29T04:18:22.828 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:17:34.034142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:17:34.886659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:17:38.628632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:18:22.828 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:18:22.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.964+0000 7f84b211d640 1 Processor -- start 2024-03-29T04:18:22.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.964+0000 7f84b211d640 1 -- start start 2024-03-29T04:18:22.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.968+0000 7f84b211d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f84ac0cf870 0x7f84ac15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.968+0000 7f84b211d640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84ac0b4090 0x7f84ac0b4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.968+0000 7f84b211d640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84ac071910 con 0x7f84ac0726d0 2024-03-29T04:18:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.968+0000 7f84b211d640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f84ac0b4a40 con 0x7f84ac0cf870 2024-03-29T04:18:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.968+0000 7f84b211d640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f84ac071bf0 con 0x7f84ac0b4090 2024-03-29T04:18:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.968+0000 7f84ab7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f84ac0cf870 0x7f84ac15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.968+0000 7f84aaffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f84ac0726d0 0x7f84ac0b3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58012/0 (socket says 172.21.15.22:58012) 2024-03-29T04:18:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.968+0000 7f84aaffd640 1 -- 172.21.15.22:0/2077600004 learned_addr learned my addr 172.21.15.22:0/2077600004 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.968+0000 7f84abfff640 1 --2- 172.21.15.22:0/2077600004 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84ac0b4090 0x7f84ac0b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.968+0000 7f84ab7fe640 1 -- 172.21.15.22:0/2077600004 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84ac0b4090 msgr2=0x7f84ac0b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.968+0000 7f84ab7fe640 1 --2- 172.21.15.22:0/2077600004 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84ac0b4090 0x7f84ac0b4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.968+0000 7f84ab7fe640 1 -- 172.21.15.22:0/2077600004 >> v1:172.21.15.22:6789/0 conn(0x7f84ac0726d0 legacy=0x7f84ac0b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.968+0000 7f84aa7fc640 1 -- 172.21.15.22:0/2077600004 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1965407855 0 0) 0x7f84ac071910 con 0x7f84ac0726d0 2024-03-29T04:18:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.968+0000 7f84ab7fe640 1 -- 172.21.15.22:0/2077600004 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ac071ed0 con 0x7f84ac0cf870 2024-03-29T04:18:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.968+0000 7f84abfff640 1 --2- 172.21.15.22:0/2077600004 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84ac0b4090 0x7f84ac0b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:22.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.968+0000 7f84ab7fe640 1 --2- 172.21.15.22:0/2077600004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84ac0cf870 0x7f84ac15baa0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f849c0097c0 tx=0x7f849c01ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d671d7c9be582ad9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:22.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.968+0000 7f84aa7fc640 1 -- 172.21.15.22:0/2077600004 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c0aa5f0 con 0x7f84ac0cf870 2024-03-29T04:18:22.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.968+0000 7f84aa7fc640 1 -- 172.21.15.22:0/2077600004 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f849c0af070 con 0x7f84ac0cf870 2024-03-29T04:18:22.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.968+0000 7f84aa7fc640 1 -- 172.21.15.22:0/2077600004 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c0aa9b0 con 0x7f84ac0cf870 2024-03-29T04:18:22.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84b211d640 1 -- 172.21.15.22:0/2077600004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84ac0cf870 msgr2=0x7f84ac15baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:22.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84b211d640 1 --2- 172.21.15.22:0/2077600004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84ac0cf870 0x7f84ac15baa0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f849c0097c0 tx=0x7f849c01ced0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84b211d640 1 -- 172.21.15.22:0/2077600004 shutdown_connections 2024-03-29T04:18:22.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84b211d640 1 --2- 172.21.15.22:0/2077600004 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84ac0b4090 0x7f84ac0b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84b211d640 1 --2- 172.21.15.22:0/2077600004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84ac0cf870 0x7f84ac15baa0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84b211d640 1 -- 172.21.15.22:0/2077600004 >> 172.21.15.22:0/2077600004 conn(0x7f84ac0d00a0 msgr2=0x7f84ac0d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:22.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84b211d640 1 -- 172.21.15.22:0/2077600004 shutdown_connections 2024-03-29T04:18:22.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84b211d640 1 -- 172.21.15.22:0/2077600004 wait complete. 2024-03-29T04:18:22.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84b211d640 1 Processor -- start 2024-03-29T04:18:22.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84b211d640 1 -- start start 2024-03-29T04:18:22.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84b211d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84ac0726d0 0x7f84ac1492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84b211d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84ac0b4090 0x7f84ac0ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84b211d640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84ac0cf870 0x7f84ac0afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84b211d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f84ac06e000 con 0x7f84ac0b4090 2024-03-29T04:18:22.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84b211d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f84ac06de80 con 0x7f84ac0726d0 2024-03-29T04:18:22.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84b211d640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f84ac06e180 con 0x7f84ac0cf870 2024-03-29T04:18:22.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84aaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84ac0b4090 0x7f84ac0ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:22.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84aaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84ac0b4090 0x7f84ac0ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59610/0 (socket says 172.21.15.22:59610) 2024-03-29T04:18:22.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84aaffd640 1 -- 172.21.15.22:0/1796043254 learned_addr learned my addr 172.21.15.22:0/1796043254 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:22.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84abfff640 1 --2- 172.21.15.22:0/1796043254 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84ac0cf870 0x7f84ac0afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:22.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84ab7fe640 1 --2- 172.21.15.22:0/1796043254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84ac0726d0 0x7f84ac1492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:22.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84aaffd640 1 -- 172.21.15.22:0/1796043254 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84ac0cf870 msgr2=0x7f84ac0afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:22.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84aaffd640 1 --2- 172.21.15.22:0/1796043254 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84ac0cf870 0x7f84ac0afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84aaffd640 1 -- 172.21.15.22:0/1796043254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84ac0726d0 msgr2=0x7f84ac1492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:22.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84aaffd640 1 --2- 172.21.15.22:0/1796043254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84ac0726d0 0x7f84ac1492a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:22.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84aaffd640 1 -- 172.21.15.22:0/1796043254 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ac1556c0 con 0x7f84ac0b4090 2024-03-29T04:18:22.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84ab7fe640 1 --2- 172.21.15.22:0/1796043254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84ac0726d0 0x7f84ac1492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:22.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84aaffd640 1 --2- 172.21.15.22:0/1796043254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84ac0b4090 0x7f84ac0ce210 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f84a000c550 tx=0x7f84a000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:22.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f8493fff640 1 -- 172.21.15.22:0/1796043254 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a0017070 con 0x7f84ac0b4090 2024-03-29T04:18:22.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f8493fff640 1 -- 172.21.15.22:0/1796043254 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84a0005d10 con 0x7f84ac0b4090 2024-03-29T04:18:22.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84b211d640 1 -- 172.21.15.22:0/1796043254 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84ac1559a0 con 0x7f84ac0b4090 2024-03-29T04:18:22.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f8493fff640 1 -- 172.21.15.22:0/1796043254 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a0007330 con 0x7f84ac0b4090 2024-03-29T04:18:22.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84b211d640 1 -- 172.21.15.22:0/1796043254 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84ac155f60 con 0x7f84ac0b4090 2024-03-29T04:18:22.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.972+0000 7f84b211d640 1 -- 172.21.15.22:0/1796043254 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84ac068900 con 0x7f84ac0b4090 2024-03-29T04:18:22.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.976+0000 7f8493fff640 1 -- 172.21.15.22:0/1796043254 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f84a001a070 con 0x7f84ac0b4090 2024-03-29T04:18:22.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.976+0000 7f8493fff640 1 --2- 172.21.15.22:0/1796043254 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8498067ff0 0x7f849806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:22.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.976+0000 7f8493fff640 1 -- 172.21.15.22:0/1796043254 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f84a0086050 con 0x7f84ac0b4090 2024-03-29T04:18:22.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.976+0000 7f84ab7fe640 1 --2- 172.21.15.22:0/1796043254 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8498067ff0 0x7f849806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:22.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.976+0000 7f84ab7fe640 1 --2- 172.21.15.22:0/1796043254 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8498067ff0 0x7f849806a4b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f849c0ae000 tx=0x7f849c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:22.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:22.980+0000 7f8493fff640 1 -- 172.21.15.22:0/1796043254 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f84a0021020 con 0x7f84ac0b4090 2024-03-29T04:18:23.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.140+0000 7f84b211d640 1 -- 172.21.15.22:0/1796043254 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f84ac0ce750 con 0x7f84ac0b4090 2024-03-29T04:18:23.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.140+0000 7f8493fff640 1 -- 172.21.15.22:0/1796043254 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f84a0056570 con 0x7f84ac0b4090 2024-03-29T04:18:23.121 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:18:23.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.140+0000 7f84b211d640 1 -- 172.21.15.22:0/1796043254 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8498067ff0 msgr2=0x7f849806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:23.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.140+0000 7f84b211d640 1 --2- 172.21.15.22:0/1796043254 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8498067ff0 0x7f849806a4b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f849c0ae000 tx=0x7f849c008040 comp rx=0 tx=0).stop 2024-03-29T04:18:23.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.140+0000 7f84b211d640 1 -- 172.21.15.22:0/1796043254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84ac0b4090 msgr2=0x7f84ac0ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:23.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.140+0000 7f84b211d640 1 --2- 172.21.15.22:0/1796043254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84ac0b4090 0x7f84ac0ce210 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f84a000c550 tx=0x7f84a000ca10 comp rx=0 tx=0).stop 2024-03-29T04:18:23.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.140+0000 7f84b211d640 1 -- 172.21.15.22:0/1796043254 shutdown_connections 2024-03-29T04:18:23.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.140+0000 7f84b211d640 1 --2- 172.21.15.22:0/1796043254 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f84ac0cf870 0x7f84ac0afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.140+0000 7f84b211d640 1 --2- 172.21.15.22:0/1796043254 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8498067ff0 0x7f849806a4b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.140+0000 7f84b211d640 1 --2- 172.21.15.22:0/1796043254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84ac0b4090 0x7f84ac0ce210 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.140+0000 7f84b211d640 1 --2- 172.21.15.22:0/1796043254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84ac0726d0 0x7f84ac1492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.140+0000 7f84b211d640 1 -- 172.21.15.22:0/1796043254 >> 172.21.15.22:0/1796043254 conn(0x7f84ac0d00a0 msgr2=0x7f84ac0b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:23.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.144+0000 7f84b211d640 1 -- 172.21.15.22:0/1796043254 shutdown_connections 2024-03-29T04:18:23.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.144+0000 7f84b211d640 1 -- 172.21.15.22:0/1796043254 wait complete. 2024-03-29T04:18:23.139 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:18:23.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.284+0000 7fcdc7bb6640 1 Processor -- start 2024-03-29T04:18:23.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.284+0000 7fcdc7bb6640 1 -- start start 2024-03-29T04:18:23.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc7bb6640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fcdc00cf6c0 0x7fcdc0169030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc7bb6640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdc006c850 con 0x7fcdc015e4c0 2024-03-29T04:18:23.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc7bb6640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdc006cb30 con 0x7fcdc00b9e30 2024-03-29T04:18:23.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc7bb6640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fcdc00ba290 con 0x7fcdc00cf6c0 2024-03-29T04:18:23.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc5952640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fcdc015e4c0 0x7fcdc006c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58016/0 (socket says 172.21.15.22:58016) 2024-03-29T04:18:23.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc5952640 1 -- 172.21.15.22:0/3109905351 learned_addr learned my addr 172.21.15.22:0/3109905351 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:23.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc6153640 1 --2- 172.21.15.22:0/3109905351 >> v2:172.21.15.151:3300/0 conn(0x7fcdc00cf6c0 0x7fcdc0169030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc4950640 1 -- 172.21.15.22:0/3109905351 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2192900752 0 0) 0x7fcdc006c850 con 0x7fcdc015e4c0 2024-03-29T04:18:23.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc4950640 1 -- 172.21.15.22:0/3109905351 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcdac003530 con 0x7fcdc015e4c0 2024-03-29T04:18:23.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc4950640 1 -- 172.21.15.22:0/3109905351 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1142835625 0 0) 0x7fcdc006cb30 con 0x7fcdc00b9e30 2024-03-29T04:18:23.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc4950640 1 -- 172.21.15.22:0/3109905351 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcdc006c850 con 0x7fcdc00b9e30 2024-03-29T04:18:23.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc4950640 1 -- 172.21.15.22:0/3109905351 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1668154140 0 0) 0x7fcdac003530 con 0x7fcdc015e4c0 2024-03-29T04:18:23.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc4950640 1 -- 172.21.15.22:0/3109905351 >> v2:172.21.15.151:3300/0 conn(0x7fcdc00cf6c0 msgr2=0x7fcdc0169030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:23.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc4950640 1 --2- 172.21.15.22:0/3109905351 >> v2:172.21.15.151:3300/0 conn(0x7fcdc00cf6c0 0x7fcdc0169030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc4950640 1 -- 172.21.15.22:0/3109905351 >> v1:172.21.15.151:6789/0 conn(0x7fcdc00b9e30 legacy=0x7fcdc00cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:23.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc4950640 1 -- 172.21.15.22:0/3109905351 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc006ce10 con 0x7fcdc015e4c0 2024-03-29T04:18:23.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc4950640 1 -- 172.21.15.22:0/3109905351 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7fcdb8001a00 con 0x7fcdc015e4c0 2024-03-29T04:18:23.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc4950640 1 -- 172.21.15.22:0/3109905351 >> v1:172.21.15.22:6789/0 conn(0x7fcdc015e4c0 legacy=0x7fcdc006c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:23.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc4950640 1 --2- 172.21.15.22:0/3109905351 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcdac003ba0 0x7fcdac003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc4950640 1 --2- 172.21.15.22:0/3109905351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac004810 0x7fcdac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc4950640 1 --2- 172.21.15.22:0/3109905351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdac0051d0 0x7fcdac00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc4950640 1 -- 172.21.15.22:0/3109905351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcdac00c0a0 con 0x7fcdac004810 2024-03-29T04:18:23.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc4950640 1 -- 172.21.15.22:0/3109905351 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcdac00c220 con 0x7fcdac0051d0 2024-03-29T04:18:23.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc4950640 1 -- 172.21.15.22:0/3109905351 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fcdac00c420 con 0x7fcdac003ba0 2024-03-29T04:18:23.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc5151640 1 --2- 172.21.15.22:0/3109905351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac004810 0x7fcdac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc5952640 1 --2- 172.21.15.22:0/3109905351 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcdac003ba0 0x7fcdac003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc6153640 1 --2- 172.21.15.22:0/3109905351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdac0051d0 0x7fcdac00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc5151640 1 -- 172.21.15.22:0/3109905351 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcdac003ba0 msgr2=0x7fcdac003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:23.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc5151640 1 --2- 172.21.15.22:0/3109905351 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcdac003ba0 0x7fcdac003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc5151640 1 -- 172.21.15.22:0/3109905351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdac0051d0 msgr2=0x7fcdac00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:23.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc5151640 1 --2- 172.21.15.22:0/3109905351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdac0051d0 0x7fcdac00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc5151640 1 -- 172.21.15.22:0/3109905351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdac003530 con 0x7fcdac004810 2024-03-29T04:18:23.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc6153640 1 -- 172.21.15.22:0/3109905351 reap_dead start 2024-03-29T04:18:23.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.288+0000 7fcdc5952640 1 --2- 172.21.15.22:0/3109905351 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcdac003ba0 0x7fcdac003fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc5151640 1 --2- 172.21.15.22:0/3109905351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac004810 0x7fcdac004c90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00018d0 tx=0x7fcdb001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf6aaa4d8a3984f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc4950640 1 -- 172.21.15.22:0/3109905351 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0001520 con 0x7fcdac004810 2024-03-29T04:18:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc4950640 1 -- 172.21.15.22:0/3109905351 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdb00052f0 con 0x7fcdac004810 2024-03-29T04:18:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc4950640 1 -- 172.21.15.22:0/3109905351 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb00a88f0 con 0x7fcdac004810 2024-03-29T04:18:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc7bb6640 1 -- 172.21.15.22:0/3109905351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac004810 msgr2=0x7fcdac004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc7bb6640 1 --2- 172.21.15.22:0/3109905351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac004810 0x7fcdac004c90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00018d0 tx=0x7fcdb001ec90 comp rx=0 tx=0).stop 2024-03-29T04:18:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc7bb6640 1 -- 172.21.15.22:0/3109905351 shutdown_connections 2024-03-29T04:18:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc7bb6640 1 --2- 172.21.15.22:0/3109905351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac004810 0x7fcdac004c90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc7bb6640 1 -- 172.21.15.22:0/3109905351 >> 172.21.15.22:0/3109905351 conn(0x7fcdc00b6410 msgr2=0x7fcdc00d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc7bb6640 1 -- 172.21.15.22:0/3109905351 shutdown_connections 2024-03-29T04:18:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc7bb6640 1 -- 172.21.15.22:0/3109905351 wait complete. 2024-03-29T04:18:23.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc7bb6640 1 Processor -- start 2024-03-29T04:18:23.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc7bb6640 1 -- start start 2024-03-29T04:18:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc7bb6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac004810 0x7fcdc015ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc7bb6640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcdc015c1e0 0x7fcdc0157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc7bb6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdc015c620 0x7fcdc01578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc7bb6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc00cf340 con 0x7fcdac004810 2024-03-29T04:18:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc7bb6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc00cf1c0 con 0x7fcdc015c620 2024-03-29T04:18:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc7bb6640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fcdc00cf4c0 con 0x7fcdc015c1e0 2024-03-29T04:18:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc5952640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac004810 0x7fcdc015ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc5952640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac004810 0x7fcdc015ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59628/0 (socket says 172.21.15.22:59628) 2024-03-29T04:18:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.292+0000 7fcdc5952640 1 -- 172.21.15.22:0/1581024269 learned_addr learned my addr 172.21.15.22:0/1581024269 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.296+0000 7fcdc6153640 1 --2- 172.21.15.22:0/1581024269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdc015c620 0x7fcdc01578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.296+0000 7fcdc5151640 1 --2- 172.21.15.22:0/1581024269 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcdc015c1e0 0x7fcdc0157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.296+0000 7fcdc5952640 1 -- 172.21.15.22:0/1581024269 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcdc015c1e0 msgr2=0x7fcdc0157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:23.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.296+0000 7fcdc5952640 1 --2- 172.21.15.22:0/1581024269 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcdc015c1e0 0x7fcdc0157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.296+0000 7fcdc5952640 1 -- 172.21.15.22:0/1581024269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdc015c620 msgr2=0x7fcdc01578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:23.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.296+0000 7fcdc5952640 1 --2- 172.21.15.22:0/1581024269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdc015c620 0x7fcdc01578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.296+0000 7fcdc5952640 1 -- 172.21.15.22:0/1581024269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc015f850 con 0x7fcdac004810 2024-03-29T04:18:23.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.296+0000 7fcdc5952640 1 --2- 172.21.15.22:0/1581024269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac004810 0x7fcdc015ff10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8007a10 tx=0x7fcdb800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:23.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.296+0000 7fcdb67fc640 1 -- 172.21.15.22:0/1581024269 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb80019e0 con 0x7fcdac004810 2024-03-29T04:18:23.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.296+0000 7fcdb67fc640 1 -- 172.21.15.22:0/1581024269 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdb8018070 con 0x7fcdac004810 2024-03-29T04:18:23.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.296+0000 7fcdc7bb6640 1 -- 172.21.15.22:0/1581024269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc015f570 con 0x7fcdac004810 2024-03-29T04:18:23.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.296+0000 7fcdb67fc640 1 -- 172.21.15.22:0/1581024269 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8013620 con 0x7fcdac004810 2024-03-29T04:18:23.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.296+0000 7fcdc7bb6640 1 -- 172.21.15.22:0/1581024269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc00cba60 con 0x7fcdac004810 2024-03-29T04:18:23.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.296+0000 7fcdc7bb6640 1 -- 172.21.15.22:0/1581024269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdc0068900 con 0x7fcdac004810 2024-03-29T04:18:23.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.296+0000 7fcdb67fc640 1 -- 172.21.15.22:0/1581024269 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fcdb80137c0 con 0x7fcdac004810 2024-03-29T04:18:23.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.296+0000 7fcdb67fc640 1 --2- 172.21.15.22:0/1581024269 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcda4067fc0 0x7fcda406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.296+0000 7fcdb67fc640 1 -- 172.21.15.22:0/1581024269 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fcdb8086e50 con 0x7fcdac004810 2024-03-29T04:18:23.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.296+0000 7fcdc5151640 1 --2- 172.21.15.22:0/1581024269 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcda4067fc0 0x7fcda406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.296+0000 7fcdc5151640 1 --2- 172.21.15.22:0/1581024269 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcda4067fc0 0x7fcda406a480 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00b0000 tx=0x7fcdb00aa070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:23.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.300+0000 7fcdb67fc640 1 -- 172.21.15.22:0/1581024269 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fcdb8057200 con 0x7fcdac004810 2024-03-29T04:18:23.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.464+0000 7fcdc7bb6640 1 -- 172.21.15.22:0/1581024269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcdc006c2b0 con 0x7fcdac004810 2024-03-29T04:18:23.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.464+0000 7fcdb67fc640 1 -- 172.21.15.22:0/1581024269 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7fcdb8056b00 con 0x7fcdac004810 2024-03-29T04:18:23.447 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:23.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.468+0000 7fcdc7bb6640 1 -- 172.21.15.22:0/1581024269 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcda4067fc0 msgr2=0x7fcda406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:23.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.468+0000 7fcdc7bb6640 1 --2- 172.21.15.22:0/1581024269 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcda4067fc0 0x7fcda406a480 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00b0000 tx=0x7fcdb00aa070 comp rx=0 tx=0).stop 2024-03-29T04:18:23.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.468+0000 7fcdc7bb6640 1 -- 172.21.15.22:0/1581024269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac004810 msgr2=0x7fcdc015ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:23.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.468+0000 7fcdc7bb6640 1 --2- 172.21.15.22:0/1581024269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac004810 0x7fcdc015ff10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8007a10 tx=0x7fcdb800e530 comp rx=0 tx=0).stop 2024-03-29T04:18:23.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.468+0000 7fcdc7bb6640 1 -- 172.21.15.22:0/1581024269 shutdown_connections 2024-03-29T04:18:23.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.468+0000 7fcdc7bb6640 1 --2- 172.21.15.22:0/1581024269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdc015c620 0x7fcdc01578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.468+0000 7fcdc7bb6640 1 --2- 172.21.15.22:0/1581024269 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcdc015c1e0 0x7fcdc0157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.468+0000 7fcdc7bb6640 1 --2- 172.21.15.22:0/1581024269 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcda4067fc0 0x7fcda406a480 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.468+0000 7fcdc7bb6640 1 --2- 172.21.15.22:0/1581024269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac004810 0x7fcdc015ff10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.468+0000 7fcdc7bb6640 1 -- 172.21.15.22:0/1581024269 >> 172.21.15.22:0/1581024269 conn(0x7fcdc00b6410 msgr2=0x7fcdc0162060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:23.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.468+0000 7fcdc7bb6640 1 -- 172.21.15.22:0/1581024269 shutdown_connections 2024-03-29T04:18:23.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.468+0000 7fcdc7bb6640 1 -- 172.21.15.22:0/1581024269 wait complete. 2024-03-29T04:18:23.466 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:18:23.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.608+0000 7fde3ec2a640 1 Processor -- start 2024-03-29T04:18:23.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.608+0000 7fde3ec2a640 1 -- start start 2024-03-29T04:18:23.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde3ec2a640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde3806c4d0 0x7fde380ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde3ec2a640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde380b5470 con 0x7fde3806c070 2024-03-29T04:18:23.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde3ec2a640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde380b5750 con 0x7fde38147e00 2024-03-29T04:18:23.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde3ec2a640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fde380b5a30 con 0x7fde3806c4d0 2024-03-29T04:18:23.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde37fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fde3806c070 0x7fde380adbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58030/0 (socket says 172.21.15.22:58030) 2024-03-29T04:18:23.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde37fff640 1 -- 172.21.15.22:0/3437804664 learned_addr learned my addr 172.21.15.22:0/3437804664 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:23.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde3d1c7640 1 --2- 172.21.15.22:0/3437804664 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde3806c4d0 0x7fde380ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde377fe640 1 -- 172.21.15.22:0/3437804664 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 412090989 0 0) 0x7fde380b5470 con 0x7fde3806c070 2024-03-29T04:18:23.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde377fe640 1 -- 172.21.15.22:0/3437804664 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde28003530 con 0x7fde3806c070 2024-03-29T04:18:23.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde377fe640 1 -- 172.21.15.22:0/3437804664 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 568089039 0 0) 0x7fde28003530 con 0x7fde3806c070 2024-03-29T04:18:23.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde377fe640 1 -- 172.21.15.22:0/3437804664 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde3806c4d0 msgr2=0x7fde380ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:23.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde377fe640 1 --2- 172.21.15.22:0/3437804664 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde3806c4d0 0x7fde380ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde377fe640 1 -- 172.21.15.22:0/3437804664 >> v1:172.21.15.151:6789/0 conn(0x7fde38147e00 legacy=0x7fde380b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:23.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde377fe640 1 -- 172.21.15.22:0/3437804664 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde380b5d10 con 0x7fde3806c070 2024-03-29T04:18:23.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde377fe640 1 -- 172.21.15.22:0/3437804664 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7fde24001680 con 0x7fde3806c070 2024-03-29T04:18:23.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde377fe640 1 -- 172.21.15.22:0/3437804664 >> v1:172.21.15.22:6789/0 conn(0x7fde3806c070 legacy=0x7fde380adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:23.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde377fe640 1 --2- 172.21.15.22:0/3437804664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde280039f0 0x7fde28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde377fe640 1 --2- 172.21.15.22:0/3437804664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde280043a0 0x7fde28004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde377fe640 1 --2- 172.21.15.22:0/3437804664 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde3806c4d0 0x7fde280050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde377fe640 1 -- 172.21.15.22:0/3437804664 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fde380b5a30 con 0x7fde280043a0 2024-03-29T04:18:23.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde377fe640 1 -- 172.21.15.22:0/3437804664 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde2800b9d0 con 0x7fde280039f0 2024-03-29T04:18:23.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde377fe640 1 -- 172.21.15.22:0/3437804664 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fde2800bbd0 con 0x7fde3806c4d0 2024-03-29T04:18:23.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde37fff640 1 --2- 172.21.15.22:0/3437804664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde280043a0 0x7fde28004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde3c9c6640 1 --2- 172.21.15.22:0/3437804664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde280039f0 0x7fde28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde3d1c7640 1 --2- 172.21.15.22:0/3437804664 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde3806c4d0 0x7fde280050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde37fff640 1 -- 172.21.15.22:0/3437804664 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde3806c4d0 msgr2=0x7fde280050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:23.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde37fff640 1 --2- 172.21.15.22:0/3437804664 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde3806c4d0 0x7fde280050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde37fff640 1 -- 172.21.15.22:0/3437804664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde280039f0 msgr2=0x7fde28003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:23.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde37fff640 1 --2- 172.21.15.22:0/3437804664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde280039f0 0x7fde28003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde37fff640 1 -- 172.21.15.22:0/3437804664 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde28003530 con 0x7fde280043a0 2024-03-29T04:18:23.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.612+0000 7fde3d1c7640 1 --2- 172.21.15.22:0/3437804664 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde3806c4d0 0x7fde280050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:23.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde37fff640 1 --2- 172.21.15.22:0/3437804664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde280043a0 0x7fde28004870 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fde2400c240 tx=0x7fde2401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f41d902fb29856 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde377fe640 1 -- 172.21.15.22:0/3437804664 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde24007d40 con 0x7fde280043a0 2024-03-29T04:18:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde377fe640 1 -- 172.21.15.22:0/3437804664 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde240a9070 con 0x7fde280043a0 2024-03-29T04:18:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde377fe640 1 -- 172.21.15.22:0/3437804664 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde240a5de0 con 0x7fde280043a0 2024-03-29T04:18:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde3ec2a640 1 -- 172.21.15.22:0/3437804664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde280043a0 msgr2=0x7fde28004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde3ec2a640 1 --2- 172.21.15.22:0/3437804664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde280043a0 0x7fde28004870 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fde2400c240 tx=0x7fde2401cc80 comp rx=0 tx=0).stop 2024-03-29T04:18:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde3d1c7640 1 -- 172.21.15.22:0/3437804664 reap_dead start 2024-03-29T04:18:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde3ec2a640 1 -- 172.21.15.22:0/3437804664 shutdown_connections 2024-03-29T04:18:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde3ec2a640 1 -- 172.21.15.22:0/3437804664 >> 172.21.15.22:0/3437804664 conn(0x7fde380d1070 msgr2=0x7fde380d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde3ec2a640 1 -- 172.21.15.22:0/3437804664 shutdown_connections 2024-03-29T04:18:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde3ec2a640 1 -- 172.21.15.22:0/3437804664 wait complete. 2024-03-29T04:18:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde3ec2a640 1 Processor -- start 2024-03-29T04:18:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde3ec2a640 1 -- start start 2024-03-29T04:18:23.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde3ec2a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde38160e40 0x7fde380b2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde3ec2a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde38161220 0x7fde38143f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde3ec2a640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde380d0c80 0x7fde3813fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde3ec2a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fde3806b880 con 0x7fde38161220 2024-03-29T04:18:23.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde3ec2a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde380b5100 con 0x7fde38160e40 2024-03-29T04:18:23.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde3ec2a640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fde380ae2d0 con 0x7fde380d0c80 2024-03-29T04:18:23.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde37fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde38161220 0x7fde38143f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde37fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde38161220 0x7fde38143f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59646/0 (socket says 172.21.15.22:59646) 2024-03-29T04:18:23.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde37fff640 1 -- 172.21.15.22:0/4021334877 learned_addr learned my addr 172.21.15.22:0/4021334877 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:23.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde3c9c6640 1 --2- 172.21.15.22:0/4021334877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde38160e40 0x7fde380b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde3d1c7640 1 --2- 172.21.15.22:0/4021334877 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde380d0c80 0x7fde3813fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde37fff640 1 -- 172.21.15.22:0/4021334877 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde380d0c80 msgr2=0x7fde3813fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:23.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde37fff640 1 --2- 172.21.15.22:0/4021334877 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde380d0c80 0x7fde3813fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde37fff640 1 -- 172.21.15.22:0/4021334877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde38160e40 msgr2=0x7fde380b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:23.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde37fff640 1 --2- 172.21.15.22:0/4021334877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde38160e40 0x7fde380b2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde37fff640 1 -- 172.21.15.22:0/4021334877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde380b2a30 con 0x7fde38161220 2024-03-29T04:18:23.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.616+0000 7fde37fff640 1 --2- 172.21.15.22:0/4021334877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde38161220 0x7fde38143f10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fde240986a0 tx=0x7fde28005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:23.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.620+0000 7fde357fa640 1 -- 172.21.15.22:0/4021334877 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde240a5b90 con 0x7fde38161220 2024-03-29T04:18:23.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.620+0000 7fde357fa640 1 -- 172.21.15.22:0/4021334877 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde240a5d30 con 0x7fde38161220 2024-03-29T04:18:23.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.620+0000 7fde357fa640 1 -- 172.21.15.22:0/4021334877 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde240073d0 con 0x7fde38161220 2024-03-29T04:18:23.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.620+0000 7fde3ec2a640 1 -- 172.21.15.22:0/4021334877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde380cfec0 con 0x7fde38161220 2024-03-29T04:18:23.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.620+0000 7fde3ec2a640 1 -- 172.21.15.22:0/4021334877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde380d0480 con 0x7fde38161220 2024-03-29T04:18:23.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.620+0000 7fde3ec2a640 1 -- 172.21.15.22:0/4021334877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde38068900 con 0x7fde38161220 2024-03-29T04:18:23.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.620+0000 7fde357fa640 1 -- 172.21.15.22:0/4021334877 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fde24007570 con 0x7fde38161220 2024-03-29T04:18:23.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.620+0000 7fde357fa640 1 --2- 172.21.15.22:0/4021334877 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fde1c067ff0 0x7fde1c06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.620+0000 7fde357fa640 1 -- 172.21.15.22:0/4021334877 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fde241206c0 con 0x7fde38161220 2024-03-29T04:18:23.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.620+0000 7fde3c9c6640 1 --2- 172.21.15.22:0/4021334877 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fde1c067ff0 0x7fde1c06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.620+0000 7fde3c9c6640 1 --2- 172.21.15.22:0/4021334877 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fde1c067ff0 0x7fde1c06a4b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fde2c012450 tx=0x7fde2c00a880 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:23.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.624+0000 7fde357fa640 1 -- 172.21.15.22:0/4021334877 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fde24016990 con 0x7fde38161220 2024-03-29T04:18:23.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.780+0000 7fde3ec2a640 1 -- 172.21.15.22:0/4021334877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fde38070a90 con 0x7fde38161220 2024-03-29T04:18:23.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.780+0000 7fde357fa640 1 -- 172.21.15.22:0/4021334877 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fde240f0b00 con 0x7fde38161220 2024-03-29T04:18:23.761 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-29T04:18:23.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.784+0000 7fde3ec2a640 1 -- 172.21.15.22:0/4021334877 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fde1c067ff0 msgr2=0x7fde1c06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:23.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.784+0000 7fde3ec2a640 1 --2- 172.21.15.22:0/4021334877 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fde1c067ff0 0x7fde1c06a4b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fde2c012450 tx=0x7fde2c00a880 comp rx=0 tx=0).stop 2024-03-29T04:18:23.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.784+0000 7fde3ec2a640 1 -- 172.21.15.22:0/4021334877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde38161220 msgr2=0x7fde38143f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:23.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.784+0000 7fde3ec2a640 1 --2- 172.21.15.22:0/4021334877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde38161220 0x7fde38143f10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fde240986a0 tx=0x7fde28005840 comp rx=0 tx=0).stop 2024-03-29T04:18:23.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.784+0000 7fde3ec2a640 1 -- 172.21.15.22:0/4021334877 shutdown_connections 2024-03-29T04:18:23.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.784+0000 7fde3ec2a640 1 --2- 172.21.15.22:0/4021334877 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fde380d0c80 0x7fde3813fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.784+0000 7fde3ec2a640 1 --2- 172.21.15.22:0/4021334877 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fde1c067ff0 0x7fde1c06a4b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.784+0000 7fde3ec2a640 1 --2- 172.21.15.22:0/4021334877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde38161220 0x7fde38143f10 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.784+0000 7fde3ec2a640 1 --2- 172.21.15.22:0/4021334877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde38160e40 0x7fde380b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.784+0000 7fde3ec2a640 1 -- 172.21.15.22:0/4021334877 >> 172.21.15.22:0/4021334877 conn(0x7fde380d1070 msgr2=0x7fde380d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:23.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.784+0000 7fde3ec2a640 1 -- 172.21.15.22:0/4021334877 shutdown_connections 2024-03-29T04:18:23.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.784+0000 7fde3ec2a640 1 -- 172.21.15.22:0/4021334877 wait complete. 2024-03-29T04:18:23.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:18:23.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.916+0000 7f4779f25640 1 Processor -- start 2024-03-29T04:18:23.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.916+0000 7f4779f25640 1 -- start start 2024-03-29T04:18:23.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.916+0000 7f4779f25640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f477406b660 0x7f477415c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.916+0000 7f4779f25640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f477406f9e0 0x7f477406fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.916+0000 7f4779f25640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f477406baf0 con 0x7f477415c390 2024-03-29T04:18:23.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.916+0000 7f4779f25640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f47740cfcf0 con 0x7f477406f9e0 2024-03-29T04:18:23.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.916+0000 7f4779f25640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f477406bdd0 con 0x7f477406b660 2024-03-29T04:18:23.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.916+0000 7f4773fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f477406f9e0 0x7f477406fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.916+0000 7f4773fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f477406f9e0 0x7f477406fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59662/0 (socket says 172.21.15.22:59662) 2024-03-29T04:18:23.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.916+0000 7f4773fff640 1 -- 172.21.15.22:0/1668771875 learned_addr learned my addr 172.21.15.22:0/1668771875 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:23.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.916+0000 7f4772ffd640 1 --2- 172.21.15.22:0/1668771875 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f477406b660 0x7f477415c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.916+0000 7f4773fff640 1 -- 172.21.15.22:0/1668771875 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f477406b660 msgr2=0x7f477415c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:23.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.916+0000 7f4773fff640 1 --2- 172.21.15.22:0/1668771875 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f477406b660 0x7f477415c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.916+0000 7f4773fff640 1 -- 172.21.15.22:0/1668771875 >> v1:172.21.15.151:6789/0 conn(0x7f477415c390 legacy=0x7f47740d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:23.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.916+0000 7f4773fff640 1 -- 172.21.15.22:0/1668771875 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477406c0b0 con 0x7f477406f9e0 2024-03-29T04:18:23.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4773fff640 1 --2- 172.21.15.22:0/1668771875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f477406f9e0 0x7f477406fe70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4764009870 tx=0x7f476401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447aea86b18beb38 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:23.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f47727fc640 1 -- 172.21.15.22:0/1668771875 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47640b1070 con 0x7f477406f9e0 2024-03-29T04:18:23.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f47727fc640 1 -- 172.21.15.22:0/1668771875 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47640ac630 con 0x7f477406f9e0 2024-03-29T04:18:23.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f47727fc640 1 -- 172.21.15.22:0/1668771875 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47640a8050 con 0x7f477406f9e0 2024-03-29T04:18:23.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4779f25640 1 -- 172.21.15.22:0/1668771875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f477406f9e0 msgr2=0x7f477406fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:23.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4779f25640 1 --2- 172.21.15.22:0/1668771875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f477406f9e0 0x7f477406fe70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4764009870 tx=0x7f476401c520 comp rx=0 tx=0).stop 2024-03-29T04:18:23.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4779f25640 1 -- 172.21.15.22:0/1668771875 shutdown_connections 2024-03-29T04:18:23.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4779f25640 1 --2- 172.21.15.22:0/1668771875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f477406f9e0 0x7f477406fe70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4779f25640 1 --2- 172.21.15.22:0/1668771875 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f477406b660 0x7f477415c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4779f25640 1 -- 172.21.15.22:0/1668771875 >> 172.21.15.22:0/1668771875 conn(0x7f47740d1070 msgr2=0x7f47740d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:23.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4779f25640 1 -- 172.21.15.22:0/1668771875 shutdown_connections 2024-03-29T04:18:23.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4779f25640 1 -- 172.21.15.22:0/1668771875 wait complete. 2024-03-29T04:18:23.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4779f25640 1 Processor -- start 2024-03-29T04:18:23.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4779f25640 1 -- start start 2024-03-29T04:18:23.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4779f25640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f477406f9e0 0x7f4774154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4779f25640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f477415c390 0x7f4774151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4779f25640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47740b0eb0 0x7f4774147300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4779f25640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f477406d520 con 0x7f47740b0eb0 2024-03-29T04:18:23.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4779f25640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f477406d3a0 con 0x7f477415c390 2024-03-29T04:18:23.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4779f25640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f477406d6a0 con 0x7f477406f9e0 2024-03-29T04:18:23.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4773fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47740b0eb0 0x7f4774147300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4773fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47740b0eb0 0x7f4774147300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59664/0 (socket says 172.21.15.22:59664) 2024-03-29T04:18:23.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4773fff640 1 -- 172.21.15.22:0/831512987 learned_addr learned my addr 172.21.15.22:0/831512987 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:23.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f47737fe640 1 --2- 172.21.15.22:0/831512987 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f477406f9e0 0x7f4774154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4772ffd640 1 --2- 172.21.15.22:0/831512987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f477415c390 0x7f4774151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4773fff640 1 -- 172.21.15.22:0/831512987 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f477406f9e0 msgr2=0x7f4774154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:23.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4773fff640 1 --2- 172.21.15.22:0/831512987 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f477406f9e0 0x7f4774154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4773fff640 1 -- 172.21.15.22:0/831512987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f477415c390 msgr2=0x7f4774151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:23.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4773fff640 1 --2- 172.21.15.22:0/831512987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f477415c390 0x7f4774151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:23.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.920+0000 7f4773fff640 1 -- 172.21.15.22:0/831512987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4774151560 con 0x7f47740b0eb0 2024-03-29T04:18:23.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.924+0000 7f4773fff640 1 --2- 172.21.15.22:0/831512987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47740b0eb0 0x7f4774147300 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4764009870 tx=0x7f47640192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:23.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.924+0000 7f475bfff640 1 -- 172.21.15.22:0/831512987 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47640b1040 con 0x7f47740b0eb0 2024-03-29T04:18:23.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.924+0000 7f475bfff640 1 -- 172.21.15.22:0/831512987 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47640ba070 con 0x7f47740b0eb0 2024-03-29T04:18:23.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.924+0000 7f4779f25640 1 -- 172.21.15.22:0/831512987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4774158e00 con 0x7f47740b0eb0 2024-03-29T04:18:23.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.924+0000 7f475bfff640 1 -- 172.21.15.22:0/831512987 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47640a8040 con 0x7f47740b0eb0 2024-03-29T04:18:23.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.924+0000 7f4779f25640 1 -- 172.21.15.22:0/831512987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4774156270 con 0x7f47740b0eb0 2024-03-29T04:18:23.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.924+0000 7f4779f25640 1 -- 172.21.15.22:0/831512987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4774068900 con 0x7f47740b0eb0 2024-03-29T04:18:23.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.924+0000 7f475bfff640 1 -- 172.21.15.22:0/831512987 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f47640b5640 con 0x7f47740b0eb0 2024-03-29T04:18:23.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.924+0000 7f475bfff640 1 --2- 172.21.15.22:0/831512987 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4760067fc0 0x7f476006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:23.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.924+0000 7f475bfff640 1 -- 172.21.15.22:0/831512987 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f4764120580 con 0x7f47740b0eb0 2024-03-29T04:18:23.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.924+0000 7f47737fe640 1 --2- 172.21.15.22:0/831512987 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4760067fc0 0x7f476006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:23.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.924+0000 7f47737fe640 1 --2- 172.21.15.22:0/831512987 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4760067fc0 0x7f476006a480 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f476c005530 tx=0x7f476c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:23.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:23.928+0000 7f475bfff640 1 -- 172.21.15.22:0/831512987 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f4764016110 con 0x7f47740b0eb0 2024-03-29T04:18:24.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.088+0000 7f4779f25640 1 -- 172.21.15.22:0/831512987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f47740d1f80 con 0x7f47740b0eb0 2024-03-29T04:18:24.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.088+0000 7f475bfff640 1 -- 172.21.15.22:0/831512987 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f47640f0930 con 0x7f47740b0eb0 2024-03-29T04:18:24.070 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:24.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.092+0000 7f4779f25640 1 -- 172.21.15.22:0/831512987 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4760067fc0 msgr2=0x7f476006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:24.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.092+0000 7f4779f25640 1 --2- 172.21.15.22:0/831512987 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4760067fc0 0x7f476006a480 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f476c005530 tx=0x7f476c009040 comp rx=0 tx=0).stop 2024-03-29T04:18:24.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.092+0000 7f4779f25640 1 -- 172.21.15.22:0/831512987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47740b0eb0 msgr2=0x7f4774147300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:24.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.092+0000 7f4779f25640 1 --2- 172.21.15.22:0/831512987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47740b0eb0 0x7f4774147300 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4764009870 tx=0x7f47640192c0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.092+0000 7f4779f25640 1 -- 172.21.15.22:0/831512987 shutdown_connections 2024-03-29T04:18:24.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.092+0000 7f4779f25640 1 --2- 172.21.15.22:0/831512987 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4760067fc0 0x7f476006a480 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.092+0000 7f4779f25640 1 --2- 172.21.15.22:0/831512987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47740b0eb0 0x7f4774147300 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.092+0000 7f4779f25640 1 --2- 172.21.15.22:0/831512987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f477415c390 0x7f4774151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.092+0000 7f4779f25640 1 --2- 172.21.15.22:0/831512987 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f477406f9e0 0x7f4774154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.092+0000 7f4779f25640 1 -- 172.21.15.22:0/831512987 >> 172.21.15.22:0/831512987 conn(0x7f47740d1070 msgr2=0x7f47740d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:24.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.092+0000 7f4779f25640 1 -- 172.21.15.22:0/831512987 shutdown_connections 2024-03-29T04:18:24.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.092+0000 7f4779f25640 1 -- 172.21.15.22:0/831512987 wait complete. 2024-03-29T04:18:24.096 INFO:tasks.cephfs_test_runner:Starting test: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-29T04:18:24.096 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:24.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.228+0000 7fede97d9640 1 Processor -- start 2024-03-29T04:18:24.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.228+0000 7fede97d9640 1 -- start start 2024-03-29T04:18:24.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.228+0000 7fede97d9640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fede4161cb0 0x7fede415f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:24.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.228+0000 7fede97d9640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fede415ff90 0x7fede40b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:24.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.228+0000 7fede97d9640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fede40725d0 0x7fede4072a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:24.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.228+0000 7fede97d9640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fede40cee60 con 0x7fede4161cb0 2024-03-29T04:18:24.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.228+0000 7fede97d9640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fede40b5ba0 con 0x7fede40725d0 2024-03-29T04:18:24.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.228+0000 7fede97d9640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fede40b5e80 con 0x7fede415ff90 2024-03-29T04:18:24.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.228+0000 7fede2ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fede4161cb0 0x7fede415f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:24.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.228+0000 7fede2ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fede4161cb0 0x7fede415f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59680/0 (socket says 172.21.15.22:59680) 2024-03-29T04:18:24.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.228+0000 7fede2ffd640 1 -- 172.21.15.22:0/2166500686 learned_addr learned my addr 172.21.15.22:0/2166500686 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:24.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.228+0000 7fede27fc640 1 --2- 172.21.15.22:0/2166500686 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fede415ff90 0x7fede40b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:24.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.228+0000 7fede37fe640 1 --2- 172.21.15.22:0/2166500686 >> v2:172.21.15.151:3300/0 conn(0x7fede40725d0 0x7fede4072a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:24.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.228+0000 7fede2ffd640 1 -- 172.21.15.22:0/2166500686 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fede415ff90 msgr2=0x7fede40b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:24.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.228+0000 7fede2ffd640 1 --2- 172.21.15.22:0/2166500686 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fede415ff90 0x7fede40b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.228+0000 7fede2ffd640 1 -- 172.21.15.22:0/2166500686 >> v2:172.21.15.151:3300/0 conn(0x7fede40725d0 msgr2=0x7fede4072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:24.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.228+0000 7fede2ffd640 1 --2- 172.21.15.22:0/2166500686 >> v2:172.21.15.151:3300/0 conn(0x7fede40725d0 0x7fede4072a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.228+0000 7fede2ffd640 1 -- 172.21.15.22:0/2166500686 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede40b6160 con 0x7fede4161cb0 2024-03-29T04:18:24.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.228+0000 7fede37fe640 1 --2- 172.21.15.22:0/2166500686 >> v2:172.21.15.151:3300/0 conn(0x7fede40725d0 0x7fede4072a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:24.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.228+0000 7fede27fc640 1 --2- 172.21.15.22:0/2166500686 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fede415ff90 0x7fede40b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:24.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.232+0000 7fede2ffd640 1 --2- 172.21.15.22:0/2166500686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fede4161cb0 0x7fede415f790 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fede40b2c00 tx=0x7fedd401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ce35bdf31f80c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:24.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.232+0000 7fede1ffb640 1 -- 172.21.15.22:0/2166500686 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd40b2070 con 0x7fede4161cb0 2024-03-29T04:18:24.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.232+0000 7fede1ffb640 1 -- 172.21.15.22:0/2166500686 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd40a9070 con 0x7fede4161cb0 2024-03-29T04:18:24.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.232+0000 7fede1ffb640 1 -- 172.21.15.22:0/2166500686 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd40ad880 con 0x7fede4161cb0 2024-03-29T04:18:24.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.232+0000 7fede97d9640 1 -- 172.21.15.22:0/2166500686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fede4161cb0 msgr2=0x7fede415f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:24.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.232+0000 7fede97d9640 1 --2- 172.21.15.22:0/2166500686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fede4161cb0 0x7fede415f790 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fede40b2c00 tx=0x7fedd401c490 comp rx=0 tx=0).stop 2024-03-29T04:18:24.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.232+0000 7fede97d9640 1 -- 172.21.15.22:0/2166500686 shutdown_connections 2024-03-29T04:18:24.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.232+0000 7fede97d9640 1 --2- 172.21.15.22:0/2166500686 >> v2:172.21.15.151:3300/0 conn(0x7fede40725d0 0x7fede4072a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.232+0000 7fede97d9640 1 --2- 172.21.15.22:0/2166500686 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fede415ff90 0x7fede40b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.232+0000 7fede97d9640 1 --2- 172.21.15.22:0/2166500686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fede4161cb0 0x7fede415f790 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.232+0000 7fede97d9640 1 -- 172.21.15.22:0/2166500686 >> 172.21.15.22:0/2166500686 conn(0x7fede40d0f60 msgr2=0x7fede40d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:24.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.232+0000 7fede97d9640 1 -- 172.21.15.22:0/2166500686 shutdown_connections 2024-03-29T04:18:24.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.232+0000 7fede97d9640 1 -- 172.21.15.22:0/2166500686 wait complete. 2024-03-29T04:18:24.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.232+0000 7fede97d9640 1 Processor -- start 2024-03-29T04:18:24.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.232+0000 7fede97d9640 1 -- start start 2024-03-29T04:18:24.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede97d9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fede40725d0 0x7fede413a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:24.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede97d9640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fede415ff90 0x7fede40af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:24.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede97d9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fede4161cb0 0x7fede40afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:24.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede97d9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fede406e010 con 0x7fede4161cb0 2024-03-29T04:18:24.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede97d9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fede406de90 con 0x7fede40725d0 2024-03-29T04:18:24.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede97d9640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fede40cf5d0 con 0x7fede415ff90 2024-03-29T04:18:24.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fede4161cb0 0x7fede40afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:24.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fede4161cb0 0x7fede40afe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59688/0 (socket says 172.21.15.22:59688) 2024-03-29T04:18:24.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede37fe640 1 -- 172.21.15.22:0/649218679 learned_addr learned my addr 172.21.15.22:0/649218679 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:24.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede27fc640 1 --2- 172.21.15.22:0/649218679 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fede415ff90 0x7fede40af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:24.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede2ffd640 1 --2- 172.21.15.22:0/649218679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fede40725d0 0x7fede413a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:24.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede37fe640 1 -- 172.21.15.22:0/649218679 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fede415ff90 msgr2=0x7fede40af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:24.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede37fe640 1 --2- 172.21.15.22:0/649218679 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fede415ff90 0x7fede40af8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede37fe640 1 -- 172.21.15.22:0/649218679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fede40725d0 msgr2=0x7fede413a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:24.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede37fe640 1 --2- 172.21.15.22:0/649218679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fede40725d0 0x7fede413a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede37fe640 1 -- 172.21.15.22:0/649218679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede40b0770 con 0x7fede4161cb0 2024-03-29T04:18:24.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede2ffd640 1 --2- 172.21.15.22:0/649218679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fede40725d0 0x7fede413a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:24.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede37fe640 1 --2- 172.21.15.22:0/649218679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fede4161cb0 0x7fede40afe10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7feddc00cf20 tx=0x7feddc00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:24.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fedcb7fe640 1 -- 172.21.15.22:0/649218679 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc01a070 con 0x7fede4161cb0 2024-03-29T04:18:24.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fedcb7fe640 1 -- 172.21.15.22:0/649218679 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feddc00d840 con 0x7fede4161cb0 2024-03-29T04:18:24.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede97d9640 1 -- 172.21.15.22:0/649218679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede40b0490 con 0x7fede4161cb0 2024-03-29T04:18:24.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fedcb7fe640 1 -- 172.21.15.22:0/649218679 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc00dc30 con 0x7fede4161cb0 2024-03-29T04:18:24.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede97d9640 1 -- 172.21.15.22:0/649218679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede4152510 con 0x7fede4161cb0 2024-03-29T04:18:24.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede97d9640 1 -- 172.21.15.22:0/649218679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede4068890 con 0x7fede4161cb0 2024-03-29T04:18:24.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fedcb7fe640 1 -- 172.21.15.22:0/649218679 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7feddc01d070 con 0x7fede4161cb0 2024-03-29T04:18:24.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fedcb7fe640 1 --2- 172.21.15.22:0/649218679 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fedd0067fc0 0x7fedd006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:24.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fedcb7fe640 1 -- 172.21.15.22:0/649218679 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7feddc087110 con 0x7fede4161cb0 2024-03-29T04:18:24.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede2ffd640 1 --2- 172.21.15.22:0/649218679 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fedd0067fc0 0x7fedd006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:24.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.236+0000 7fede2ffd640 1 --2- 172.21.15.22:0/649218679 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fedd0067fc0 0x7fedd006a480 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fedd40af000 tx=0x7fedd4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:24.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.240+0000 7fedcb7fe640 1 -- 172.21.15.22:0/649218679 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7feddc057550 con 0x7fede4161cb0 2024-03-29T04:18:24.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.396+0000 7fede97d9640 1 -- 172.21.15.22:0/649218679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fede40ce7c0 con 0x7fede4161cb0 2024-03-29T04:18:24.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.400+0000 7fedcb7fe640 1 -- 172.21.15.22:0/649218679 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18847 (secure 0 0 0) 0x7feddc056e50 con 0x7fede4161cb0 2024-03-29T04:18:24.380 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:24.380 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:18:16.893439+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:24.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.400+0000 7fede97d9640 1 -- 172.21.15.22:0/649218679 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fedd0067fc0 msgr2=0x7fedd006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:24.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.400+0000 7fede97d9640 1 --2- 172.21.15.22:0/649218679 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fedd0067fc0 0x7fedd006a480 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fedd40af000 tx=0x7fedd4008040 comp rx=0 tx=0).stop 2024-03-29T04:18:24.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.400+0000 7fede97d9640 1 -- 172.21.15.22:0/649218679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fede4161cb0 msgr2=0x7fede40afe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:24.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.400+0000 7fede97d9640 1 --2- 172.21.15.22:0/649218679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fede4161cb0 0x7fede40afe10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7feddc00cf20 tx=0x7feddc00a7d0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.400+0000 7fede97d9640 1 -- 172.21.15.22:0/649218679 shutdown_connections 2024-03-29T04:18:24.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.400+0000 7fede97d9640 1 --2- 172.21.15.22:0/649218679 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fedd0067fc0 0x7fedd006a480 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.400+0000 7fede97d9640 1 --2- 172.21.15.22:0/649218679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fede4161cb0 0x7fede40afe10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.400+0000 7fede97d9640 1 --2- 172.21.15.22:0/649218679 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fede415ff90 0x7fede40af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.400+0000 7fede97d9640 1 --2- 172.21.15.22:0/649218679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fede40725d0 0x7fede413a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.400+0000 7fede97d9640 1 -- 172.21.15.22:0/649218679 >> 172.21.15.22:0/649218679 conn(0x7fede40d0f60 msgr2=0x7fede4154ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.400+0000 7fede97d9640 1 -- 172.21.15.22:0/649218679 shutdown_connections 2024-03-29T04:18:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.400+0000 7fede97d9640 1 -- 172.21.15.22:0/649218679 wait complete. 2024-03-29T04:18:24.398 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:17:34.034142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:17:34.886659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:17:38.628632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:18:24.398 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:18:24.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.540+0000 7f7be765f640 1 Processor -- start 2024-03-29T04:18:24.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.540+0000 7f7be765f640 1 -- start start 2024-03-29T04:18:24.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.540+0000 7f7be765f640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f7be013f140 0x7f7be01606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:24.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.540+0000 7f7be765f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7be006cd80 0x7f7be00ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:24.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.540+0000 7f7be765f640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7be006d710 con 0x7f7be015ff40 2024-03-29T04:18:24.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.540+0000 7f7be765f640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f7be006e320 con 0x7f7be006cd80 2024-03-29T04:18:24.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.540+0000 7f7be765f640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f7be006d9f0 con 0x7f7be013f140 2024-03-29T04:18:24.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.544+0000 7f7be5bfc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7be006cd80 0x7f7be00ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:24.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.544+0000 7f7be4bfa640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f7be015ff40 0x7f7be006c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58040/0 (socket says 172.21.15.22:58040) 2024-03-29T04:18:24.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.544+0000 7f7be4bfa640 1 -- 172.21.15.22:0/1369675461 learned_addr learned my addr 172.21.15.22:0/1369675461 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:24.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.544+0000 7f7be53fb640 1 --2- 172.21.15.22:0/1369675461 >> v2:172.21.15.151:3300/0 conn(0x7f7be013f140 0x7f7be01606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:24.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.544+0000 7f7be5bfc640 1 -- 172.21.15.22:0/1369675461 >> v2:172.21.15.151:3300/0 conn(0x7f7be013f140 msgr2=0x7f7be01606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:24.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.544+0000 7f7be5bfc640 1 --2- 172.21.15.22:0/1369675461 >> v2:172.21.15.151:3300/0 conn(0x7f7be013f140 0x7f7be01606d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.544+0000 7f7be5bfc640 1 -- 172.21.15.22:0/1369675461 >> v1:172.21.15.22:6789/0 conn(0x7f7be015ff40 legacy=0x7f7be006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:24.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.544+0000 7f7be5bfc640 1 -- 172.21.15.22:0/1369675461 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be006dcd0 con 0x7f7be006cd80 2024-03-29T04:18:24.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.544+0000 7f7bd7fff640 1 -- 172.21.15.22:0/1369675461 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1622932137 0 0) 0x7f7be006d710 con 0x7f7be015ff40 2024-03-29T04:18:24.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.544+0000 7f7be5bfc640 1 --2- 172.21.15.22:0/1369675461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be006cd80 0x7f7be00ce940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7be00cfe30 tx=0x7f7bd801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68e741a8a99d732 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:24.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.544+0000 7f7bd7fff640 1 -- 172.21.15.22:0/1369675461 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd80b2070 con 0x7f7be006cd80 2024-03-29T04:18:24.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.544+0000 7f7bd7fff640 1 -- 172.21.15.22:0/1369675461 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd80a59e0 con 0x7f7be006cd80 2024-03-29T04:18:24.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.544+0000 7f7bd7fff640 1 -- 172.21.15.22:0/1369675461 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd80a5db0 con 0x7f7be006cd80 2024-03-29T04:18:24.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.544+0000 7f7be765f640 1 -- 172.21.15.22:0/1369675461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be006cd80 msgr2=0x7f7be00ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:24.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.544+0000 7f7be765f640 1 --2- 172.21.15.22:0/1369675461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be006cd80 0x7f7be00ce940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7be00cfe30 tx=0x7f7bd801c900 comp rx=0 tx=0).stop 2024-03-29T04:18:24.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.544+0000 7f7be765f640 1 -- 172.21.15.22:0/1369675461 shutdown_connections 2024-03-29T04:18:24.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.544+0000 7f7be765f640 1 --2- 172.21.15.22:0/1369675461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be006cd80 0x7f7be00ce940 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.544+0000 7f7be765f640 1 --2- 172.21.15.22:0/1369675461 >> v2:172.21.15.151:3300/0 conn(0x7f7be013f140 0x7f7be01606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.544+0000 7f7be765f640 1 -- 172.21.15.22:0/1369675461 >> 172.21.15.22:0/1369675461 conn(0x7f7be00d00a0 msgr2=0x7f7be00d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:24.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.544+0000 7f7be765f640 1 -- 172.21.15.22:0/1369675461 shutdown_connections 2024-03-29T04:18:24.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.544+0000 7f7be765f640 1 -- 172.21.15.22:0/1369675461 wait complete. 2024-03-29T04:18:24.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be765f640 1 Processor -- start 2024-03-29T04:18:24.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be765f640 1 -- start start 2024-03-29T04:18:24.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be765f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7be006cd80 0x7f7be013a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:24.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be765f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7be013f140 0x7f7be013d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:24.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be765f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be015ff40 0x7f7be013e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:24.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be765f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7be00cf810 con 0x7f7be015ff40 2024-03-29T04:18:24.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be765f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7be00cf690 con 0x7f7be006cd80 2024-03-29T04:18:24.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be765f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7be00cf990 con 0x7f7be013f140 2024-03-29T04:18:24.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be5bfc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be015ff40 0x7f7be013e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:24.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be5bfc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be015ff40 0x7f7be013e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59702/0 (socket says 172.21.15.22:59702) 2024-03-29T04:18:24.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be5bfc640 1 -- 172.21.15.22:0/3820026950 learned_addr learned my addr 172.21.15.22:0/3820026950 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:24.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be53fb640 1 --2- 172.21.15.22:0/3820026950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7be006cd80 0x7f7be013a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:24.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be4bfa640 1 --2- 172.21.15.22:0/3820026950 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7be013f140 0x7f7be013d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:24.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be5bfc640 1 -- 172.21.15.22:0/3820026950 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7be013f140 msgr2=0x7f7be013d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:24.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be5bfc640 1 --2- 172.21.15.22:0/3820026950 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7be013f140 0x7f7be013d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be5bfc640 1 -- 172.21.15.22:0/3820026950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7be006cd80 msgr2=0x7f7be013a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:24.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be5bfc640 1 --2- 172.21.15.22:0/3820026950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7be006cd80 0x7f7be013a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be5bfc640 1 -- 172.21.15.22:0/3820026950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be013c890 con 0x7f7be015ff40 2024-03-29T04:18:24.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be5bfc640 1 --2- 172.21.15.22:0/3820026950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be015ff40 0x7f7be013e1b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7bd80b0040 tx=0x7f7bd80196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:24.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7bd5ffb640 1 -- 172.21.15.22:0/3820026950 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd80b2050 con 0x7f7be015ff40 2024-03-29T04:18:24.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7bd5ffb640 1 -- 172.21.15.22:0/3820026950 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd80be070 con 0x7f7be015ff40 2024-03-29T04:18:24.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be765f640 1 -- 172.21.15.22:0/3820026950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be013cb70 con 0x7f7be015ff40 2024-03-29T04:18:24.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7bd5ffb640 1 -- 172.21.15.22:0/3820026950 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd80adb10 con 0x7f7be015ff40 2024-03-29T04:18:24.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be765f640 1 -- 172.21.15.22:0/3820026950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be013d130 con 0x7f7be015ff40 2024-03-29T04:18:24.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7be765f640 1 -- 172.21.15.22:0/3820026950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7be00688e0 con 0x7f7be015ff40 2024-03-29T04:18:24.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.548+0000 7f7bd5ffb640 1 -- 172.21.15.22:0/3820026950 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f7bd80adcb0 con 0x7f7be015ff40 2024-03-29T04:18:24.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.552+0000 7f7bd5ffb640 1 --2- 172.21.15.22:0/3820026950 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7bc4067ff0 0x7f7bc406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:24.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.552+0000 7f7bd5ffb640 1 -- 172.21.15.22:0/3820026950 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f7bd8121780 con 0x7f7be015ff40 2024-03-29T04:18:24.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.552+0000 7f7be53fb640 1 --2- 172.21.15.22:0/3820026950 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7bc4067ff0 0x7f7bc406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:24.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.552+0000 7f7be53fb640 1 --2- 172.21.15.22:0/3820026950 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7bc4067ff0 0x7f7bc406a4b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00b9d0 tx=0x7f7bdc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:24.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.556+0000 7f7bd5ffb640 1 -- 172.21.15.22:0/3820026950 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f7bd80c7020 con 0x7f7be015ff40 2024-03-29T04:18:24.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.720+0000 7f7be765f640 1 -- 172.21.15.22:0/3820026950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7be0072280 con 0x7f7be015ff40 2024-03-29T04:18:24.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.720+0000 7f7bd5ffb640 1 -- 172.21.15.22:0/3820026950 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f7bd8016520 con 0x7f7be015ff40 2024-03-29T04:18:24.700 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:18:24.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.720+0000 7f7be765f640 1 -- 172.21.15.22:0/3820026950 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7bc4067ff0 msgr2=0x7f7bc406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:24.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.720+0000 7f7be765f640 1 --2- 172.21.15.22:0/3820026950 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7bc4067ff0 0x7f7bc406a4b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00b9d0 tx=0x7f7bdc008040 comp rx=0 tx=0).stop 2024-03-29T04:18:24.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.720+0000 7f7be765f640 1 -- 172.21.15.22:0/3820026950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be015ff40 msgr2=0x7f7be013e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:24.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.720+0000 7f7be765f640 1 --2- 172.21.15.22:0/3820026950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be015ff40 0x7f7be013e1b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7bd80b0040 tx=0x7f7bd80196a0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.720+0000 7f7be765f640 1 -- 172.21.15.22:0/3820026950 shutdown_connections 2024-03-29T04:18:24.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.720+0000 7f7be765f640 1 --2- 172.21.15.22:0/3820026950 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7bc4067ff0 0x7f7bc406a4b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.720+0000 7f7be765f640 1 --2- 172.21.15.22:0/3820026950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be015ff40 0x7f7be013e1b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.720+0000 7f7be765f640 1 --2- 172.21.15.22:0/3820026950 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7be013f140 0x7f7be013d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.720+0000 7f7be765f640 1 --2- 172.21.15.22:0/3820026950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7be006cd80 0x7f7be013a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.720+0000 7f7be765f640 1 -- 172.21.15.22:0/3820026950 >> 172.21.15.22:0/3820026950 conn(0x7f7be00d00a0 msgr2=0x7f7be00b08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:24.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.720+0000 7f7be765f640 1 -- 172.21.15.22:0/3820026950 shutdown_connections 2024-03-29T04:18:24.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.720+0000 7f7be765f640 1 -- 172.21.15.22:0/3820026950 wait complete. 2024-03-29T04:18:24.719 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:18:24.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.864+0000 7f50049e4640 1 Processor -- start 2024-03-29T04:18:24.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.864+0000 7f50049e4640 1 -- start start 2024-03-29T04:18:24.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f50049e4640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f500015baa0 0x7f50000b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:24.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f50049e4640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50000b3c40 con 0x7f50000b4090 2024-03-29T04:18:24.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f50049e4640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5000071910 con 0x7f50000cf870 2024-03-29T04:18:24.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f50049e4640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f5000071bf0 con 0x7f500015baa0 2024-03-29T04:18:24.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffed9d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f50000b4090 0x7f50000b44b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58050/0 (socket says 172.21.15.22:58050) 2024-03-29T04:18:24.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffed9d640 1 -- 172.21.15.22:0/3785214177 learned_addr learned my addr 172.21.15.22:0/3785214177 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:24.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ff7fff640 1 --2- 172.21.15.22:0/3785214177 >> v2:172.21.15.151:3300/0 conn(0x7f500015baa0 0x7f50000b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:24.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffdd9b640 1 -- 172.21.15.22:0/3785214177 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3723840071 0 0) 0x7f50000b3c40 con 0x7f50000b4090 2024-03-29T04:18:24.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffdd9b640 1 -- 172.21.15.22:0/3785214177 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4fec004640 con 0x7f50000b4090 2024-03-29T04:18:24.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffdd9b640 1 -- 172.21.15.22:0/3785214177 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3476633594 0 0) 0x7f4fec004640 con 0x7f50000b4090 2024-03-29T04:18:24.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffdd9b640 1 -- 172.21.15.22:0/3785214177 >> v2:172.21.15.151:3300/0 conn(0x7f500015baa0 msgr2=0x7f50000b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:24.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffdd9b640 1 --2- 172.21.15.22:0/3785214177 >> v2:172.21.15.151:3300/0 conn(0x7f500015baa0 0x7f50000b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffdd9b640 1 -- 172.21.15.22:0/3785214177 >> v1:172.21.15.151:6789/0 conn(0x7f50000cf870 legacy=0x7f50000726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:24.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffdd9b640 1 -- 172.21.15.22:0/3785214177 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5000071ed0 con 0x7f50000b4090 2024-03-29T04:18:24.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffdd9b640 1 -- 172.21.15.22:0/3785214177 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f4ff8001a00 con 0x7f50000b4090 2024-03-29T04:18:24.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffdd9b640 1 -- 172.21.15.22:0/3785214177 >> v1:172.21.15.22:6789/0 conn(0x7f50000b4090 legacy=0x7f50000b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:24.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffdd9b640 1 --2- 172.21.15.22:0/3785214177 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4fec004ad0 0x7f4fec004ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:24.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffdd9b640 1 --2- 172.21.15.22:0/3785214177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4fec005920 0x7f4fec005da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:24.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffdd9b640 1 --2- 172.21.15.22:0/3785214177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fec0062e0 0x7f4fec00cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:24.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffdd9b640 1 -- 172.21.15.22:0/3785214177 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5000071bf0 con 0x7f4fec0062e0 2024-03-29T04:18:24.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffdd9b640 1 -- 172.21.15.22:0/3785214177 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec00d180 con 0x7f4fec005920 2024-03-29T04:18:24.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffdd9b640 1 -- 172.21.15.22:0/3785214177 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4fec00d380 con 0x7f4fec004ad0 2024-03-29T04:18:24.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffed9d640 1 --2- 172.21.15.22:0/3785214177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fec0062e0 0x7f4fec00cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:24.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffe59c640 1 --2- 172.21.15.22:0/3785214177 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4fec004ad0 0x7f4fec004ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:24.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ff7fff640 1 --2- 172.21.15.22:0/3785214177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4fec005920 0x7f4fec005da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:24.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffed9d640 1 -- 172.21.15.22:0/3785214177 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4fec004ad0 msgr2=0x7f4fec004ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:24.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffed9d640 1 --2- 172.21.15.22:0/3785214177 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4fec004ad0 0x7f4fec004ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffed9d640 1 -- 172.21.15.22:0/3785214177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4fec005920 msgr2=0x7f4fec005da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:24.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffed9d640 1 --2- 172.21.15.22:0/3785214177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4fec005920 0x7f4fec005da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffed9d640 1 -- 172.21.15.22:0/3785214177 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fec004640 con 0x7f4fec0062e0 2024-03-29T04:18:24.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.868+0000 7f4ffe59c640 1 --2- 172.21.15.22:0/3785214177 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4fec004ad0 0x7f4fec004ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:24.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.872+0000 7f4ffed9d640 1 -- 172.21.15.22:0/3785214177 reap_dead start 2024-03-29T04:18:24.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.872+0000 7f4ffed9d640 1 --2- 172.21.15.22:0/3785214177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fec0062e0 0x7f4fec00cae0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800a850 tx=0x7f4ff801d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211b5b2b91904255 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:24.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.872+0000 7f4ffdd9b640 1 -- 172.21.15.22:0/3785214177 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff800a4a0 con 0x7f4fec0062e0 2024-03-29T04:18:24.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.872+0000 7f4ffdd9b640 1 -- 172.21.15.22:0/3785214177 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ff80ad650 con 0x7f4fec0062e0 2024-03-29T04:18:24.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.872+0000 7f4ffdd9b640 1 -- 172.21.15.22:0/3785214177 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff80ada40 con 0x7f4fec0062e0 2024-03-29T04:18:24.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.872+0000 7f50049e4640 1 -- 172.21.15.22:0/3785214177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fec0062e0 msgr2=0x7f4fec00cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:24.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.872+0000 7f50049e4640 1 --2- 172.21.15.22:0/3785214177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fec0062e0 0x7f4fec00cae0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800a850 tx=0x7f4ff801d3a0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.872+0000 7f50049e4640 1 -- 172.21.15.22:0/3785214177 shutdown_connections 2024-03-29T04:18:24.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.872+0000 7f50049e4640 1 --2- 172.21.15.22:0/3785214177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fec0062e0 0x7f4fec00cae0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.872+0000 7f50049e4640 1 -- 172.21.15.22:0/3785214177 >> 172.21.15.22:0/3785214177 conn(0x7f50000d00a0 msgr2=0x7f50000d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:24.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.872+0000 7f50049e4640 1 -- 172.21.15.22:0/3785214177 shutdown_connections 2024-03-29T04:18:24.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.872+0000 7f50049e4640 1 -- 172.21.15.22:0/3785214177 wait complete. 2024-03-29T04:18:24.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.872+0000 7f50049e4640 1 Processor -- start 2024-03-29T04:18:24.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.872+0000 7f50049e4640 1 -- start start 2024-03-29T04:18:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.872+0000 7f50049e4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4fec0062e0 0x7f500013a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.872+0000 7f50049e4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f500013aea0 0x7f500013a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.872+0000 7f50049e4640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f500014c7a0 0x7f500014cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.872+0000 7f50049e4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f50000cfc50 con 0x7f500013aea0 2024-03-29T04:18:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.872+0000 7f50049e4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f50000b3e10 con 0x7f4fec0062e0 2024-03-29T04:18:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.872+0000 7f50049e4640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f50000b3700 con 0x7f500014c7a0 2024-03-29T04:18:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f4ff7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f500013aea0 0x7f500013a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f4ff7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f500013aea0 0x7f500013a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59726/0 (socket says 172.21.15.22:59726) 2024-03-29T04:18:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f4ff7fff640 1 -- 172.21.15.22:0/3221344189 learned_addr learned my addr 172.21.15.22:0/3221344189 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f4ffe59c640 1 --2- 172.21.15.22:0/3221344189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4fec0062e0 0x7f500013a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f4ffed9d640 1 --2- 172.21.15.22:0/3221344189 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f500014c7a0 0x7f500014cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f4ff7fff640 1 -- 172.21.15.22:0/3221344189 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f500014c7a0 msgr2=0x7f500014cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f4ff7fff640 1 --2- 172.21.15.22:0/3221344189 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f500014c7a0 0x7f500014cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f4ff7fff640 1 -- 172.21.15.22:0/3221344189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4fec0062e0 msgr2=0x7f500013a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f4ff7fff640 1 --2- 172.21.15.22:0/3221344189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4fec0062e0 0x7f500013a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f4ff7fff640 1 -- 172.21.15.22:0/3221344189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50000cced0 con 0x7f500013aea0 2024-03-29T04:18:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f4ffe59c640 1 --2- 172.21.15.22:0/3221344189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4fec0062e0 0x7f500013a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f4ff7fff640 1 --2- 172.21.15.22:0/3221344189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f500013aea0 0x7f500013a750 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8010890 tx=0x7f4fe8010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f4ff6ffd640 1 -- 172.21.15.22:0/3221344189 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe800f040 con 0x7f500013aea0 2024-03-29T04:18:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f4ff6ffd640 1 -- 172.21.15.22:0/3221344189 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fe800eb60 con 0x7f500013aea0 2024-03-29T04:18:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f50049e4640 1 -- 172.21.15.22:0/3221344189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f500014d560 con 0x7f500013aea0 2024-03-29T04:18:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f4ff6ffd640 1 -- 172.21.15.22:0/3221344189 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe801c400 con 0x7f500013aea0 2024-03-29T04:18:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f50049e4640 1 -- 172.21.15.22:0/3221344189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50000cd190 con 0x7f500013aea0 2024-03-29T04:18:24.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f50049e4640 1 -- 172.21.15.22:0/3221344189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5000068900 con 0x7f500013aea0 2024-03-29T04:18:24.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f4ff6ffd640 1 -- 172.21.15.22:0/3221344189 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f4fe801c5a0 con 0x7f500013aea0 2024-03-29T04:18:24.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f4ff6ffd640 1 --2- 172.21.15.22:0/3221344189 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4fe0067fc0 0x7f4fe006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:24.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f4ff6ffd640 1 -- 172.21.15.22:0/3221344189 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f4fe8086fa0 con 0x7f500013aea0 2024-03-29T04:18:24.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f4ffe59c640 1 --2- 172.21.15.22:0/3221344189 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4fe0067fc0 0x7f4fe006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:24.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.876+0000 7f4ffe59c640 1 --2- 172.21.15.22:0/3221344189 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4fe0067fc0 0x7f4fe006a480 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0000c10 tx=0x7f4ff0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:24.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:24.880+0000 7f4ff6ffd640 1 -- 172.21.15.22:0/3221344189 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f4fe8018030 con 0x7f500013aea0 2024-03-29T04:18:25.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.036+0000 7f50049e4640 1 -- 172.21.15.22:0/3221344189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f50000cc210 con 0x7f500013aea0 2024-03-29T04:18:25.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.036+0000 7f4ff6ffd640 1 -- 172.21.15.22:0/3221344189 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f4fe8057350 con 0x7f500013aea0 2024-03-29T04:18:25.019 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:25.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.040+0000 7f50049e4640 1 -- 172.21.15.22:0/3221344189 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4fe0067fc0 msgr2=0x7f4fe006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:25.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.040+0000 7f50049e4640 1 --2- 172.21.15.22:0/3221344189 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4fe0067fc0 0x7f4fe006a480 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0000c10 tx=0x7f4ff0009040 comp rx=0 tx=0).stop 2024-03-29T04:18:25.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.040+0000 7f50049e4640 1 -- 172.21.15.22:0/3221344189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f500013aea0 msgr2=0x7f500013a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:25.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.040+0000 7f50049e4640 1 --2- 172.21.15.22:0/3221344189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f500013aea0 0x7f500013a750 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8010890 tx=0x7f4fe8010d50 comp rx=0 tx=0).stop 2024-03-29T04:18:25.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.040+0000 7f50049e4640 1 -- 172.21.15.22:0/3221344189 shutdown_connections 2024-03-29T04:18:25.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.040+0000 7f50049e4640 1 --2- 172.21.15.22:0/3221344189 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f500014c7a0 0x7f500014cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.040+0000 7f50049e4640 1 --2- 172.21.15.22:0/3221344189 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4fe0067fc0 0x7f4fe006a480 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.040+0000 7f50049e4640 1 --2- 172.21.15.22:0/3221344189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f500013aea0 0x7f500013a750 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.040+0000 7f50049e4640 1 --2- 172.21.15.22:0/3221344189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4fec0062e0 0x7f500013a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.040+0000 7f50049e4640 1 -- 172.21.15.22:0/3221344189 >> 172.21.15.22:0/3221344189 conn(0x7f50000d00a0 msgr2=0x7f50000b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:25.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.040+0000 7f50049e4640 1 -- 172.21.15.22:0/3221344189 shutdown_connections 2024-03-29T04:18:25.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.040+0000 7f50049e4640 1 -- 172.21.15.22:0/3221344189 wait complete. 2024-03-29T04:18:25.036 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:18:25.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.168+0000 7fd8a711c640 1 Processor -- start 2024-03-29T04:18:25.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.168+0000 7fd8a711c640 1 -- start start 2024-03-29T04:18:25.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.168+0000 7fd8a711c640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd8a015e4c0 0x7fd8a00b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.168+0000 7fd8a711c640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd8a006c070 0x7fd8a00cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.168+0000 7fd8a711c640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8a006c850 con 0x7fd8a00cf4f0 2024-03-29T04:18:25.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.168+0000 7fd8a711c640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fd8a015e8a0 con 0x7fd8a006c070 2024-03-29T04:18:25.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.168+0000 7fd8a711c640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd8a006cb30 con 0x7fd8a015e4c0 2024-03-29T04:18:25.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.168+0000 7fd8a56b9640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fd8a00cf4f0 0x7fd8a0161f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:56990/0 (socket says 172.21.15.22:56990) 2024-03-29T04:18:25.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.168+0000 7fd8a56b9640 1 -- 172.21.15.22:0/2535796288 learned_addr learned my addr 172.21.15.22:0/2535796288 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:25.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd897fff640 1 --2- 172.21.15.22:0/2535796288 >> v2:172.21.15.151:3300/0 conn(0x7fd8a006c070 0x7fd8a00cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8a4eb8640 1 --2- 172.21.15.22:0/2535796288 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd8a015e4c0 0x7fd8a00b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8977fe640 1 -- 172.21.15.22:0/2535796288 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 385483179 0 0) 0x7fd8a006c850 con 0x7fd8a00cf4f0 2024-03-29T04:18:25.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8977fe640 1 -- 172.21.15.22:0/2535796288 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd888003530 con 0x7fd8a00cf4f0 2024-03-29T04:18:25.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8977fe640 1 -- 172.21.15.22:0/2535796288 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1708428703 0 0) 0x7fd888003530 con 0x7fd8a00cf4f0 2024-03-29T04:18:25.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8977fe640 1 -- 172.21.15.22:0/2535796288 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd8a015e4c0 msgr2=0x7fd8a00b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:25.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8977fe640 1 --2- 172.21.15.22:0/2535796288 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd8a015e4c0 0x7fd8a00b9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8977fe640 1 -- 172.21.15.22:0/2535796288 >> v2:172.21.15.151:3300/0 conn(0x7fd8a006c070 msgr2=0x7fd8a00cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8977fe640 1 --2- 172.21.15.22:0/2535796288 >> v2:172.21.15.151:3300/0 conn(0x7fd8a006c070 0x7fd8a00cefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8977fe640 1 -- 172.21.15.22:0/2535796288 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a006ce10 con 0x7fd8a00cf4f0 2024-03-29T04:18:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8977fe640 1 -- 172.21.15.22:0/2535796288 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7fd898001a00 con 0x7fd8a00cf4f0 2024-03-29T04:18:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8977fe640 1 -- 172.21.15.22:0/2535796288 >> v1:172.21.15.22:6789/0 conn(0x7fd8a00cf4f0 legacy=0x7fd8a0161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8977fe640 1 --2- 172.21.15.22:0/2535796288 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd8a015e4c0 0x7fd8880039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8977fe640 1 --2- 172.21.15.22:0/2535796288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd888003f40 0x7fd888004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8977fe640 1 --2- 172.21.15.22:0/2535796288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd888004d50 0x7fd8880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8977fe640 1 -- 172.21.15.22:0/2535796288 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a015e8a0 con 0x7fd888004d50 2024-03-29T04:18:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8977fe640 1 -- 172.21.15.22:0/2535796288 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a006cb30 con 0x7fd888003f40 2024-03-29T04:18:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8977fe640 1 -- 172.21.15.22:0/2535796288 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd88800b9d0 con 0x7fd8a015e4c0 2024-03-29T04:18:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8a56b9640 1 --2- 172.21.15.22:0/2535796288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd888004d50 0x7fd8880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8a4eb8640 1 --2- 172.21.15.22:0/2535796288 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd8a015e4c0 0x7fd8880039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd897fff640 1 --2- 172.21.15.22:0/2535796288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd888003f40 0x7fd888004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8a56b9640 1 -- 172.21.15.22:0/2535796288 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd8a015e4c0 msgr2=0x7fd8880039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8a56b9640 1 --2- 172.21.15.22:0/2535796288 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd8a015e4c0 0x7fd8880039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8a56b9640 1 -- 172.21.15.22:0/2535796288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd888003f40 msgr2=0x7fd888004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:25.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8a56b9640 1 --2- 172.21.15.22:0/2535796288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd888003f40 0x7fd888004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8a56b9640 1 -- 172.21.15.22:0/2535796288 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd888003530 con 0x7fd888004d50 2024-03-29T04:18:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8a56b9640 1 --2- 172.21.15.22:0/2535796288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd888004d50 0x7fd8880051d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd89800a870 tx=0x7fd89801d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241eecc5c41a29f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8977fe640 1 -- 172.21.15.22:0/2535796288 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89800a4c0 con 0x7fd888004d50 2024-03-29T04:18:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8977fe640 1 -- 172.21.15.22:0/2535796288 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8980072d0 con 0x7fd888004d50 2024-03-29T04:18:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8977fe640 1 -- 172.21.15.22:0/2535796288 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8980076c0 con 0x7fd888004d50 2024-03-29T04:18:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8a711c640 1 -- 172.21.15.22:0/2535796288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd888004d50 msgr2=0x7fd8880051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8a711c640 1 --2- 172.21.15.22:0/2535796288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd888004d50 0x7fd8880051d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd89800a870 tx=0x7fd89801d340 comp rx=0 tx=0).stop 2024-03-29T04:18:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8a56b9640 1 -- 172.21.15.22:0/2535796288 reap_dead start 2024-03-29T04:18:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8a711c640 1 -- 172.21.15.22:0/2535796288 shutdown_connections 2024-03-29T04:18:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8a711c640 1 -- 172.21.15.22:0/2535796288 >> 172.21.15.22:0/2535796288 conn(0x7fd8a00b6410 msgr2=0x7fd8a00d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:25.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8a711c640 1 -- 172.21.15.22:0/2535796288 shutdown_connections 2024-03-29T04:18:25.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.172+0000 7fd8a711c640 1 -- 172.21.15.22:0/2535796288 wait complete. 2024-03-29T04:18:25.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8a711c640 1 Processor -- start 2024-03-29T04:18:25.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8a711c640 1 -- start start 2024-03-29T04:18:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8a711c640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd888004d50 0x7fd8a0160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8a711c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8a014dfe0 0x7fd8a0155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8a711c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8a01559b0 0x7fd8a0156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8a711c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a00d3f00 con 0x7fd8a01559b0 2024-03-29T04:18:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8a711c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a00d3d80 con 0x7fd8a014dfe0 2024-03-29T04:18:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8a711c640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd8a00d4080 con 0x7fd888004d50 2024-03-29T04:18:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8a56b9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8a01559b0 0x7fd8a0156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8a56b9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8a01559b0 0x7fd8a0156110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36666/0 (socket says 172.21.15.22:36666) 2024-03-29T04:18:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8a56b9640 1 -- 172.21.15.22:0/4039694711 learned_addr learned my addr 172.21.15.22:0/4039694711 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd897fff640 1 --2- 172.21.15.22:0/4039694711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8a014dfe0 0x7fd8a0155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8a4eb8640 1 --2- 172.21.15.22:0/4039694711 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd888004d50 0x7fd8a0160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8a56b9640 1 -- 172.21.15.22:0/4039694711 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd888004d50 msgr2=0x7fd8a0160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8a56b9640 1 --2- 172.21.15.22:0/4039694711 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd888004d50 0x7fd8a0160aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8a56b9640 1 -- 172.21.15.22:0/4039694711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8a014dfe0 msgr2=0x7fd8a0155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8a56b9640 1 --2- 172.21.15.22:0/4039694711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8a014dfe0 0x7fd8a0155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8a56b9640 1 -- 172.21.15.22:0/4039694711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a015f780 con 0x7fd8a01559b0 2024-03-29T04:18:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8a56b9640 1 --2- 172.21.15.22:0/4039694711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8a01559b0 0x7fd8a0156110 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd898000f90 tx=0x7fd888005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8957fa640 1 -- 172.21.15.22:0/4039694711 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8980b2040 con 0x7fd8a01559b0 2024-03-29T04:18:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8957fa640 1 -- 172.21.15.22:0/4039694711 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8980017c0 con 0x7fd8a01559b0 2024-03-29T04:18:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8957fa640 1 -- 172.21.15.22:0/4039694711 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd898002040 con 0x7fd8a01559b0 2024-03-29T04:18:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8a711c640 1 -- 172.21.15.22:0/4039694711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a015fa00 con 0x7fd8a01559b0 2024-03-29T04:18:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8a711c640 1 -- 172.21.15.22:0/4039694711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a015ffc0 con 0x7fd8a01559b0 2024-03-29T04:18:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8a711c640 1 -- 172.21.15.22:0/4039694711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8a0068900 con 0x7fd8a01559b0 2024-03-29T04:18:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.176+0000 7fd8957fa640 1 -- 172.21.15.22:0/4039694711 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fd8980be060 con 0x7fd8a01559b0 2024-03-29T04:18:25.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.180+0000 7fd8957fa640 1 --2- 172.21.15.22:0/4039694711 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd88c067fc0 0x7fd88c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.180+0000 7fd8957fa640 1 -- 172.21.15.22:0/4039694711 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fd898120410 con 0x7fd8a01559b0 2024-03-29T04:18:25.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.180+0000 7fd8a4eb8640 1 --2- 172.21.15.22:0/4039694711 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd88c067fc0 0x7fd88c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.180+0000 7fd8a4eb8640 1 --2- 172.21.15.22:0/4039694711 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd88c067fc0 0x7fd88c06a480 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd89c009970 tx=0x7fd89c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:25.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.184+0000 7fd8957fa640 1 -- 172.21.15.22:0/4039694711 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fd8a015f780 con 0x7fd8a01559b0 2024-03-29T04:18:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.340+0000 7fd8a711c640 1 -- 172.21.15.22:0/4039694711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd8a015df00 con 0x7fd8a01559b0 2024-03-29T04:18:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.340+0000 7fd8957fa640 1 -- 172.21.15.22:0/4039694711 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fd8980170b0 con 0x7fd8a01559b0 2024-03-29T04:18:25.321 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-29T04:18:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.344+0000 7fd8a711c640 1 -- 172.21.15.22:0/4039694711 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd88c067fc0 msgr2=0x7fd88c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.344+0000 7fd8a711c640 1 --2- 172.21.15.22:0/4039694711 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd88c067fc0 0x7fd88c06a480 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd89c009970 tx=0x7fd89c008040 comp rx=0 tx=0).stop 2024-03-29T04:18:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.344+0000 7fd8a711c640 1 -- 172.21.15.22:0/4039694711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8a01559b0 msgr2=0x7fd8a0156110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.344+0000 7fd8a711c640 1 --2- 172.21.15.22:0/4039694711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8a01559b0 0x7fd8a0156110 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd898000f90 tx=0x7fd888005840 comp rx=0 tx=0).stop 2024-03-29T04:18:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.344+0000 7fd8a711c640 1 -- 172.21.15.22:0/4039694711 shutdown_connections 2024-03-29T04:18:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.344+0000 7fd8a711c640 1 --2- 172.21.15.22:0/4039694711 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd88c067fc0 0x7fd88c06a480 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.344+0000 7fd8a711c640 1 --2- 172.21.15.22:0/4039694711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8a01559b0 0x7fd8a0156110 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.344+0000 7fd8a711c640 1 --2- 172.21.15.22:0/4039694711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8a014dfe0 0x7fd8a0155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.344+0000 7fd8a711c640 1 --2- 172.21.15.22:0/4039694711 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd888004d50 0x7fd8a0160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.344+0000 7fd8a711c640 1 -- 172.21.15.22:0/4039694711 >> 172.21.15.22:0/4039694711 conn(0x7fd8a00b6410 msgr2=0x7fd8a00d1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.344+0000 7fd8a711c640 1 -- 172.21.15.22:0/4039694711 shutdown_connections 2024-03-29T04:18:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.344+0000 7fd8a711c640 1 -- 172.21.15.22:0/4039694711 wait complete. 2024-03-29T04:18:25.339 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:18:25.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.480+0000 7f498aa57640 1 Processor -- start 2024-03-29T04:18:25.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.480+0000 7f498aa57640 1 -- start start 2024-03-29T04:18:25.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.480+0000 7f498aa57640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f498413f140 0x7f49841606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.480+0000 7f498aa57640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f498406cd80 0x7f49840ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.480+0000 7f498aa57640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f498406d710 con 0x7f498415ff40 2024-03-29T04:18:25.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.480+0000 7f498aa57640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f498406e320 con 0x7f498413f140 2024-03-29T04:18:25.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.480+0000 7f498aa57640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f498406d9f0 con 0x7f498406cd80 2024-03-29T04:18:25.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.480+0000 7f4983fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f498413f140 0x7f49841606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.480+0000 7f497bfff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f498415ff40 0x7f498406c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:56992/0 (socket says 172.21.15.22:56992) 2024-03-29T04:18:25.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.480+0000 7f497bfff640 1 -- 172.21.15.22:0/3792288682 learned_addr learned my addr 172.21.15.22:0/3792288682 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:25.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.480+0000 7f4988ff4640 1 --2- 172.21.15.22:0/3792288682 >> v2:172.21.15.151:3300/0 conn(0x7f498406cd80 0x7f49840ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.480+0000 7f4983fff640 1 -- 172.21.15.22:0/3792288682 >> v2:172.21.15.151:3300/0 conn(0x7f498406cd80 msgr2=0x7f49840ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:25.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.480+0000 7f4983fff640 1 --2- 172.21.15.22:0/3792288682 >> v2:172.21.15.151:3300/0 conn(0x7f498406cd80 0x7f49840ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.480+0000 7f4983fff640 1 -- 172.21.15.22:0/3792288682 >> v1:172.21.15.22:6789/0 conn(0x7f498415ff40 legacy=0x7f498406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:25.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.480+0000 7f4983fff640 1 -- 172.21.15.22:0/3792288682 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498406dcd0 con 0x7f498413f140 2024-03-29T04:18:25.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.480+0000 7f4988ff4640 1 --2- 172.21.15.22:0/3792288682 >> v2:172.21.15.151:3300/0 conn(0x7f498406cd80 0x7f49840ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:25.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.484+0000 7f4983fff640 1 --2- 172.21.15.22:0/3792288682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f498413f140 0x7f49841606d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f498415c870 tx=0x7f497401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1080e0e03853f354 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:25.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.484+0000 7f49837fe640 1 -- 172.21.15.22:0/3792288682 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49740a7070 con 0x7f498413f140 2024-03-29T04:18:25.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.484+0000 7f49837fe640 1 -- 172.21.15.22:0/3792288682 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49740b0070 con 0x7f498413f140 2024-03-29T04:18:25.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.484+0000 7f49837fe640 1 -- 172.21.15.22:0/3792288682 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49740ab890 con 0x7f498413f140 2024-03-29T04:18:25.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.484+0000 7f498aa57640 1 -- 172.21.15.22:0/3792288682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f498413f140 msgr2=0x7f49841606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:25.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.484+0000 7f498aa57640 1 --2- 172.21.15.22:0/3792288682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f498413f140 0x7f49841606d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f498415c870 tx=0x7f497401c450 comp rx=0 tx=0).stop 2024-03-29T04:18:25.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.484+0000 7f498aa57640 1 -- 172.21.15.22:0/3792288682 shutdown_connections 2024-03-29T04:18:25.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.484+0000 7f498aa57640 1 --2- 172.21.15.22:0/3792288682 >> v2:172.21.15.151:3300/0 conn(0x7f498406cd80 0x7f49840ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.484+0000 7f498aa57640 1 --2- 172.21.15.22:0/3792288682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f498413f140 0x7f49841606d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.484+0000 7f498aa57640 1 -- 172.21.15.22:0/3792288682 >> 172.21.15.22:0/3792288682 conn(0x7f49840d00a0 msgr2=0x7f49840d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:25.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.484+0000 7f498aa57640 1 -- 172.21.15.22:0/3792288682 shutdown_connections 2024-03-29T04:18:25.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.484+0000 7f498aa57640 1 -- 172.21.15.22:0/3792288682 wait complete. 2024-03-29T04:18:25.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f498aa57640 1 Processor -- start 2024-03-29T04:18:25.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f498aa57640 1 -- start start 2024-03-29T04:18:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f498aa57640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f498406cd80 0x7f498413a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f498aa57640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f498413f140 0x7f49840721e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f498aa57640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f498415ff40 0x7f4984072720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f498aa57640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f49840cf810 con 0x7f498406cd80 2024-03-29T04:18:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f498aa57640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f49840cf690 con 0x7f498413f140 2024-03-29T04:18:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f498aa57640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f49840cf990 con 0x7f498415ff40 2024-03-29T04:18:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f4983fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f498406cd80 0x7f498413a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f4983fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f498406cd80 0x7f498413a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36684/0 (socket says 172.21.15.22:36684) 2024-03-29T04:18:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f4983fff640 1 -- 172.21.15.22:0/182768200 learned_addr learned my addr 172.21.15.22:0/182768200 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f497bfff640 1 --2- 172.21.15.22:0/182768200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f498413f140 0x7f49840721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f4988ff4640 1 --2- 172.21.15.22:0/182768200 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f498415ff40 0x7f4984072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f4983fff640 1 -- 172.21.15.22:0/182768200 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f498415ff40 msgr2=0x7f4984072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f4983fff640 1 --2- 172.21.15.22:0/182768200 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f498415ff40 0x7f4984072720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f4983fff640 1 -- 172.21.15.22:0/182768200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f498413f140 msgr2=0x7f49840721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f4983fff640 1 --2- 172.21.15.22:0/182768200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f498413f140 0x7f49840721e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f4983fff640 1 -- 172.21.15.22:0/182768200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498404fa00 con 0x7f498406cd80 2024-03-29T04:18:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f4983fff640 1 --2- 172.21.15.22:0/182768200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f498406cd80 0x7f498413a420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f49740ad040 tx=0x7f497401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f49817fa640 1 -- 172.21.15.22:0/182768200 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49740bf610 con 0x7f498406cd80 2024-03-29T04:18:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f49817fa640 1 -- 172.21.15.22:0/182768200 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49740bf7b0 con 0x7f498406cd80 2024-03-29T04:18:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f49817fa640 1 -- 172.21.15.22:0/182768200 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49740bfba0 con 0x7f498406cd80 2024-03-29T04:18:25.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f498aa57640 1 -- 172.21.15.22:0/182768200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4984153b00 con 0x7f498406cd80 2024-03-29T04:18:25.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.488+0000 7f498aa57640 1 -- 172.21.15.22:0/182768200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49841540c0 con 0x7f498406cd80 2024-03-29T04:18:25.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.492+0000 7f498aa57640 1 -- 172.21.15.22:0/182768200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4984068900 con 0x7f498406cd80 2024-03-29T04:18:25.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.492+0000 7f49817fa640 1 -- 172.21.15.22:0/182768200 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f49740bd020 con 0x7f498406cd80 2024-03-29T04:18:25.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.492+0000 7f49817fa640 1 --2- 172.21.15.22:0/182768200 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4964067fc0 0x7f496406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.492+0000 7f49817fa640 1 -- 172.21.15.22:0/182768200 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f4974120be0 con 0x7f498406cd80 2024-03-29T04:18:25.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.492+0000 7f497bfff640 1 --2- 172.21.15.22:0/182768200 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4964067fc0 0x7f496406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.492+0000 7f497bfff640 1 --2- 172.21.15.22:0/182768200 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4964067fc0 0x7f496406a480 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f498413d780 tx=0x7f496c007e90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:25.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.496+0000 7f49817fa640 1 -- 172.21.15.22:0/182768200 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f49740c6030 con 0x7f498406cd80 2024-03-29T04:18:25.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.660+0000 7f498aa57640 1 -- 172.21.15.22:0/182768200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f49841507e0 con 0x7f498406cd80 2024-03-29T04:18:25.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.660+0000 7f49817fa640 1 -- 172.21.15.22:0/182768200 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f4974016070 con 0x7f498406cd80 2024-03-29T04:18:25.641 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:25.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.660+0000 7f498aa57640 1 -- 172.21.15.22:0/182768200 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4964067fc0 msgr2=0x7f496406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:25.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.660+0000 7f498aa57640 1 --2- 172.21.15.22:0/182768200 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4964067fc0 0x7f496406a480 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f498413d780 tx=0x7f496c007e90 comp rx=0 tx=0).stop 2024-03-29T04:18:25.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.660+0000 7f498aa57640 1 -- 172.21.15.22:0/182768200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f498406cd80 msgr2=0x7f498413a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:25.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.660+0000 7f498aa57640 1 --2- 172.21.15.22:0/182768200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f498406cd80 0x7f498413a420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f49740ad040 tx=0x7f497401ac90 comp rx=0 tx=0).stop 2024-03-29T04:18:25.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.664+0000 7f498aa57640 1 -- 172.21.15.22:0/182768200 shutdown_connections 2024-03-29T04:18:25.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.664+0000 7f498aa57640 1 --2- 172.21.15.22:0/182768200 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f498415ff40 0x7f4984072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.664+0000 7f498aa57640 1 --2- 172.21.15.22:0/182768200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f498413f140 0x7f49840721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.664+0000 7f498aa57640 1 --2- 172.21.15.22:0/182768200 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4964067fc0 0x7f496406a480 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.664+0000 7f498aa57640 1 --2- 172.21.15.22:0/182768200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f498406cd80 0x7f498413a420 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.664+0000 7f498aa57640 1 -- 172.21.15.22:0/182768200 >> 172.21.15.22:0/182768200 conn(0x7f49840d00a0 msgr2=0x7f49840b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:25.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.664+0000 7f498aa57640 1 -- 172.21.15.22:0/182768200 shutdown_connections 2024-03-29T04:18:25.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.664+0000 7f498aa57640 1 -- 172.21.15.22:0/182768200 wait complete. 2024-03-29T04:18:25.661 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining' 2024-03-29T04:18:25.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1bbc69640 1 Processor -- start 2024-03-29T04:18:25.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1bbc69640 1 -- start start 2024-03-29T04:18:25.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1bbc69640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa1b4144270 0x7fa1b4144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1bbc69640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1b40b4ba0 con 0x7fa1b40b2ef0 2024-03-29T04:18:25.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1bbc69640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1b40b33f0 con 0x7fa1b415f7a0 2024-03-29T04:18:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1bbc69640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fa1b4144fa0 con 0x7fa1b4144270 2024-03-29T04:18:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b9204640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fa1b40b2ef0 0x7fa1b40b4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:56998/0 (socket says 172.21.15.22:56998) 2024-03-29T04:18:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b9204640 1 -- 172.21.15.22:0/933245537 learned_addr learned my addr 172.21.15.22:0/933245537 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b9a05640 1 --2- 172.21.15.22:0/933245537 >> v2:172.21.15.22:3300/0 conn(0x7fa1b4144270 0x7fa1b4144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b8a03640 1 -- 172.21.15.22:0/933245537 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 889703462 0 0) 0x7fa1b40b4ba0 con 0x7fa1b40b2ef0 2024-03-29T04:18:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b8a03640 1 -- 172.21.15.22:0/933245537 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1a0003530 con 0x7fa1b40b2ef0 2024-03-29T04:18:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b8a03640 1 -- 172.21.15.22:0/933245537 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 864968698 0 0) 0x7fa1a0003530 con 0x7fa1b40b2ef0 2024-03-29T04:18:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b8a03640 1 -- 172.21.15.22:0/933245537 >> v2:172.21.15.22:3300/0 conn(0x7fa1b4144270 msgr2=0x7fa1b4144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b8a03640 1 --2- 172.21.15.22:0/933245537 >> v2:172.21.15.22:3300/0 conn(0x7fa1b4144270 0x7fa1b4144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b8a03640 1 -- 172.21.15.22:0/933245537 >> v1:172.21.15.151:6789/0 conn(0x7fa1b415f7a0 legacy=0x7fa1b406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b8a03640 1 -- 172.21.15.22:0/933245537 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b40b36d0 con 0x7fa1b40b2ef0 2024-03-29T04:18:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b8a03640 1 -- 172.21.15.22:0/933245537 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7fa1a4001680 con 0x7fa1b40b2ef0 2024-03-29T04:18:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b8a03640 1 -- 172.21.15.22:0/933245537 >> v1:172.21.15.22:6789/0 conn(0x7fa1b40b2ef0 legacy=0x7fa1b40b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b8a03640 1 --2- 172.21.15.22:0/933245537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a0003a10 0x7fa1a0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b8a03640 1 --2- 172.21.15.22:0/933245537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a0004810 0x7fa1a0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b8a03640 1 --2- 172.21.15.22:0/933245537 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1a00051d0 0x7fa1a000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b8a03640 1 -- 172.21.15.22:0/933245537 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a000c0a0 con 0x7fa1a0004810 2024-03-29T04:18:25.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b8a03640 1 -- 172.21.15.22:0/933245537 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a000c250 con 0x7fa1a0003a10 2024-03-29T04:18:25.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b8a03640 1 -- 172.21.15.22:0/933245537 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a000c450 con 0x7fa1a00051d0 2024-03-29T04:18:25.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b9a05640 1 --2- 172.21.15.22:0/933245537 >> v2:172.21.15.22:3300/0 conn(0x7fa1b4144270 0x7fa1b4144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:18:25.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b9204640 1 --2- 172.21.15.22:0/933245537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a0004810 0x7fa1a0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1ba206640 1 --2- 172.21.15.22:0/933245537 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1a00051d0 0x7fa1a000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b9204640 1 -- 172.21.15.22:0/933245537 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1a00051d0 msgr2=0x7fa1a000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:25.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b9204640 1 --2- 172.21.15.22:0/933245537 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1a00051d0 0x7fa1a000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b9204640 1 -- 172.21.15.22:0/933245537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a0003a10 msgr2=0x7fa1a0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:25.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b9204640 1 --2- 172.21.15.22:0/933245537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a0003a10 0x7fa1a0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1b9204640 1 -- 172.21.15.22:0/933245537 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a0003530 con 0x7fa1a0004810 2024-03-29T04:18:25.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1ba206640 1 --2- 172.21.15.22:0/933245537 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1a00051d0 0x7fa1a000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:25.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.808+0000 7fa1ba206640 1 -- 172.21.15.22:0/933245537 reap_dead start 2024-03-29T04:18:25.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.812+0000 7fa1b9204640 1 --2- 172.21.15.22:0/933245537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a0004810 0x7fa1a0004c90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400c0e0 tx=0x7fa1a401f2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad47d11fd70518 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:25.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.812+0000 7fa1b8a03640 1 -- 172.21.15.22:0/933245537 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4007c30 con 0x7fa1a0004810 2024-03-29T04:18:25.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.812+0000 7fa1b8a03640 1 -- 172.21.15.22:0/933245537 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a40ac020 con 0x7fa1a0004810 2024-03-29T04:18:25.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.812+0000 7fa1b8a03640 1 -- 172.21.15.22:0/933245537 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a40058e0 con 0x7fa1a0004810 2024-03-29T04:18:25.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.812+0000 7fa1bbc69640 1 -- 172.21.15.22:0/933245537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a0004810 msgr2=0x7fa1a0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:25.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.812+0000 7fa1bbc69640 1 --2- 172.21.15.22:0/933245537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a0004810 0x7fa1a0004c90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400c0e0 tx=0x7fa1a401f2d0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.812+0000 7fa1bbc69640 1 -- 172.21.15.22:0/933245537 shutdown_connections 2024-03-29T04:18:25.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.812+0000 7fa1bbc69640 1 --2- 172.21.15.22:0/933245537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a0004810 0x7fa1a0004c90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.812+0000 7fa1bbc69640 1 -- 172.21.15.22:0/933245537 >> 172.21.15.22:0/933245537 conn(0x7fa1b40cfee0 msgr2=0x7fa1b40d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:25.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.812+0000 7fa1bbc69640 1 -- 172.21.15.22:0/933245537 shutdown_connections 2024-03-29T04:18:25.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.812+0000 7fa1bbc69640 1 -- 172.21.15.22:0/933245537 wait complete. 2024-03-29T04:18:25.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.812+0000 7fa1bbc69640 1 Processor -- start 2024-03-29T04:18:25.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.812+0000 7fa1bbc69640 1 -- start start 2024-03-29T04:18:25.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.812+0000 7fa1bbc69640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a0004810 0x7fa1b41f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.812+0000 7fa1bbc69640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1b40ca560 0x7fa1b406d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.812+0000 7fa1bbc69640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1b406dbb0 0x7fa1b40cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.812+0000 7fa1bbc69640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b40b3f70 con 0x7fa1a0004810 2024-03-29T04:18:25.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.812+0000 7fa1bbc69640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b40b3a10 con 0x7fa1b40ca560 2024-03-29T04:18:25.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.812+0000 7fa1bbc69640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa1b415fc30 con 0x7fa1b406dbb0 2024-03-29T04:18:25.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1b9a05640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a0004810 0x7fa1b41f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1b9a05640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a0004810 0x7fa1b41f3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36710/0 (socket says 172.21.15.22:36710) 2024-03-29T04:18:25.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1b9a05640 1 -- 172.21.15.22:0/2348246092 learned_addr learned my addr 172.21.15.22:0/2348246092 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:25.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1ba206640 1 --2- 172.21.15.22:0/2348246092 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1b406dbb0 0x7fa1b40cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1b9204640 1 --2- 172.21.15.22:0/2348246092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1b40ca560 0x7fa1b406d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1b9a05640 1 -- 172.21.15.22:0/2348246092 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1b406dbb0 msgr2=0x7fa1b40cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:25.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1b9a05640 1 --2- 172.21.15.22:0/2348246092 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1b406dbb0 0x7fa1b40cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1b9a05640 1 -- 172.21.15.22:0/2348246092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1b40ca560 msgr2=0x7fa1b406d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:25.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1b9a05640 1 --2- 172.21.15.22:0/2348246092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1b40ca560 0x7fa1b406d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:25.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1b9a05640 1 -- 172.21.15.22:0/2348246092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b40cec80 con 0x7fa1a0004810 2024-03-29T04:18:25.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1b9204640 1 --2- 172.21.15.22:0/2348246092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1b40ca560 0x7fa1b406d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:25.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1b9a05640 1 --2- 172.21.15.22:0/2348246092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a0004810 0x7fa1b41f3c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000eb90 tx=0x7fa1b000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:25.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1aa7fc640 1 -- 172.21.15.22:0/2348246092 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0017070 con 0x7fa1a0004810 2024-03-29T04:18:25.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1aa7fc640 1 -- 172.21.15.22:0/2348246092 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1b000ee90 con 0x7fa1a0004810 2024-03-29T04:18:25.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1bbc69640 1 -- 172.21.15.22:0/2348246092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b40c94c0 con 0x7fa1a0004810 2024-03-29T04:18:25.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1aa7fc640 1 -- 172.21.15.22:0/2348246092 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b000cdf0 con 0x7fa1a0004810 2024-03-29T04:18:25.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1bbc69640 1 -- 172.21.15.22:0/2348246092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b40c9a80 con 0x7fa1a0004810 2024-03-29T04:18:25.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1bbc69640 1 -- 172.21.15.22:0/2348246092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1b4068890 con 0x7fa1a0004810 2024-03-29T04:18:25.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1aa7fc640 1 -- 172.21.15.22:0/2348246092 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fa1b001a070 con 0x7fa1a0004810 2024-03-29T04:18:25.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1aa7fc640 1 --2- 172.21.15.22:0/2348246092 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa198067fc0 0x7fa19806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:25.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1aa7fc640 1 -- 172.21.15.22:0/2348246092 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fa1b0086ac0 con 0x7fa1a0004810 2024-03-29T04:18:25.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1b9204640 1 --2- 172.21.15.22:0/2348246092 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa198067fc0 0x7fa19806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:25.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.816+0000 7fa1b9204640 1 --2- 172.21.15.22:0/2348246092 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa198067fc0 0x7fa19806a480 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40ad000 tx=0x7fa1b40b35c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:25.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:25.820+0000 7fa1aa7fc640 1 -- 172.21.15.22:0/2348246092 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fa1b00566f0 con 0x7fa1a0004810 2024-03-29T04:18:26.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:26.044+0000 7fa1bbc69640 1 -- 172.21.15.22:0/2348246092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]} v 0) -- 0x7fa1b40722d0 con 0x7fa1a0004810 2024-03-29T04:18:26.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:26.952+0000 7fa1aa7fc640 1 -- 172.21.15.22:0/2348246092 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]}]=0 v71) ==== 150+0+0 (secure 0 0 0) 0x7fa1b005a9f0 con 0x7fa1a0004810 2024-03-29T04:18:26.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:26.956+0000 7fa1bbc69640 1 -- 172.21.15.22:0/2348246092 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa198067fc0 msgr2=0x7fa19806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:26.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:26.956+0000 7fa1bbc69640 1 --2- 172.21.15.22:0/2348246092 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa198067fc0 0x7fa19806a480 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40ad000 tx=0x7fa1b40b35c0 comp rx=0 tx=0).stop 2024-03-29T04:18:26.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:26.956+0000 7fa1bbc69640 1 -- 172.21.15.22:0/2348246092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a0004810 msgr2=0x7fa1b41f3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:26.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:26.956+0000 7fa1bbc69640 1 --2- 172.21.15.22:0/2348246092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a0004810 0x7fa1b41f3c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000eb90 tx=0x7fa1b000c520 comp rx=0 tx=0).stop 2024-03-29T04:18:26.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:26.956+0000 7fa1bbc69640 1 -- 172.21.15.22:0/2348246092 shutdown_connections 2024-03-29T04:18:26.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:26.956+0000 7fa1bbc69640 1 --2- 172.21.15.22:0/2348246092 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa1b406dbb0 0x7fa1b40cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:26.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:26.956+0000 7fa1bbc69640 1 --2- 172.21.15.22:0/2348246092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1b40ca560 0x7fa1b406d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:26.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:26.956+0000 7fa1bbc69640 1 --2- 172.21.15.22:0/2348246092 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa198067fc0 0x7fa19806a480 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:26.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:26.956+0000 7fa1bbc69640 1 --2- 172.21.15.22:0/2348246092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a0004810 0x7fa1b41f3c80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:26.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:26.956+0000 7fa1bbc69640 1 -- 172.21.15.22:0/2348246092 >> 172.21.15.22:0/2348246092 conn(0x7fa1b40cfee0 msgr2=0x7fa1b40cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:26.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:26.956+0000 7fa1bbc69640 1 -- 172.21.15.22:0/2348246092 shutdown_connections 2024-03-29T04:18:26.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:26.956+0000 7fa1bbc69640 1 -- 172.21.15.22:0/2348246092 wait complete. 2024-03-29T04:18:26.959 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-29T04:18:27.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.112+0000 7f189110e640 1 Processor -- start 2024-03-29T04:18:27.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.112+0000 7f189110e640 1 -- start start 2024-03-29T04:18:27.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.112+0000 7f189110e640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f188c1605d0 0x7f188c0b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:27.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.112+0000 7f189110e640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f188c0cccb0 0x7f188c06f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:27.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.112+0000 7f189110e640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f188c06e040 con 0x7f188c1459b0 2024-03-29T04:18:27.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.112+0000 7f189110e640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f188c06e990 con 0x7f188c1605d0 2024-03-29T04:18:27.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.112+0000 7f189110e640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f188c06ec70 con 0x7f188c0cccb0 2024-03-29T04:18:27.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.112+0000 7f188ad9d640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f188c1459b0 0x7f188c15fec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.22:35632/0 (socket says 172.21.15.22:35632) 2024-03-29T04:18:27.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.112+0000 7f188ad9d640 1 -- 172.21.15.22:0/2683145605 learned_addr learned my addr 172.21.15.22:0/2683145605 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:27.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.112+0000 7f188a59c640 1 --2- 172.21.15.22:0/2683145605 >> v2:172.21.15.151:3300/0 conn(0x7f188c1605d0 0x7f188c0b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:27.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.112+0000 7f188b59e640 1 --2- 172.21.15.22:0/2683145605 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f188c0cccb0 0x7f188c06f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:27.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.112+0000 7f1889d9b640 1 -- 172.21.15.22:0/2683145605 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1924846307 0 0) 0x7f188c06e040 con 0x7f188c1459b0 2024-03-29T04:18:27.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.112+0000 7f1889d9b640 1 -- 172.21.15.22:0/2683145605 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1874003530 con 0x7f188c1459b0 2024-03-29T04:18:27.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.112+0000 7f188b59e640 1 -- 172.21.15.22:0/2683145605 >> v2:172.21.15.151:3300/0 conn(0x7f188c1605d0 msgr2=0x7f188c0b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:27.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.112+0000 7f188b59e640 1 --2- 172.21.15.22:0/2683145605 >> v2:172.21.15.151:3300/0 conn(0x7f188c1605d0 0x7f188c0b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.112+0000 7f188b59e640 1 -- 172.21.15.22:0/2683145605 >> v1:172.21.15.151:6789/0 conn(0x7f188c1459b0 legacy=0x7f188c15fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:27.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.112+0000 7f188b59e640 1 -- 172.21.15.22:0/2683145605 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188c06ef50 con 0x7f188c0cccb0 2024-03-29T04:18:27.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.116+0000 7f188b59e640 1 --2- 172.21.15.22:0/2683145605 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f188c0cccb0 0x7f188c06f470 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f187c0097c0 tx=0x7f187c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b66ccc69e8101cf4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:27.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.116+0000 7f1889d9b640 1 -- 172.21.15.22:0/2683145605 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c0b0070 con 0x7f188c0cccb0 2024-03-29T04:18:27.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.116+0000 7f1889d9b640 1 -- 172.21.15.22:0/2683145605 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f187c0ab600 con 0x7f188c0cccb0 2024-03-29T04:18:27.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.116+0000 7f1889d9b640 1 -- 172.21.15.22:0/2683145605 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c0ab9d0 con 0x7f188c0cccb0 2024-03-29T04:18:27.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.116+0000 7f189110e640 1 -- 172.21.15.22:0/2683145605 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f188c0cccb0 msgr2=0x7f188c06f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:27.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.116+0000 7f189110e640 1 --2- 172.21.15.22:0/2683145605 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f188c0cccb0 0x7f188c06f470 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f187c0097c0 tx=0x7f187c01c4a0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.116+0000 7f189110e640 1 -- 172.21.15.22:0/2683145605 shutdown_connections 2024-03-29T04:18:27.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.116+0000 7f189110e640 1 --2- 172.21.15.22:0/2683145605 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f188c0cccb0 0x7f188c06f470 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.116+0000 7f189110e640 1 --2- 172.21.15.22:0/2683145605 >> v2:172.21.15.151:3300/0 conn(0x7f188c1605d0 0x7f188c0b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.116+0000 7f189110e640 1 -- 172.21.15.22:0/2683145605 >> 172.21.15.22:0/2683145605 conn(0x7f188c0b2c70 msgr2=0x7f188c0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:27.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.116+0000 7f189110e640 1 -- 172.21.15.22:0/2683145605 shutdown_connections 2024-03-29T04:18:27.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.116+0000 7f189110e640 1 -- 172.21.15.22:0/2683145605 wait complete. 2024-03-29T04:18:27.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f189110e640 1 Processor -- start 2024-03-29T04:18:27.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f189110e640 1 -- start start 2024-03-29T04:18:27.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f189110e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f188c0cccb0 0x7f188c140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:27.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f189110e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f188c1459b0 0x7f188c140080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:27.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f189110e640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f188c1605d0 0x7f188c15a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:27.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f189110e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f188c06dbd0 con 0x7f188c1459b0 2024-03-29T04:18:27.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f189110e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f188c145fa0 con 0x7f188c0cccb0 2024-03-29T04:18:27.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f189110e640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f188c06dd50 con 0x7f188c1605d0 2024-03-29T04:18:27.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f188a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f188c1459b0 0x7f188c140080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:27.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f188a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f188c1459b0 0x7f188c140080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36716/0 (socket says 172.21.15.22:36716) 2024-03-29T04:18:27.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f188a59c640 1 -- 172.21.15.22:0/2207897783 learned_addr learned my addr 172.21.15.22:0/2207897783 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:27.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f188ad9d640 1 --2- 172.21.15.22:0/2207897783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f188c0cccb0 0x7f188c140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:27.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f188b59e640 1 --2- 172.21.15.22:0/2207897783 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f188c1605d0 0x7f188c15a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:27.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f188a59c640 1 -- 172.21.15.22:0/2207897783 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f188c1605d0 msgr2=0x7f188c15a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:27.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f188a59c640 1 --2- 172.21.15.22:0/2207897783 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f188c1605d0 0x7f188c15a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f188a59c640 1 -- 172.21.15.22:0/2207897783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f188c0cccb0 msgr2=0x7f188c140ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:27.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f188a59c640 1 --2- 172.21.15.22:0/2207897783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f188c0cccb0 0x7f188c140ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f188a59c640 1 -- 172.21.15.22:0/2207897783 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188c1405c0 con 0x7f188c1459b0 2024-03-29T04:18:27.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f188a59c640 1 --2- 172.21.15.22:0/2207897783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f188c1459b0 0x7f188c140080 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f188000e960 tx=0x7f188000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:27.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f18737fe640 1 -- 172.21.15.22:0/2207897783 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1880017070 con 0x7f188c1459b0 2024-03-29T04:18:27.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f18737fe640 1 -- 172.21.15.22:0/2207897783 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f188000ca00 con 0x7f188c1459b0 2024-03-29T04:18:27.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f18737fe640 1 -- 172.21.15.22:0/2207897783 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188000cdb0 con 0x7f188c1459b0 2024-03-29T04:18:27.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f189110e640 1 -- 172.21.15.22:0/2207897783 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188c0ca640 con 0x7f188c1459b0 2024-03-29T04:18:27.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f189110e640 1 -- 172.21.15.22:0/2207897783 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f188c0cac00 con 0x7f188c1459b0 2024-03-29T04:18:27.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f189110e640 1 -- 172.21.15.22:0/2207897783 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f188c15fc30 con 0x7f188c1459b0 2024-03-29T04:18:27.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.120+0000 7f18737fe640 1 -- 172.21.15.22:0/2207897783 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f188001a070 con 0x7f188c1459b0 2024-03-29T04:18:27.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.124+0000 7f18737fe640 1 --2- 172.21.15.22:0/2207897783 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1878067fc0 0x7f187806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:27.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.124+0000 7f18737fe640 1 -- 172.21.15.22:0/2207897783 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f1880086810 con 0x7f188c1459b0 2024-03-29T04:18:27.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.124+0000 7f188ad9d640 1 --2- 172.21.15.22:0/2207897783 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1878067fc0 0x7f187806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:27.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.124+0000 7f188ad9d640 1 --2- 172.21.15.22:0/2207897783 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1878067fc0 0x7f187806a480 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1884005c10 tx=0x7f1884009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:27.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.128+0000 7f18737fe640 1 -- 172.21.15.22:0/2207897783 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1880056600 con 0x7f188c1459b0 2024-03-29T04:18:27.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.288+0000 7f189110e640 1 -- 172.21.15.22:0/2207897783 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f188c0cb650 con 0x7f188c1459b0 2024-03-29T04:18:27.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.292+0000 7f18737fe640 1 -- 172.21.15.22:0/2207897783 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f188005a900 con 0x7f188c1459b0 2024-03-29T04:18:27.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.296+0000 7f189110e640 1 -- 172.21.15.22:0/2207897783 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1878067fc0 msgr2=0x7f187806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:27.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.296+0000 7f189110e640 1 --2- 172.21.15.22:0/2207897783 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1878067fc0 0x7f187806a480 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1884005c10 tx=0x7f1884009040 comp rx=0 tx=0).stop 2024-03-29T04:18:27.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.296+0000 7f189110e640 1 -- 172.21.15.22:0/2207897783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f188c1459b0 msgr2=0x7f188c140080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:27.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.296+0000 7f189110e640 1 --2- 172.21.15.22:0/2207897783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f188c1459b0 0x7f188c140080 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f188000e960 tx=0x7f188000ee20 comp rx=0 tx=0).stop 2024-03-29T04:18:27.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.296+0000 7f189110e640 1 -- 172.21.15.22:0/2207897783 shutdown_connections 2024-03-29T04:18:27.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.296+0000 7f189110e640 1 --2- 172.21.15.22:0/2207897783 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f188c1605d0 0x7f188c15a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.296+0000 7f189110e640 1 --2- 172.21.15.22:0/2207897783 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1878067fc0 0x7f187806a480 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.296+0000 7f189110e640 1 --2- 172.21.15.22:0/2207897783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f188c1459b0 0x7f188c140080 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.296+0000 7f189110e640 1 --2- 172.21.15.22:0/2207897783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f188c0cccb0 0x7f188c140ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.296+0000 7f189110e640 1 -- 172.21.15.22:0/2207897783 >> 172.21.15.22:0/2207897783 conn(0x7f188c0b2c70 msgr2=0x7f188c144bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:27.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.296+0000 7f189110e640 1 -- 172.21.15.22:0/2207897783 shutdown_connections 2024-03-29T04:18:27.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.296+0000 7f189110e640 1 -- 172.21.15.22:0/2207897783 wait complete. 2024-03-29T04:18:27.292 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T04:18:27.292 DEBUG:teuthology.orchestra.run.smithi022:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T04:18:27.345 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T04:18:27.375 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T04:18:27.375 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:18:27.375 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:18:27.381 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:18:27.381 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:18:27.427 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:27.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.616+0000 7fdad4b1d640 1 Processor -- start 2024-03-29T04:18:27.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.616+0000 7fdad4b1d640 1 -- start start 2024-03-29T04:18:27.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.616+0000 7fdad4b1d640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdad0145e10 0x7fdad00cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:27.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.616+0000 7fdad4b1d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fdad00d0220 0x7fdad015fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:27.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.616+0000 7fdad4b1d640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdad00d06c0 con 0x7fdad0160a90 2024-03-29T04:18:27.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.616+0000 7fdad4b1d640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fdad00d09a0 con 0x7fdad00d0220 2024-03-29T04:18:27.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.616+0000 7fdad4b1d640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fdad00d0c80 con 0x7fdad0145e10 2024-03-29T04:18:27.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.616+0000 7fdacdd9b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fdad00d0220 0x7fdad015fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:27.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.616+0000 7fdacdd9b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fdad00d0220 0x7fdad015fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36718/0 (socket says 172.21.15.22:36718) 2024-03-29T04:18:27.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.616+0000 7fdacdd9b640 1 -- 172.21.15.22:0/609261237 learned_addr learned my addr 172.21.15.22:0/609261237 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:27.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.616+0000 7fdacdd9b640 1 -- 172.21.15.22:0/609261237 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdad0145e10 msgr2=0x7fdad00cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:27.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.616+0000 7fdace59c640 1 --2- 172.21.15.22:0/609261237 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdad0145e10 0x7fdad00cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:27.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.616+0000 7fdacdd9b640 1 --2- 172.21.15.22:0/609261237 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdad0145e10 0x7fdad00cfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.616+0000 7fdacdd9b640 1 -- 172.21.15.22:0/609261237 >> v1:172.21.15.22:6789/0 conn(0x7fdad0160a90 legacy=0x7fdad00b6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:27.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.616+0000 7fdacdd9b640 1 -- 172.21.15.22:0/609261237 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad00d0f60 con 0x7fdad00d0220 2024-03-29T04:18:27.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdacdd9b640 1 --2- 172.21.15.22:0/609261237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdad00d0220 0x7fdad015fd60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009870 tx=0x7fdac401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fec648911c989c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:27.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdacd59a640 1 -- 172.21.15.22:0/609261237 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac40a7070 con 0x7fdad00d0220 2024-03-29T04:18:27.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdacd59a640 1 -- 172.21.15.22:0/609261237 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac40b0070 con 0x7fdad00d0220 2024-03-29T04:18:27.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdacd59a640 1 -- 172.21.15.22:0/609261237 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac40ab890 con 0x7fdad00d0220 2024-03-29T04:18:27.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdad4b1d640 1 -- 172.21.15.22:0/609261237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdad00d0220 msgr2=0x7fdad015fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:27.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdad4b1d640 1 --2- 172.21.15.22:0/609261237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdad00d0220 0x7fdad015fd60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009870 tx=0x7fdac401c560 comp rx=0 tx=0).stop 2024-03-29T04:18:27.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdad4b1d640 1 -- 172.21.15.22:0/609261237 shutdown_connections 2024-03-29T04:18:27.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdad4b1d640 1 --2- 172.21.15.22:0/609261237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdad00d0220 0x7fdad015fd60 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdad4b1d640 1 --2- 172.21.15.22:0/609261237 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdad0145e10 0x7fdad00cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdad4b1d640 1 -- 172.21.15.22:0/609261237 >> 172.21.15.22:0/609261237 conn(0x7fdad00ceb90 msgr2=0x7fdad00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:27.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdad4b1d640 1 -- 172.21.15.22:0/609261237 shutdown_connections 2024-03-29T04:18:27.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdad4b1d640 1 -- 172.21.15.22:0/609261237 wait complete. 2024-03-29T04:18:27.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdad4b1d640 1 Processor -- start 2024-03-29T04:18:27.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdad4b1d640 1 -- start start 2024-03-29T04:18:27.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdad4b1d640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdad00d0220 0x7fdad0159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:27.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdad4b1d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdad0145e10 0x7fdad01f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:27.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdad4b1d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdad0160a90 0x7fdad0141760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:27.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdad4b1d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdad0169490 con 0x7fdad0160a90 2024-03-29T04:18:27.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdad4b1d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdad0169310 con 0x7fdad0145e10 2024-03-29T04:18:27.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdad4b1d640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fdad0169610 con 0x7fdad00d0220 2024-03-29T04:18:27.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdaced9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdad0160a90 0x7fdad0141760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:27.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdaced9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdad0160a90 0x7fdad0141760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36734/0 (socket says 172.21.15.22:36734) 2024-03-29T04:18:27.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.620+0000 7fdaced9d640 1 -- 172.21.15.22:0/934101081 learned_addr learned my addr 172.21.15.22:0/934101081 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.624+0000 7fdace59c640 1 --2- 172.21.15.22:0/934101081 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdad00d0220 0x7fdad0159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.624+0000 7fdacdd9b640 1 --2- 172.21.15.22:0/934101081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdad0145e10 0x7fdad01f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.624+0000 7fdaced9d640 1 -- 172.21.15.22:0/934101081 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdad00d0220 msgr2=0x7fdad0159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.624+0000 7fdaced9d640 1 --2- 172.21.15.22:0/934101081 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdad00d0220 0x7fdad0159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.624+0000 7fdaced9d640 1 -- 172.21.15.22:0/934101081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdad0145e10 msgr2=0x7fdad01f65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.624+0000 7fdaced9d640 1 --2- 172.21.15.22:0/934101081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdad0145e10 0x7fdad01f65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.624+0000 7fdaced9d640 1 -- 172.21.15.22:0/934101081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad01f6ae0 con 0x7fdad0160a90 2024-03-29T04:18:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.624+0000 7fdaced9d640 1 --2- 172.21.15.22:0/934101081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdad0160a90 0x7fdad0141760 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdac800c730 tx=0x7fdac800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.624+0000 7fdabeffd640 1 -- 172.21.15.22:0/934101081 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8003b80 con 0x7fdad0160a90 2024-03-29T04:18:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.624+0000 7fdabeffd640 1 -- 172.21.15.22:0/934101081 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac8003d20 con 0x7fdad0160a90 2024-03-29T04:18:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.624+0000 7fdad4b1d640 1 -- 172.21.15.22:0/934101081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad0161410 con 0x7fdad0160a90 2024-03-29T04:18:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.624+0000 7fdabeffd640 1 -- 172.21.15.22:0/934101081 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8005310 con 0x7fdad0160a90 2024-03-29T04:18:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.624+0000 7fdad4b1d640 1 -- 172.21.15.22:0/934101081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad015d700 con 0x7fdad0160a90 2024-03-29T04:18:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.624+0000 7fdad4b1d640 1 -- 172.21.15.22:0/934101081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad015d210 con 0x7fdad0160a90 2024-03-29T04:18:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.624+0000 7fdabeffd640 1 -- 172.21.15.22:0/934101081 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fdac80054b0 con 0x7fdad0160a90 2024-03-29T04:18:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.624+0000 7fdabeffd640 1 --2- 172.21.15.22:0/934101081 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdab0068040 0x7fdab006a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.624+0000 7fdabeffd640 1 -- 172.21.15.22:0/934101081 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fdac8086630 con 0x7fdad0160a90 2024-03-29T04:18:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.624+0000 7fdace59c640 1 --2- 172.21.15.22:0/934101081 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdab0068040 0x7fdab006a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:27.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.624+0000 7fdace59c640 1 --2- 172.21.15.22:0/934101081 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdab0068040 0x7fdab006a500 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdad01f3710 tx=0x7fdac0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:27.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.628+0000 7fdabeffd640 1 -- 172.21.15.22:0/934101081 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fdad01f6ae0 con 0x7fdad0160a90 2024-03-29T04:18:27.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.792+0000 7fdad4b1d640 1 -- 172.21.15.22:0/934101081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdad00b5940 con 0x7fdad0160a90 2024-03-29T04:18:27.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.796+0000 7fdabeffd640 1 -- 172.21.15.22:0/934101081 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18847 (secure 0 0 0) 0x7fdac8056ac0 con 0x7fdad0160a90 2024-03-29T04:18:27.777 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:27.777 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:18:16.893439+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:27.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.800+0000 7fdad4b1d640 1 -- 172.21.15.22:0/934101081 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdab0068040 msgr2=0x7fdab006a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:27.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.800+0000 7fdad4b1d640 1 --2- 172.21.15.22:0/934101081 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdab0068040 0x7fdab006a500 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdad01f3710 tx=0x7fdac0008040 comp rx=0 tx=0).stop 2024-03-29T04:18:27.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.800+0000 7fdad4b1d640 1 -- 172.21.15.22:0/934101081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdad0160a90 msgr2=0x7fdad0141760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:27.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.800+0000 7fdad4b1d640 1 --2- 172.21.15.22:0/934101081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdad0160a90 0x7fdad0141760 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdac800c730 tx=0x7fdac800cbf0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.800+0000 7fdad4b1d640 1 -- 172.21.15.22:0/934101081 shutdown_connections 2024-03-29T04:18:27.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.800+0000 7fdad4b1d640 1 --2- 172.21.15.22:0/934101081 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdab0068040 0x7fdab006a500 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.800+0000 7fdad4b1d640 1 --2- 172.21.15.22:0/934101081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdad0160a90 0x7fdad0141760 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.800+0000 7fdad4b1d640 1 --2- 172.21.15.22:0/934101081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdad0145e10 0x7fdad01f65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.800+0000 7fdad4b1d640 1 --2- 172.21.15.22:0/934101081 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdad00d0220 0x7fdad0159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.800+0000 7fdad4b1d640 1 -- 172.21.15.22:0/934101081 >> 172.21.15.22:0/934101081 conn(0x7fdad00ceb90 msgr2=0x7fdad00b4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:27.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.800+0000 7fdad4b1d640 1 -- 172.21.15.22:0/934101081 shutdown_connections 2024-03-29T04:18:27.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.800+0000 7fdad4b1d640 1 -- 172.21.15.22:0/934101081 wait complete. 2024-03-29T04:18:27.797 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:17:34.034142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:17:34.886659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:17:38.628632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:18:27.797 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:18:27.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.948+0000 7fd0584c9640 1 Processor -- start 2024-03-29T04:18:27.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.948+0000 7fd0584c9640 1 -- start start 2024-03-29T04:18:27.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.948+0000 7fd0584c9640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd05015e4c0 0x7fd0500b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:27.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.948+0000 7fd0584c9640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd0500cf6c0 0x7fd0500d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:27.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.948+0000 7fd0584c9640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd05006c850 con 0x7fd05006c070 2024-03-29T04:18:27.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.948+0000 7fd0584c9640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fd05015e8a0 con 0x7fd0500cf6c0 2024-03-29T04:18:27.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.948+0000 7fd0584c9640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fd05006cb30 con 0x7fd05015e4c0 2024-03-29T04:18:27.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.948+0000 7fd056a66640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd0500cf6c0 0x7fd0500d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:27.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.948+0000 7fd056a66640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd0500cf6c0 0x7fd0500d2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36746/0 (socket says 172.21.15.22:36746) 2024-03-29T04:18:27.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.948+0000 7fd056a66640 1 -- 172.21.15.22:0/2082920203 learned_addr learned my addr 172.21.15.22:0/2082920203 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:27.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.948+0000 7fd056265640 1 --2- 172.21.15.22:0/2082920203 >> v2:172.21.15.151:3300/0 conn(0x7fd05015e4c0 0x7fd0500b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:27.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.948+0000 7fd056a66640 1 -- 172.21.15.22:0/2082920203 >> v2:172.21.15.151:3300/0 conn(0x7fd05015e4c0 msgr2=0x7fd0500b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:27.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.948+0000 7fd056a66640 1 --2- 172.21.15.22:0/2082920203 >> v2:172.21.15.151:3300/0 conn(0x7fd05015e4c0 0x7fd0500b9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.948+0000 7fd056a66640 1 -- 172.21.15.22:0/2082920203 >> v1:172.21.15.151:6789/0 conn(0x7fd05006c070 legacy=0x7fd0500cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:27.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.948+0000 7fd056a66640 1 -- 172.21.15.22:0/2082920203 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05006ce10 con 0x7fd0500cf6c0 2024-03-29T04:18:27.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd056a66640 1 --2- 172.21.15.22:0/2082920203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0500cf6c0 0x7fd0500d2d70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd0500d0cc0 tx=0x7fd04801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83fe0d152823da01 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:27.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd055263640 1 -- 172.21.15.22:0/2082920203 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0480b0070 con 0x7fd0500cf6c0 2024-03-29T04:18:27.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd055263640 1 -- 172.21.15.22:0/2082920203 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0480ab600 con 0x7fd0500cf6c0 2024-03-29T04:18:27.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd055263640 1 -- 172.21.15.22:0/2082920203 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0480ab9d0 con 0x7fd0500cf6c0 2024-03-29T04:18:27.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd0584c9640 1 -- 172.21.15.22:0/2082920203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0500cf6c0 msgr2=0x7fd0500d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:27.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd0584c9640 1 --2- 172.21.15.22:0/2082920203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0500cf6c0 0x7fd0500d2d70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd0500d0cc0 tx=0x7fd04801c520 comp rx=0 tx=0).stop 2024-03-29T04:18:27.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd0584c9640 1 -- 172.21.15.22:0/2082920203 shutdown_connections 2024-03-29T04:18:27.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd0584c9640 1 --2- 172.21.15.22:0/2082920203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0500cf6c0 0x7fd0500d2d70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd0584c9640 1 --2- 172.21.15.22:0/2082920203 >> v2:172.21.15.151:3300/0 conn(0x7fd05015e4c0 0x7fd0500b9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd0584c9640 1 -- 172.21.15.22:0/2082920203 >> 172.21.15.22:0/2082920203 conn(0x7fd0500b6410 msgr2=0x7fd0500d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:27.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd0584c9640 1 -- 172.21.15.22:0/2082920203 shutdown_connections 2024-03-29T04:18:27.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd0584c9640 1 -- 172.21.15.22:0/2082920203 wait complete. 2024-03-29T04:18:27.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd0584c9640 1 Processor -- start 2024-03-29T04:18:27.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd0584c9640 1 -- start start 2024-03-29T04:18:27.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd0584c9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd05006c070 0x7fd0501537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:27.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd0584c9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0500cf6c0 0x7fd05015d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:27.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd0584c9640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd05015e4c0 0x7fd05015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:27.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd0584c9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd0500d3e90 con 0x7fd0500cf6c0 2024-03-29T04:18:27.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd0584c9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd0500d3d10 con 0x7fd05006c070 2024-03-29T04:18:27.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd0584c9640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd0500d4010 con 0x7fd05015e4c0 2024-03-29T04:18:27.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd055a64640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0500cf6c0 0x7fd05015d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:27.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd055a64640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0500cf6c0 0x7fd05015d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36762/0 (socket says 172.21.15.22:36762) 2024-03-29T04:18:27.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd055a64640 1 -- 172.21.15.22:0/3255337190 learned_addr learned my addr 172.21.15.22:0/3255337190 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:27.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd056265640 1 --2- 172.21.15.22:0/3255337190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd05006c070 0x7fd0501537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:27.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd056a66640 1 --2- 172.21.15.22:0/3255337190 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd05015e4c0 0x7fd05015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:27.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd055a64640 1 -- 172.21.15.22:0/3255337190 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd05015e4c0 msgr2=0x7fd05015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:27.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd055a64640 1 --2- 172.21.15.22:0/3255337190 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd05015e4c0 0x7fd05015a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd055a64640 1 -- 172.21.15.22:0/3255337190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd05006c070 msgr2=0x7fd0501537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:27.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd055a64640 1 --2- 172.21.15.22:0/3255337190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd05006c070 0x7fd0501537b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:27.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd055a64640 1 -- 172.21.15.22:0/3255337190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0500c8310 con 0x7fd0500cf6c0 2024-03-29T04:18:27.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd056265640 1 --2- 172.21.15.22:0/3255337190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd05006c070 0x7fd0501537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:27.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.952+0000 7fd055a64640 1 --2- 172.21.15.22:0/3255337190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0500cf6c0 0x7fd05015d0e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd04000c420 tx=0x7fd04000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:27.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.956+0000 7fd046ffd640 1 -- 172.21.15.22:0/3255337190 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0400052d0 con 0x7fd0500cf6c0 2024-03-29T04:18:27.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.956+0000 7fd046ffd640 1 -- 172.21.15.22:0/3255337190 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd040005470 con 0x7fd0500cf6c0 2024-03-29T04:18:27.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.956+0000 7fd0584c9640 1 -- 172.21.15.22:0/3255337190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0500c85f0 con 0x7fd0500cf6c0 2024-03-29T04:18:27.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.956+0000 7fd046ffd640 1 -- 172.21.15.22:0/3255337190 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd040007260 con 0x7fd0500cf6c0 2024-03-29T04:18:27.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.956+0000 7fd0584c9640 1 -- 172.21.15.22:0/3255337190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0500c8bb0 con 0x7fd0500cf6c0 2024-03-29T04:18:27.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.956+0000 7fd0584c9640 1 -- 172.21.15.22:0/3255337190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd050068900 con 0x7fd0500cf6c0 2024-03-29T04:18:27.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.956+0000 7fd046ffd640 1 -- 172.21.15.22:0/3255337190 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fd040007400 con 0x7fd0500cf6c0 2024-03-29T04:18:27.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.956+0000 7fd046ffd640 1 --2- 172.21.15.22:0/3255337190 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd034067fc0 0x7fd03406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:27.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.956+0000 7fd046ffd640 1 -- 172.21.15.22:0/3255337190 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fd040085e80 con 0x7fd0500cf6c0 2024-03-29T04:18:27.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.956+0000 7fd056265640 1 --2- 172.21.15.22:0/3255337190 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd034067fc0 0x7fd03406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:27.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.956+0000 7fd056265640 1 --2- 172.21.15.22:0/3255337190 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd034067fc0 0x7fd03406a480 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009fd0 tx=0x7fd04c00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:27.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:27.960+0000 7fd046ffd640 1 -- 172.21.15.22:0/3255337190 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fd040056210 con 0x7fd0500cf6c0 2024-03-29T04:18:28.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.124+0000 7fd0584c9640 1 -- 172.21.15.22:0/3255337190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd0500b8ec0 con 0x7fd0500cf6c0 2024-03-29T04:18:28.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.124+0000 7fd046ffd640 1 -- 172.21.15.22:0/3255337190 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7fd040055b10 con 0x7fd0500cf6c0 2024-03-29T04:18:28.104 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:18:28.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.124+0000 7fd0584c9640 1 -- 172.21.15.22:0/3255337190 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd034067fc0 msgr2=0x7fd03406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:28.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.124+0000 7fd0584c9640 1 --2- 172.21.15.22:0/3255337190 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd034067fc0 0x7fd03406a480 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009fd0 tx=0x7fd04c00b040 comp rx=0 tx=0).stop 2024-03-29T04:18:28.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.124+0000 7fd0584c9640 1 -- 172.21.15.22:0/3255337190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0500cf6c0 msgr2=0x7fd05015d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:28.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.124+0000 7fd0584c9640 1 --2- 172.21.15.22:0/3255337190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0500cf6c0 0x7fd05015d0e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd04000c420 tx=0x7fd04000c8e0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.124+0000 7fd0584c9640 1 -- 172.21.15.22:0/3255337190 shutdown_connections 2024-03-29T04:18:28.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.124+0000 7fd0584c9640 1 --2- 172.21.15.22:0/3255337190 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd05015e4c0 0x7fd05015a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.124+0000 7fd0584c9640 1 --2- 172.21.15.22:0/3255337190 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd034067fc0 0x7fd03406a480 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.124+0000 7fd0584c9640 1 --2- 172.21.15.22:0/3255337190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0500cf6c0 0x7fd05015d0e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.124+0000 7fd0584c9640 1 --2- 172.21.15.22:0/3255337190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd05006c070 0x7fd0501537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.124+0000 7fd0584c9640 1 -- 172.21.15.22:0/3255337190 >> 172.21.15.22:0/3255337190 conn(0x7fd0500b6410 msgr2=0x7fd0500b85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:28.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.124+0000 7fd0584c9640 1 -- 172.21.15.22:0/3255337190 shutdown_connections 2024-03-29T04:18:28.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.124+0000 7fd0584c9640 1 -- 172.21.15.22:0/3255337190 wait complete. 2024-03-29T04:18:28.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:18:28.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.272+0000 7f0116eab640 1 Processor -- start 2024-03-29T04:18:28.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.272+0000 7f0116eab640 1 -- start start 2024-03-29T04:18:28.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.272+0000 7f0116eab640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f011015a900 0x7f01100afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:28.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.272+0000 7f0116eab640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f011006c070 0x7f011006eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:28.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.272+0000 7f0116eab640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f011006f2e0 0x7f011006db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:28.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.272+0000 7f0116eab640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f011006f750 con 0x7f011006c070 2024-03-29T04:18:28.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.272+0000 7f0116eab640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f011015ace0 con 0x7f011015a900 2024-03-29T04:18:28.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.272+0000 7f0116eab640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f011006c570 con 0x7f011006f2e0 2024-03-29T04:18:28.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.272+0000 7f010ffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f011006c070 0x7f011006eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:28.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.272+0000 7f010ffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f011006c070 0x7f011006eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36768/0 (socket says 172.21.15.22:36768) 2024-03-29T04:18:28.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.272+0000 7f010ffff640 1 -- 172.21.15.22:0/494917689 learned_addr learned my addr 172.21.15.22:0/494917689 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:28.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.272+0000 7f0114c47640 1 --2- 172.21.15.22:0/494917689 >> v2:172.21.15.151:3300/0 conn(0x7f011015a900 0x7f01100afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:28.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.272+0000 7f0115448640 1 --2- 172.21.15.22:0/494917689 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f011006f2e0 0x7f011006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:28.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.272+0000 7f010ffff640 1 -- 172.21.15.22:0/494917689 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f011006f2e0 msgr2=0x7f011006db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:28.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.272+0000 7f010ffff640 1 --2- 172.21.15.22:0/494917689 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f011006f2e0 0x7f011006db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.272+0000 7f010ffff640 1 -- 172.21.15.22:0/494917689 >> v2:172.21.15.151:3300/0 conn(0x7f011015a900 msgr2=0x7f01100afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:28.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.272+0000 7f010ffff640 1 --2- 172.21.15.22:0/494917689 >> v2:172.21.15.151:3300/0 conn(0x7f011015a900 0x7f01100afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.272+0000 7f010ffff640 1 -- 172.21.15.22:0/494917689 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011006c850 con 0x7f011006c070 2024-03-29T04:18:28.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.276+0000 7f010ffff640 1 --2- 172.21.15.22:0/494917689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f011006c070 0x7f011006eda0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f01100b50a0 tx=0x7f00f801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3be677963ffdb2f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:28.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.276+0000 7f010f7fe640 1 -- 172.21.15.22:0/494917689 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f80ab430 con 0x7f011006c070 2024-03-29T04:18:28.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.276+0000 7f010f7fe640 1 -- 172.21.15.22:0/494917689 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f80b0070 con 0x7f011006c070 2024-03-29T04:18:28.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.276+0000 7f010f7fe640 1 -- 172.21.15.22:0/494917689 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f80ab7f0 con 0x7f011006c070 2024-03-29T04:18:28.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.276+0000 7f0116eab640 1 -- 172.21.15.22:0/494917689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f011006c070 msgr2=0x7f011006eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:28.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.276+0000 7f0116eab640 1 --2- 172.21.15.22:0/494917689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f011006c070 0x7f011006eda0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f01100b50a0 tx=0x7f00f801c4f0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.276+0000 7f0116eab640 1 -- 172.21.15.22:0/494917689 shutdown_connections 2024-03-29T04:18:28.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.276+0000 7f0116eab640 1 --2- 172.21.15.22:0/494917689 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f011006f2e0 0x7f011006db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.276+0000 7f0116eab640 1 --2- 172.21.15.22:0/494917689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f011006c070 0x7f011006eda0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.276+0000 7f0116eab640 1 --2- 172.21.15.22:0/494917689 >> v2:172.21.15.151:3300/0 conn(0x7f011015a900 0x7f01100afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.276+0000 7f0116eab640 1 -- 172.21.15.22:0/494917689 >> 172.21.15.22:0/494917689 conn(0x7f01100d0c90 msgr2=0x7f01100d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:28.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.276+0000 7f0116eab640 1 -- 172.21.15.22:0/494917689 shutdown_connections 2024-03-29T04:18:28.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.276+0000 7f0116eab640 1 -- 172.21.15.22:0/494917689 wait complete. 2024-03-29T04:18:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.276+0000 7f0116eab640 1 Processor -- start 2024-03-29T04:18:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.276+0000 7f0116eab640 1 -- start start 2024-03-29T04:18:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.276+0000 7f0116eab640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f011006c070 0x7f0110148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f0116eab640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f011006f2e0 0x7f0110156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f0116eab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f011015a900 0x7f0110157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:28.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f0116eab640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f01100b0190 con 0x7f011015a900 2024-03-29T04:18:28.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f0116eab640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f01100b0010 con 0x7f011006c070 2024-03-29T04:18:28.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f0116eab640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f011006e0a0 con 0x7f011006f2e0 2024-03-29T04:18:28.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f0115448640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f011015a900 0x7f0110157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:28.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f0115448640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f011015a900 0x7f0110157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36784/0 (socket says 172.21.15.22:36784) 2024-03-29T04:18:28.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f0115448640 1 -- 172.21.15.22:0/169306847 learned_addr learned my addr 172.21.15.22:0/169306847 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:28.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f0114c47640 1 --2- 172.21.15.22:0/169306847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f011006c070 0x7f0110148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:28.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f010ffff640 1 --2- 172.21.15.22:0/169306847 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f011006f2e0 0x7f0110156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:28.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f0115448640 1 -- 172.21.15.22:0/169306847 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f011006f2e0 msgr2=0x7f0110156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:28.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f0115448640 1 --2- 172.21.15.22:0/169306847 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f011006f2e0 0x7f0110156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f0115448640 1 -- 172.21.15.22:0/169306847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f011006c070 msgr2=0x7f0110148590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:28.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f0115448640 1 --2- 172.21.15.22:0/169306847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f011006c070 0x7f0110148590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f0115448640 1 -- 172.21.15.22:0/169306847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0110157a90 con 0x7f011015a900 2024-03-29T04:18:28.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f0115448640 1 --2- 172.21.15.22:0/169306847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f011015a900 0x7f0110157410 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f010000d830 tx=0x7f010000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:28.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f010d7fa640 1 -- 172.21.15.22:0/169306847 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100018020 con 0x7f011015a900 2024-03-29T04:18:28.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f010d7fa640 1 -- 172.21.15.22:0/169306847 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01000096c0 con 0x7f011015a900 2024-03-29T04:18:28.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f010d7fa640 1 -- 172.21.15.22:0/169306847 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100009ab0 con 0x7f011015a900 2024-03-29T04:18:28.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f0116eab640 1 -- 172.21.15.22:0/169306847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0110157d70 con 0x7f011015a900 2024-03-29T04:18:28.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f0116eab640 1 -- 172.21.15.22:0/169306847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0110158330 con 0x7f011015a900 2024-03-29T04:18:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f0116eab640 1 -- 172.21.15.22:0/169306847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0110068900 con 0x7f011015a900 2024-03-29T04:18:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f010d7fa640 1 -- 172.21.15.22:0/169306847 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f010001b020 con 0x7f011015a900 2024-03-29T04:18:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f010d7fa640 1 --2- 172.21.15.22:0/169306847 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f00f0067fc0 0x7f00f006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f010d7fa640 1 -- 172.21.15.22:0/169306847 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f0100087020 con 0x7f011015a900 2024-03-29T04:18:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.280+0000 7f0114c47640 1 --2- 172.21.15.22:0/169306847 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f00f0067fc0 0x7f00f006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:28.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.284+0000 7f0114c47640 1 --2- 172.21.15.22:0/169306847 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f00f0067fc0 0x7f00f006a480 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f010800abf0 tx=0x7f010800c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:28.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.284+0000 7f010d7fa640 1 -- 172.21.15.22:0/169306847 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f0100022020 con 0x7f011015a900 2024-03-29T04:18:28.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.448+0000 7f0116eab640 1 -- 172.21.15.22:0/169306847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0110155870 con 0x7f011015a900 2024-03-29T04:18:28.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.448+0000 7f010d7fa640 1 -- 172.21.15.22:0/169306847 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f0100011340 con 0x7f011015a900 2024-03-29T04:18:28.429 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:28.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.452+0000 7f0116eab640 1 -- 172.21.15.22:0/169306847 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f00f0067fc0 msgr2=0x7f00f006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:28.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.452+0000 7f0116eab640 1 --2- 172.21.15.22:0/169306847 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f00f0067fc0 0x7f00f006a480 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f010800abf0 tx=0x7f010800c040 comp rx=0 tx=0).stop 2024-03-29T04:18:28.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.452+0000 7f0116eab640 1 -- 172.21.15.22:0/169306847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f011015a900 msgr2=0x7f0110157410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.452+0000 7f0116eab640 1 --2- 172.21.15.22:0/169306847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f011015a900 0x7f0110157410 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f010000d830 tx=0x7f010000dcf0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.452+0000 7f0116eab640 1 -- 172.21.15.22:0/169306847 shutdown_connections 2024-03-29T04:18:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.452+0000 7f0116eab640 1 --2- 172.21.15.22:0/169306847 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f00f0067fc0 0x7f00f006a480 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.452+0000 7f0116eab640 1 --2- 172.21.15.22:0/169306847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f011015a900 0x7f0110157410 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.452+0000 7f0116eab640 1 --2- 172.21.15.22:0/169306847 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f011006f2e0 0x7f0110156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.452+0000 7f0116eab640 1 --2- 172.21.15.22:0/169306847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f011006c070 0x7f0110148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.452+0000 7f0116eab640 1 -- 172.21.15.22:0/169306847 >> 172.21.15.22:0/169306847 conn(0x7f01100d0c90 msgr2=0x7f01100b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.452+0000 7f0116eab640 1 -- 172.21.15.22:0/169306847 shutdown_connections 2024-03-29T04:18:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.452+0000 7f0116eab640 1 -- 172.21.15.22:0/169306847 wait complete. 2024-03-29T04:18:28.449 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:18:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.592+0000 7fa3eec98640 1 Processor -- start 2024-03-29T04:18:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.592+0000 7fa3eec98640 1 -- start start 2024-03-29T04:18:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.592+0000 7fa3eec98640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fa3e815afc0 0x7fa3e80b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.592+0000 7fa3eec98640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa3e80b4520 0x7fa3e806f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.592+0000 7fa3eec98640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3e80b4ba0 con 0x7fa3e806ff20 2024-03-29T04:18:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.592+0000 7fa3eec98640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fa3e8070c30 con 0x7fa3e80b4520 2024-03-29T04:18:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.592+0000 7fa3eec98640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fa3e80b4960 con 0x7fa3e815afc0 2024-03-29T04:18:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.592+0000 7fa3e7fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa3e80b4520 0x7fa3e806f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.592+0000 7fa3e7fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa3e80b4520 0x7fa3e806f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36786/0 (socket says 172.21.15.22:36786) 2024-03-29T04:18:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.592+0000 7fa3e7fff640 1 -- 172.21.15.22:0/2424457414 learned_addr learned my addr 172.21.15.22:0/2424457414 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.592+0000 7fa3eca34640 1 --2- 172.21.15.22:0/2424457414 >> v2:172.21.15.151:3300/0 conn(0x7fa3e815afc0 0x7fa3e80b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.592+0000 7fa3e7fff640 1 -- 172.21.15.22:0/2424457414 >> v2:172.21.15.151:3300/0 conn(0x7fa3e815afc0 msgr2=0x7fa3e80b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.592+0000 7fa3e7fff640 1 --2- 172.21.15.22:0/2424457414 >> v2:172.21.15.151:3300/0 conn(0x7fa3e815afc0 0x7fa3e80b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.592+0000 7fa3e7fff640 1 -- 172.21.15.22:0/2424457414 >> v1:172.21.15.22:6789/0 conn(0x7fa3e806ff20 legacy=0x7fa3e806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:28.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.592+0000 7fa3e7fff640 1 -- 172.21.15.22:0/2424457414 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e815ec60 con 0x7fa3e80b4520 2024-03-29T04:18:28.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.592+0000 7fa3e7fff640 1 --2- 172.21.15.22:0/2424457414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3e80b4520 0x7fa3e806f9e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8073e60 tx=0x7fa3d001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b35c1d81fd092c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:28.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3e77fe640 1 -- 172.21.15.22:0/2424457414 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00b4070 con 0x7fa3e80b4520 2024-03-29T04:18:28.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3e77fe640 1 -- 172.21.15.22:0/2424457414 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3d00a7ad0 con 0x7fa3e80b4520 2024-03-29T04:18:28.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3e77fe640 1 -- 172.21.15.22:0/2424457414 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00af440 con 0x7fa3e80b4520 2024-03-29T04:18:28.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3eec98640 1 -- 172.21.15.22:0/2424457414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3e80b4520 msgr2=0x7fa3e806f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:28.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3eec98640 1 --2- 172.21.15.22:0/2424457414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3e80b4520 0x7fa3e806f9e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8073e60 tx=0x7fa3d001ebf0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3eec98640 1 -- 172.21.15.22:0/2424457414 shutdown_connections 2024-03-29T04:18:28.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3eec98640 1 --2- 172.21.15.22:0/2424457414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3e80b4520 0x7fa3e806f9e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3eec98640 1 --2- 172.21.15.22:0/2424457414 >> v2:172.21.15.151:3300/0 conn(0x7fa3e815afc0 0x7fa3e80b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3eec98640 1 -- 172.21.15.22:0/2424457414 >> 172.21.15.22:0/2424457414 conn(0x7fa3e80d1070 msgr2=0x7fa3e80d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:28.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3eec98640 1 -- 172.21.15.22:0/2424457414 shutdown_connections 2024-03-29T04:18:28.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3eec98640 1 -- 172.21.15.22:0/2424457414 wait complete. 2024-03-29T04:18:28.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3eec98640 1 Processor -- start 2024-03-29T04:18:28.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3eec98640 1 -- start start 2024-03-29T04:18:28.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3eec98640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa3e806ff20 0x7fa3e815ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:28.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3eec98640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3e80b4520 0x7fa3e8143bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:28.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3eec98640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3e815afc0 0x7fa3e8144130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:28.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3eec98640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e815fa20 con 0x7fa3e815afc0 2024-03-29T04:18:28.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3eec98640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e80cf7c0 con 0x7fa3e80b4520 2024-03-29T04:18:28.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3eec98640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa3e815d780 con 0x7fa3e806ff20 2024-03-29T04:18:28.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3ed235640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3e815afc0 0x7fa3e8144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:28.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3ed235640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3e815afc0 0x7fa3e8144130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36800/0 (socket says 172.21.15.22:36800) 2024-03-29T04:18:28.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3ed235640 1 -- 172.21.15.22:0/4074965844 learned_addr learned my addr 172.21.15.22:0/4074965844 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:28.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3e7fff640 1 --2- 172.21.15.22:0/4074965844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3e80b4520 0x7fa3e8143bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:28.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3eca34640 1 --2- 172.21.15.22:0/4074965844 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa3e806ff20 0x7fa3e815ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:28.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3ed235640 1 -- 172.21.15.22:0/4074965844 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa3e806ff20 msgr2=0x7fa3e815ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:28.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3ed235640 1 --2- 172.21.15.22:0/4074965844 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa3e806ff20 0x7fa3e815ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3ed235640 1 -- 172.21.15.22:0/4074965844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3e80b4520 msgr2=0x7fa3e8143bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:28.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3ed235640 1 --2- 172.21.15.22:0/4074965844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3e80b4520 0x7fa3e8143bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3ed235640 1 -- 172.21.15.22:0/4074965844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e815b600 con 0x7fa3e815afc0 2024-03-29T04:18:28.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3eca34640 1 --2- 172.21.15.22:0/4074965844 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa3e806ff20 0x7fa3e815ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:28.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3ed235640 1 --2- 172.21.15.22:0/4074965844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3e815afc0 0x7fa3e8144130 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000c370 tx=0x7fa3e000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:28.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3e57fa640 1 -- 172.21.15.22:0/4074965844 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0005220 con 0x7fa3e815afc0 2024-03-29T04:18:28.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.596+0000 7fa3e57fa640 1 -- 172.21.15.22:0/4074965844 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e00053c0 con 0x7fa3e815afc0 2024-03-29T04:18:28.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.600+0000 7fa3eec98640 1 -- 172.21.15.22:0/4074965844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e81576e0 con 0x7fa3e815afc0 2024-03-29T04:18:28.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.600+0000 7fa3e57fa640 1 -- 172.21.15.22:0/4074965844 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0007270 con 0x7fa3e815afc0 2024-03-29T04:18:28.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.600+0000 7fa3eec98640 1 -- 172.21.15.22:0/4074965844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e815dcd0 con 0x7fa3e815afc0 2024-03-29T04:18:28.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.600+0000 7fa3eec98640 1 -- 172.21.15.22:0/4074965844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e815ded0 con 0x7fa3e815afc0 2024-03-29T04:18:28.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.600+0000 7fa3e57fa640 1 -- 172.21.15.22:0/4074965844 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fa3e0019020 con 0x7fa3e815afc0 2024-03-29T04:18:28.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.600+0000 7fa3e57fa640 1 --2- 172.21.15.22:0/4074965844 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa3c8067ff0 0x7fa3c806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:28.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.600+0000 7fa3e57fa640 1 -- 172.21.15.22:0/4074965844 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fa3e0085be0 con 0x7fa3e815afc0 2024-03-29T04:18:28.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.600+0000 7fa3eca34640 1 --2- 172.21.15.22:0/4074965844 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa3c8067ff0 0x7fa3c806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:28.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.600+0000 7fa3eca34640 1 --2- 172.21.15.22:0/4074965844 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa3c8067ff0 0x7fa3c806a4b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8003fd0 tx=0x7fa3d80039e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:28.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.604+0000 7fa3e57fa640 1 -- 172.21.15.22:0/4074965844 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fa3e815ded0 con 0x7fa3e815afc0 2024-03-29T04:18:28.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.768+0000 7fa3eec98640 1 -- 172.21.15.22:0/4074965844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa3e8068900 con 0x7fa3e815afc0 2024-03-29T04:18:28.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.768+0000 7fa3e57fa640 1 -- 172.21.15.22:0/4074965844 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fa3e0055f00 con 0x7fa3e815afc0 2024-03-29T04:18:28.751 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-29T04:18:28.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.772+0000 7fa3eec98640 1 -- 172.21.15.22:0/4074965844 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa3c8067ff0 msgr2=0x7fa3c806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:28.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.772+0000 7fa3eec98640 1 --2- 172.21.15.22:0/4074965844 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa3c8067ff0 0x7fa3c806a4b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8003fd0 tx=0x7fa3d80039e0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.772+0000 7fa3eec98640 1 -- 172.21.15.22:0/4074965844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3e815afc0 msgr2=0x7fa3e8144130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:28.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.772+0000 7fa3eec98640 1 --2- 172.21.15.22:0/4074965844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3e815afc0 0x7fa3e8144130 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000c370 tx=0x7fa3e000c830 comp rx=0 tx=0).stop 2024-03-29T04:18:28.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.772+0000 7fa3eec98640 1 -- 172.21.15.22:0/4074965844 shutdown_connections 2024-03-29T04:18:28.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.772+0000 7fa3eec98640 1 --2- 172.21.15.22:0/4074965844 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa3c8067ff0 0x7fa3c806a4b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.772+0000 7fa3eec98640 1 --2- 172.21.15.22:0/4074965844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3e815afc0 0x7fa3e8144130 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.772+0000 7fa3eec98640 1 --2- 172.21.15.22:0/4074965844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3e80b4520 0x7fa3e8143bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.772+0000 7fa3eec98640 1 --2- 172.21.15.22:0/4074965844 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa3e806ff20 0x7fa3e815ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.772+0000 7fa3eec98640 1 -- 172.21.15.22:0/4074965844 >> 172.21.15.22:0/4074965844 conn(0x7fa3e80d1070 msgr2=0x7fa3e8072ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:28.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.772+0000 7fa3eec98640 1 -- 172.21.15.22:0/4074965844 shutdown_connections 2024-03-29T04:18:28.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.772+0000 7fa3eec98640 1 -- 172.21.15.22:0/4074965844 wait complete. 2024-03-29T04:18:28.770 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:18:28.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.920+0000 7f1d31481640 1 Processor -- start 2024-03-29T04:18:28.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.920+0000 7f1d31481640 1 -- start start 2024-03-29T04:18:28.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.920+0000 7f1d31481640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f1d2c161c50 0x7f1d2c0ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:28.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.920+0000 7f1d31481640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1d2c0b7d40 0x7f1d2c0cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:28.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.920+0000 7f1d31481640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d2c0ce660 con 0x7f1d2c0cf720 2024-03-29T04:18:28.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.920+0000 7f1d31481640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f1d2c0b78b0 con 0x7f1d2c0b7d40 2024-03-29T04:18:28.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.920+0000 7f1d31481640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f1d2c0b50f0 con 0x7f1d2c161c50 2024-03-29T04:18:28.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.920+0000 7f1d2a7fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1d2c0b7d40 0x7f1d2c0cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:28.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.920+0000 7f1d2b7fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1d2c0cf720 0x7f1d2c162210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57020/0 (socket says 172.21.15.22:57020) 2024-03-29T04:18:28.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.920+0000 7f1d2a7fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1d2c0b7d40 0x7f1d2c0cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36804/0 (socket says 172.21.15.22:36804) 2024-03-29T04:18:28.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.920+0000 7f1d2b7fe640 1 -- 172.21.15.22:0/3745760099 learned_addr learned my addr 172.21.15.22:0/3745760099 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:28.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.920+0000 7f1d2affd640 1 --2- 172.21.15.22:0/3745760099 >> v2:172.21.15.151:3300/0 conn(0x7f1d2c161c50 0x7f1d2c0ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:28.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.920+0000 7f1d2a7fc640 1 -- 172.21.15.22:0/3745760099 >> v2:172.21.15.151:3300/0 conn(0x7f1d2c161c50 msgr2=0x7f1d2c0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:28.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.920+0000 7f1d2a7fc640 1 --2- 172.21.15.22:0/3745760099 >> v2:172.21.15.151:3300/0 conn(0x7f1d2c161c50 0x7f1d2c0ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.920+0000 7f1d2a7fc640 1 -- 172.21.15.22:0/3745760099 >> v1:172.21.15.22:6789/0 conn(0x7f1d2c0cf720 legacy=0x7f1d2c162210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:28.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.920+0000 7f1d2a7fc640 1 -- 172.21.15.22:0/3745760099 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2c0b53d0 con 0x7f1d2c0b7d40 2024-03-29T04:18:28.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.920+0000 7f1d2affd640 1 --2- 172.21.15.22:0/3745760099 >> v2:172.21.15.151:3300/0 conn(0x7f1d2c161c50 0x7f1d2c0ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:28.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d2a7fc640 1 --2- 172.21.15.22:0/3745760099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d2c0b7d40 0x7f1d2c0cc6b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0cea40 tx=0x7f1d1c01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611a8cdb27c27427 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:28.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d29ffb640 1 -- 172.21.15.22:0/3745760099 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c0b0070 con 0x7f1d2c0b7d40 2024-03-29T04:18:28.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d29ffb640 1 -- 172.21.15.22:0/3745760099 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d1c0ab600 con 0x7f1d2c0b7d40 2024-03-29T04:18:28.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d29ffb640 1 -- 172.21.15.22:0/3745760099 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c0ab9d0 con 0x7f1d2c0b7d40 2024-03-29T04:18:28.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d31481640 1 -- 172.21.15.22:0/3745760099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d2c0b7d40 msgr2=0x7f1d2c0cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:28.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d31481640 1 --2- 172.21.15.22:0/3745760099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d2c0b7d40 0x7f1d2c0cc6b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0cea40 tx=0x7f1d1c01c360 comp rx=0 tx=0).stop 2024-03-29T04:18:28.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d31481640 1 -- 172.21.15.22:0/3745760099 shutdown_connections 2024-03-29T04:18:28.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d31481640 1 --2- 172.21.15.22:0/3745760099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d2c0b7d40 0x7f1d2c0cc6b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d31481640 1 --2- 172.21.15.22:0/3745760099 >> v2:172.21.15.151:3300/0 conn(0x7f1d2c161c50 0x7f1d2c0ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d31481640 1 -- 172.21.15.22:0/3745760099 >> 172.21.15.22:0/3745760099 conn(0x7f1d2c0ceae0 msgr2=0x7f1d2c0ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:28.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d31481640 1 -- 172.21.15.22:0/3745760099 shutdown_connections 2024-03-29T04:18:28.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d31481640 1 -- 172.21.15.22:0/3745760099 wait complete. 2024-03-29T04:18:28.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d31481640 1 Processor -- start 2024-03-29T04:18:28.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d31481640 1 -- start start 2024-03-29T04:18:28.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d31481640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d2c0b7d40 0x7f1d2c154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:28.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d31481640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d2c0cf720 0x7f1d2c070240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:28.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d31481640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d2c161c50 0x7f1d2c0ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:28.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d31481640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c16b7c0 con 0x7f1d2c0b7d40 2024-03-29T04:18:28.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d31481640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c0cd000 con 0x7f1d2c161c50 2024-03-29T04:18:28.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d31481640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1d2c16b940 con 0x7f1d2c0cf720 2024-03-29T04:18:28.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d2affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d2c0b7d40 0x7f1d2c154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:28.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d2affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d2c0b7d40 0x7f1d2c154830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36816/0 (socket says 172.21.15.22:36816) 2024-03-29T04:18:28.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d2affd640 1 -- 172.21.15.22:0/2487937152 learned_addr learned my addr 172.21.15.22:0/2487937152 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:28.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d2b7fe640 1 --2- 172.21.15.22:0/2487937152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d2c161c50 0x7f1d2c0ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:28.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d2a7fc640 1 --2- 172.21.15.22:0/2487937152 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d2c0cf720 0x7f1d2c070240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:28.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d2affd640 1 -- 172.21.15.22:0/2487937152 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d2c0cf720 msgr2=0x7f1d2c070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:28.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d2affd640 1 --2- 172.21.15.22:0/2487937152 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d2c0cf720 0x7f1d2c070240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d2affd640 1 -- 172.21.15.22:0/2487937152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d2c161c50 msgr2=0x7f1d2c0ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:28.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d2affd640 1 --2- 172.21.15.22:0/2487937152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d2c161c50 0x7f1d2c0ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:28.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d2affd640 1 -- 172.21.15.22:0/2487937152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2c070780 con 0x7f1d2c0b7d40 2024-03-29T04:18:28.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d2affd640 1 --2- 172.21.15.22:0/2487937152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d2c0b7d40 0x7f1d2c154830 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000d830 tx=0x7f1d2000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:28.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d137fe640 1 -- 172.21.15.22:0/2487937152 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20018020 con 0x7f1d2c0b7d40 2024-03-29T04:18:28.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d137fe640 1 -- 172.21.15.22:0/2487937152 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d20009660 con 0x7f1d2c0b7d40 2024-03-29T04:18:28.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d31481640 1 -- 172.21.15.22:0/2487937152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d2c0b0160 con 0x7f1d2c0b7d40 2024-03-29T04:18:28.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.924+0000 7f1d137fe640 1 -- 172.21.15.22:0/2487937152 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20009a50 con 0x7f1d2c0b7d40 2024-03-29T04:18:28.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.928+0000 7f1d31481640 1 -- 172.21.15.22:0/2487937152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d2c154fe0 con 0x7f1d2c0b7d40 2024-03-29T04:18:28.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.928+0000 7f1d31481640 1 -- 172.21.15.22:0/2487937152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d2c068900 con 0x7f1d2c0b7d40 2024-03-29T04:18:28.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.928+0000 7f1d137fe640 1 -- 172.21.15.22:0/2487937152 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1d2001b020 con 0x7f1d2c0b7d40 2024-03-29T04:18:28.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.928+0000 7f1d137fe640 1 --2- 172.21.15.22:0/2487937152 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1d18067fc0 0x7f1d1806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:28.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.928+0000 7f1d137fe640 1 -- 172.21.15.22:0/2487937152 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f1d20086e40 con 0x7f1d2c0b7d40 2024-03-29T04:18:28.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.928+0000 7f1d2a7fc640 1 --2- 172.21.15.22:0/2487937152 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1d18067fc0 0x7f1d1806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:28.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.928+0000 7f1d2a7fc640 1 --2- 172.21.15.22:0/2487937152 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1d18067fc0 0x7f1d1806a480 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c0ad000 tx=0x7f1d1c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:28.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:28.932+0000 7f1d137fe640 1 -- 172.21.15.22:0/2487937152 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1d20022020 con 0x7f1d2c0b7d40 2024-03-29T04:18:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.092+0000 7f1d31481640 1 -- 172.21.15.22:0/2487937152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1d2c0d1c50 con 0x7f1d2c0b7d40 2024-03-29T04:18:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.092+0000 7f1d137fe640 1 -- 172.21.15.22:0/2487937152 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f1d200571e0 con 0x7f1d2c0b7d40 2024-03-29T04:18:29.075 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.096+0000 7f1d31481640 1 -- 172.21.15.22:0/2487937152 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1d18067fc0 msgr2=0x7f1d1806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.096+0000 7f1d31481640 1 --2- 172.21.15.22:0/2487937152 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1d18067fc0 0x7f1d1806a480 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c0ad000 tx=0x7f1d1c008040 comp rx=0 tx=0).stop 2024-03-29T04:18:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.096+0000 7f1d31481640 1 -- 172.21.15.22:0/2487937152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d2c0b7d40 msgr2=0x7f1d2c154830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.096+0000 7f1d31481640 1 --2- 172.21.15.22:0/2487937152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d2c0b7d40 0x7f1d2c154830 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000d830 tx=0x7f1d2000dcf0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.096+0000 7f1d31481640 1 -- 172.21.15.22:0/2487937152 shutdown_connections 2024-03-29T04:18:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.096+0000 7f1d31481640 1 --2- 172.21.15.22:0/2487937152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d2c161c50 0x7f1d2c0ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.096+0000 7f1d31481640 1 --2- 172.21.15.22:0/2487937152 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1d2c0cf720 0x7f1d2c070240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.096+0000 7f1d31481640 1 --2- 172.21.15.22:0/2487937152 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1d18067fc0 0x7f1d1806a480 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.096+0000 7f1d31481640 1 --2- 172.21.15.22:0/2487937152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d2c0b7d40 0x7f1d2c154830 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.096+0000 7f1d31481640 1 -- 172.21.15.22:0/2487937152 >> 172.21.15.22:0/2487937152 conn(0x7f1d2c0ceae0 msgr2=0x7f1d2c0b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.096+0000 7f1d31481640 1 -- 172.21.15.22:0/2487937152 shutdown_connections 2024-03-29T04:18:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.096+0000 7f1d31481640 1 -- 172.21.15.22:0/2487937152 wait complete. 2024-03-29T04:18:29.093 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:18:29.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f5325a41640 1 Processor -- start 2024-03-29T04:18:29.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f5325a41640 1 -- start start 2024-03-29T04:18:29.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f5325a41640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f53200cc910 0x7f5320166ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:29.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f5325a41640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53200ccef0 con 0x7f532006c670 2024-03-29T04:18:29.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f5325a41640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53200b3710 con 0x7f53200b41f0 2024-03-29T04:18:29.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f5325a41640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f53200b34c0 con 0x7f53200cc910 2024-03-29T04:18:29.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531effd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f532006c670 0x7f532006e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57022/0 (socket says 172.21.15.22:57022) 2024-03-29T04:18:29.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531effd640 1 -- 172.21.15.22:0/3186936255 learned_addr learned my addr 172.21.15.22:0/3186936255 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:29.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531dffb640 1 -- 172.21.15.22:0/3186936255 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1290167090 0 0) 0x7f53200ccef0 con 0x7f532006c670 2024-03-29T04:18:29.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531f7fe640 1 --2- 172.21.15.22:0/3186936255 >> v2:172.21.15.151:3300/0 conn(0x7f53200cc910 0x7f5320166ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:29.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531dffb640 1 -- 172.21.15.22:0/3186936255 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5308003530 con 0x7f532006c670 2024-03-29T04:18:29.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531dffb640 1 -- 172.21.15.22:0/3186936255 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1209835153 0 0) 0x7f5308003530 con 0x7f532006c670 2024-03-29T04:18:29.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531dffb640 1 -- 172.21.15.22:0/3186936255 >> v2:172.21.15.151:3300/0 conn(0x7f53200cc910 msgr2=0x7f5320166ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:29.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531dffb640 1 --2- 172.21.15.22:0/3186936255 >> v2:172.21.15.151:3300/0 conn(0x7f53200cc910 0x7f5320166ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531dffb640 1 -- 172.21.15.22:0/3186936255 >> v1:172.21.15.151:6789/0 conn(0x7f53200b41f0 legacy=0x7f532006ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:29.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531dffb640 1 -- 172.21.15.22:0/3186936255 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53200b39f0 con 0x7f532006c670 2024-03-29T04:18:29.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531dffb640 1 -- 172.21.15.22:0/3186936255 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f5314001680 con 0x7f532006c670 2024-03-29T04:18:29.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531dffb640 1 -- 172.21.15.22:0/3186936255 >> v1:172.21.15.22:6789/0 conn(0x7f532006c670 legacy=0x7f532006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:29.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531dffb640 1 --2- 172.21.15.22:0/3186936255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53080039c0 0x7f5308003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:29.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531dffb640 1 --2- 172.21.15.22:0/3186936255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5308004810 0x7f5308004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:29.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531dffb640 1 --2- 172.21.15.22:0/3186936255 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53080051d0 0x7f530800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:29.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531dffb640 1 -- 172.21.15.22:0/3186936255 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f53200b34c0 con 0x7f53080039c0 2024-03-29T04:18:29.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531dffb640 1 -- 172.21.15.22:0/3186936255 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f530800c120 con 0x7f5308004810 2024-03-29T04:18:29.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531dffb640 1 -- 172.21.15.22:0/3186936255 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f530800c320 con 0x7f53080051d0 2024-03-29T04:18:29.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531effd640 1 --2- 172.21.15.22:0/3186936255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53080039c0 0x7f5308003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:29.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531e7fc640 1 --2- 172.21.15.22:0/3186936255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5308004810 0x7f5308004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:29.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531f7fe640 1 --2- 172.21.15.22:0/3186936255 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53080051d0 0x7f530800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:29.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531effd640 1 -- 172.21.15.22:0/3186936255 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53080051d0 msgr2=0x7f530800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:29.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531effd640 1 --2- 172.21.15.22:0/3186936255 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53080051d0 0x7f530800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531effd640 1 -- 172.21.15.22:0/3186936255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5308004810 msgr2=0x7f5308004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:29.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531effd640 1 --2- 172.21.15.22:0/3186936255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5308004810 0x7f5308004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531effd640 1 -- 172.21.15.22:0/3186936255 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5308003530 con 0x7f53080039c0 2024-03-29T04:18:29.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.228+0000 7f531f7fe640 1 -- 172.21.15.22:0/3186936255 reap_dead start 2024-03-29T04:18:29.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f531effd640 1 --2- 172.21.15.22:0/3186936255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53080039c0 0x7f5308003de0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f531400c1c0 tx=0x7f531401cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2c11ecdbebd1dc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:29.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f531dffb640 1 -- 172.21.15.22:0/3186936255 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314007d00 con 0x7f53080039c0 2024-03-29T04:18:29.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f531dffb640 1 -- 172.21.15.22:0/3186936255 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53140a9020 con 0x7f53080039c0 2024-03-29T04:18:29.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f531dffb640 1 -- 172.21.15.22:0/3186936255 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314005b00 con 0x7f53080039c0 2024-03-29T04:18:29.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f5325a41640 1 -- 172.21.15.22:0/3186936255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53080039c0 msgr2=0x7f5308003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:29.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f5325a41640 1 --2- 172.21.15.22:0/3186936255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53080039c0 0x7f5308003de0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f531400c1c0 tx=0x7f531401cbe0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f5325a41640 1 -- 172.21.15.22:0/3186936255 shutdown_connections 2024-03-29T04:18:29.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f5325a41640 1 --2- 172.21.15.22:0/3186936255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53080039c0 0x7f5308003de0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f5325a41640 1 -- 172.21.15.22:0/3186936255 >> 172.21.15.22:0/3186936255 conn(0x7f53200cfee0 msgr2=0x7f53200d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:29.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f5325a41640 1 -- 172.21.15.22:0/3186936255 shutdown_connections 2024-03-29T04:18:29.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f5325a41640 1 -- 172.21.15.22:0/3186936255 wait complete. 2024-03-29T04:18:29.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f5325a41640 1 Processor -- start 2024-03-29T04:18:29.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f5325a41640 1 -- start start 2024-03-29T04:18:29.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f5325a41640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53080039c0 0x7f53201f8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:29.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f5325a41640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f532014ebd0 0x7f532013bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:29.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f5325a41640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5320143df0 0x7f532015e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:29.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f5325a41640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f532006cfe0 con 0x7f532014ebd0 2024-03-29T04:18:29.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f5325a41640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f532006ce60 con 0x7f5320143df0 2024-03-29T04:18:29.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f5325a41640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5320160590 con 0x7f53080039c0 2024-03-29T04:18:29.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f531e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f532014ebd0 0x7f532013bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:29.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f531e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f532014ebd0 0x7f532013bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36830/0 (socket says 172.21.15.22:36830) 2024-03-29T04:18:29.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f531e7fc640 1 -- 172.21.15.22:0/1041680588 learned_addr learned my addr 172.21.15.22:0/1041680588 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:29.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.232+0000 7f531effd640 1 --2- 172.21.15.22:0/1041680588 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53080039c0 0x7f53201f8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:29.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.236+0000 7f531f7fe640 1 --2- 172.21.15.22:0/1041680588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5320143df0 0x7f532015e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:29.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.236+0000 7f531e7fc640 1 -- 172.21.15.22:0/1041680588 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53080039c0 msgr2=0x7f53201f8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:29.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.236+0000 7f531e7fc640 1 --2- 172.21.15.22:0/1041680588 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53080039c0 0x7f53201f8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.236+0000 7f531e7fc640 1 -- 172.21.15.22:0/1041680588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5320143df0 msgr2=0x7f532015e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:29.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.236+0000 7f531e7fc640 1 --2- 172.21.15.22:0/1041680588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5320143df0 0x7f532015e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.236+0000 7f531e7fc640 1 -- 172.21.15.22:0/1041680588 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53201f9c40 con 0x7f532014ebd0 2024-03-29T04:18:29.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.236+0000 7f531e7fc640 1 --2- 172.21.15.22:0/1041680588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f532014ebd0 0x7f532013bac0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f531000f820 tx=0x7f531000fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:29.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.236+0000 7f5307fff640 1 -- 172.21.15.22:0/1041680588 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310001cf0 con 0x7f532014ebd0 2024-03-29T04:18:29.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.236+0000 7f5307fff640 1 -- 172.21.15.22:0/1041680588 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5310018070 con 0x7f532014ebd0 2024-03-29T04:18:29.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.236+0000 7f5325a41640 1 -- 172.21.15.22:0/1041680588 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53201f93a0 con 0x7f532014ebd0 2024-03-29T04:18:29.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.236+0000 7f5307fff640 1 -- 172.21.15.22:0/1041680588 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531000d730 con 0x7f532014ebd0 2024-03-29T04:18:29.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.236+0000 7f5325a41640 1 -- 172.21.15.22:0/1041680588 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53201f9960 con 0x7f532014ebd0 2024-03-29T04:18:29.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.236+0000 7f5325a41640 1 -- 172.21.15.22:0/1041680588 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5320068890 con 0x7f532014ebd0 2024-03-29T04:18:29.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.236+0000 7f5307fff640 1 -- 172.21.15.22:0/1041680588 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f531000d8d0 con 0x7f532014ebd0 2024-03-29T04:18:29.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.236+0000 7f5307fff640 1 --2- 172.21.15.22:0/1041680588 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f530c067fc0 0x7f530c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:29.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.236+0000 7f5307fff640 1 -- 172.21.15.22:0/1041680588 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f53100867c0 con 0x7f532014ebd0 2024-03-29T04:18:29.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.236+0000 7f531effd640 1 --2- 172.21.15.22:0/1041680588 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f530c067fc0 0x7f530c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:29.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.236+0000 7f531effd640 1 --2- 172.21.15.22:0/1041680588 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f530c067fc0 0x7f530c06a480 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f53140aa000 tx=0x7f53200cd0c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:29.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.240+0000 7f5307fff640 1 -- 172.21.15.22:0/1041680588 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f5310056b70 con 0x7f532014ebd0 2024-03-29T04:18:29.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.408+0000 7f5325a41640 1 -- 172.21.15.22:0/1041680588 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f53200cf460 con 0x7f532014ebd0 2024-03-29T04:18:29.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.408+0000 7f5307fff640 1 -- 172.21.15.22:0/1041680588 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f5310056470 con 0x7f532014ebd0 2024-03-29T04:18:29.391 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-03-29T04:18:29.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.412+0000 7f5325a41640 1 -- 172.21.15.22:0/1041680588 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f530c067fc0 msgr2=0x7f530c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:29.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.412+0000 7f5325a41640 1 --2- 172.21.15.22:0/1041680588 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f530c067fc0 0x7f530c06a480 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f53140aa000 tx=0x7f53200cd0c0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.412+0000 7f5325a41640 1 -- 172.21.15.22:0/1041680588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f532014ebd0 msgr2=0x7f532013bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:29.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.412+0000 7f5325a41640 1 --2- 172.21.15.22:0/1041680588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f532014ebd0 0x7f532013bac0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f531000f820 tx=0x7f531000fce0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.412+0000 7f5325a41640 1 -- 172.21.15.22:0/1041680588 shutdown_connections 2024-03-29T04:18:29.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.412+0000 7f5325a41640 1 --2- 172.21.15.22:0/1041680588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5320143df0 0x7f532015e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.412+0000 7f5325a41640 1 --2- 172.21.15.22:0/1041680588 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f530c067fc0 0x7f530c06a480 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.412+0000 7f5325a41640 1 --2- 172.21.15.22:0/1041680588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f532014ebd0 0x7f532013bac0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.412+0000 7f5325a41640 1 --2- 172.21.15.22:0/1041680588 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53080039c0 0x7f53201f8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.412+0000 7f5325a41640 1 -- 172.21.15.22:0/1041680588 >> 172.21.15.22:0/1041680588 conn(0x7f53200cfee0 msgr2=0x7f532015d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:29.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.412+0000 7f5325a41640 1 -- 172.21.15.22:0/1041680588 shutdown_connections 2024-03-29T04:18:29.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.412+0000 7f5325a41640 1 -- 172.21.15.22:0/1041680588 wait complete. 2024-03-29T04:18:29.409 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:18:29.409 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:18:29.415 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:29.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.600+0000 7fb635c89640 1 Processor -- start 2024-03-29T04:18:29.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.600+0000 7fb635c89640 1 -- start start 2024-03-29T04:18:29.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.600+0000 7fb635c89640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fb630158c40 0x7fb63006b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:29.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.600+0000 7fb635c89640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6300b0470 0x7fb63006bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:29.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.600+0000 7fb635c89640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb63006c0e0 con 0x7fb6300b0090 2024-03-29T04:18:29.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.600+0000 7fb635c89640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fb63006e780 con 0x7fb630158c40 2024-03-29T04:18:29.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.600+0000 7fb635c89640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb63006c350 con 0x7fb6300b0470 2024-03-29T04:18:29.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.600+0000 7fb62effd640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fb630158c40 0x7fb63006b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:29.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.600+0000 7fb62effd640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fb630158c40 0x7fb63006b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:59202/0 (socket says 172.21.15.22:59202) 2024-03-29T04:18:29.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.600+0000 7fb62effd640 1 -- 172.21.15.22:0/3517972462 learned_addr learned my addr 172.21.15.22:0/3517972462 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:29.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.600+0000 7fb62ffff640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6300b0470 0x7fb63006bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:29.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.600+0000 7fb62f7fe640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7fb6300b0090 0x7fb630158530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.22:35660/0 (socket says 172.21.15.22:35660) 2024-03-29T04:18:29.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.604+0000 7fb62effd640 1 -- 172.21.15.22:0/3517972462 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6300b0470 msgr2=0x7fb63006bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:29.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.604+0000 7fb62effd640 1 --2- 172.21.15.22:0/3517972462 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6300b0470 0x7fb63006bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.604+0000 7fb62effd640 1 -- 172.21.15.22:0/3517972462 >> v1:172.21.15.151:6789/0 conn(0x7fb6300b0090 legacy=0x7fb630158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:29.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.604+0000 7fb62effd640 1 -- 172.21.15.22:0/3517972462 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63006c630 con 0x7fb630158c40 2024-03-29T04:18:29.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.604+0000 7fb62effd640 1 --2- 172.21.15.22:0/3517972462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb630158c40 0x7fb63006b660 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb624009870 tx=0x7fb62401c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=577bf3abb786d40e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:29.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.604+0000 7fb62e7fc640 1 -- 172.21.15.22:0/3517972462 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6240b0070 con 0x7fb630158c40 2024-03-29T04:18:29.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.604+0000 7fb62e7fc640 1 -- 172.21.15.22:0/3517972462 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb624004230 con 0x7fb630158c40 2024-03-29T04:18:29.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.604+0000 7fb62e7fc640 1 -- 172.21.15.22:0/3517972462 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624003620 con 0x7fb630158c40 2024-03-29T04:18:29.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.604+0000 7fb635c89640 1 -- 172.21.15.22:0/3517972462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb630158c40 msgr2=0x7fb63006b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:29.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.604+0000 7fb635c89640 1 --2- 172.21.15.22:0/3517972462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb630158c40 0x7fb63006b660 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb624009870 tx=0x7fb62401c440 comp rx=0 tx=0).stop 2024-03-29T04:18:29.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.604+0000 7fb635c89640 1 -- 172.21.15.22:0/3517972462 shutdown_connections 2024-03-29T04:18:29.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.604+0000 7fb635c89640 1 --2- 172.21.15.22:0/3517972462 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6300b0470 0x7fb63006bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.604+0000 7fb635c89640 1 --2- 172.21.15.22:0/3517972462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb630158c40 0x7fb63006b660 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.604+0000 7fb635c89640 1 -- 172.21.15.22:0/3517972462 >> 172.21.15.22:0/3517972462 conn(0x7fb6300b2c70 msgr2=0x7fb6300b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:29.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.604+0000 7fb635c89640 1 -- 172.21.15.22:0/3517972462 shutdown_connections 2024-03-29T04:18:29.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.604+0000 7fb635c89640 1 -- 172.21.15.22:0/3517972462 wait complete. 2024-03-29T04:18:29.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb635c89640 1 Processor -- start 2024-03-29T04:18:29.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb635c89640 1 -- start start 2024-03-29T04:18:29.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb635c89640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6300b0090 0x7fb630151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:29.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb635c89640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6300b0470 0x7fb6301565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:29.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb635c89640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb630158c40 0x7fb6300ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:29.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb635c89640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb6300b5f00 con 0x7fb6300b0090 2024-03-29T04:18:29.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb635c89640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb6300b5d80 con 0x7fb630158c40 2024-03-29T04:18:29.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb635c89640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb6300b6080 con 0x7fb6300b0470 2024-03-29T04:18:29.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb62f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6300b0090 0x7fb630151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:29.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb62f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6300b0090 0x7fb630151800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36832/0 (socket says 172.21.15.22:36832) 2024-03-29T04:18:29.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb62f7fe640 1 -- 172.21.15.22:0/2242595230 learned_addr learned my addr 172.21.15.22:0/2242595230 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:29.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb62ffff640 1 --2- 172.21.15.22:0/2242595230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb630158c40 0x7fb6300ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:29.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb62effd640 1 --2- 172.21.15.22:0/2242595230 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6300b0470 0x7fb6301565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:29.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb62f7fe640 1 -- 172.21.15.22:0/2242595230 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6300b0470 msgr2=0x7fb6301565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:29.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb62f7fe640 1 --2- 172.21.15.22:0/2242595230 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6300b0470 0x7fb6301565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb62f7fe640 1 -- 172.21.15.22:0/2242595230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb630158c40 msgr2=0x7fb6300ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:29.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb62f7fe640 1 --2- 172.21.15.22:0/2242595230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb630158c40 0x7fb6300ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb62f7fe640 1 -- 172.21.15.22:0/2242595230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb630157020 con 0x7fb6300b0090 2024-03-29T04:18:29.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb62f7fe640 1 --2- 172.21.15.22:0/2242595230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6300b0090 0x7fb630151800 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb62000c370 tx=0x7fb62000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:29.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb617fff640 1 -- 172.21.15.22:0/2242595230 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6200074e0 con 0x7fb6300b0090 2024-03-29T04:18:29.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb617fff640 1 -- 172.21.15.22:0/2242595230 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb620007680 con 0x7fb6300b0090 2024-03-29T04:18:29.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb635c89640 1 -- 172.21.15.22:0/2242595230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6300adb30 con 0x7fb6300b0090 2024-03-29T04:18:29.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb617fff640 1 -- 172.21.15.22:0/2242595230 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620004a30 con 0x7fb6300b0090 2024-03-29T04:18:29.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb635c89640 1 -- 172.21.15.22:0/2242595230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6300ae0f0 con 0x7fb6300b0090 2024-03-29T04:18:29.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb635c89640 1 -- 172.21.15.22:0/2242595230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb630068890 con 0x7fb6300b0090 2024-03-29T04:18:29.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb617fff640 1 -- 172.21.15.22:0/2242595230 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb62001a020 con 0x7fb6300b0090 2024-03-29T04:18:29.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb617fff640 1 --2- 172.21.15.22:0/2242595230 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb61c067fc0 0x7fb61c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:29.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb617fff640 1 -- 172.21.15.22:0/2242595230 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fb620086020 con 0x7fb6300b0090 2024-03-29T04:18:29.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.608+0000 7fb62effd640 1 --2- 172.21.15.22:0/2242595230 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb61c067fc0 0x7fb61c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:29.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.612+0000 7fb62effd640 1 --2- 172.21.15.22:0/2242595230 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb61c067fc0 0x7fb61c06a480 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb6240ad000 tx=0x7fb624009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:29.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.612+0000 7fb617fff640 1 -- 172.21.15.22:0/2242595230 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fb6200563d0 con 0x7fb6300b0090 2024-03-29T04:18:29.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.768+0000 7fb635c89640 1 -- 172.21.15.22:0/2242595230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6300b4810 con 0x7fb6300b0090 2024-03-29T04:18:29.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.772+0000 7fb617fff640 1 -- 172.21.15.22:0/2242595230 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18847 (secure 0 0 0) 0x7fb620055cd0 con 0x7fb6300b0090 2024-03-29T04:18:29.753 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:29.753 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:18:16.893439+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:29.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.772+0000 7fb635c89640 1 -- 172.21.15.22:0/2242595230 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb61c067fc0 msgr2=0x7fb61c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:29.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.772+0000 7fb635c89640 1 --2- 172.21.15.22:0/2242595230 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb61c067fc0 0x7fb61c06a480 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb6240ad000 tx=0x7fb624009040 comp rx=0 tx=0).stop 2024-03-29T04:18:29.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.772+0000 7fb635c89640 1 -- 172.21.15.22:0/2242595230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6300b0090 msgr2=0x7fb630151800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:29.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.772+0000 7fb635c89640 1 --2- 172.21.15.22:0/2242595230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6300b0090 0x7fb630151800 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb62000c370 tx=0x7fb62000c830 comp rx=0 tx=0).stop 2024-03-29T04:18:29.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.772+0000 7fb635c89640 1 -- 172.21.15.22:0/2242595230 shutdown_connections 2024-03-29T04:18:29.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.772+0000 7fb635c89640 1 --2- 172.21.15.22:0/2242595230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb630158c40 0x7fb6300ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.772+0000 7fb635c89640 1 --2- 172.21.15.22:0/2242595230 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6300b0470 0x7fb6301565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.772+0000 7fb635c89640 1 --2- 172.21.15.22:0/2242595230 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb61c067fc0 0x7fb61c06a480 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.772+0000 7fb635c89640 1 --2- 172.21.15.22:0/2242595230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6300b0090 0x7fb630151800 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.772+0000 7fb635c89640 1 -- 172.21.15.22:0/2242595230 >> 172.21.15.22:0/2242595230 conn(0x7fb6300b2c70 msgr2=0x7fb6300b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:29.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.772+0000 7fb635c89640 1 -- 172.21.15.22:0/2242595230 shutdown_connections 2024-03-29T04:18:29.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.772+0000 7fb635c89640 1 -- 172.21.15.22:0/2242595230 wait complete. 2024-03-29T04:18:29.771 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:17:34.034142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:17:34.886659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:17:38.628632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:18:29.771 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:18:29.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.908+0000 7f7b3727f640 1 Processor -- start 2024-03-29T04:18:29.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.908+0000 7f7b3727f640 1 -- start start 2024-03-29T04:18:29.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.908+0000 7f7b3727f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7b300ba090 0x7f7b300cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:29.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.908+0000 7f7b3727f640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f7b300cf670 0x7f7b3006bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:29.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.908+0000 7f7b3727f640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b300d3ab0 con 0x7f7b3015e540 2024-03-29T04:18:29.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.908+0000 7f7b3727f640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f7b300cfae0 con 0x7f7b300ba090 2024-03-29T04:18:29.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.908+0000 7f7b3727f640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f7b300d4070 con 0x7f7b300cf670 2024-03-29T04:18:29.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.908+0000 7f7b3481a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7b300ba090 0x7f7b300cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:29.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.908+0000 7f7b3481a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7b300ba090 0x7f7b300cf130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36840/0 (socket says 172.21.15.22:36840) 2024-03-29T04:18:29.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.908+0000 7f7b3481a640 1 -- 172.21.15.22:0/1825764923 learned_addr learned my addr 172.21.15.22:0/1825764923 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:29.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.908+0000 7f7b3581c640 1 --2- 172.21.15.22:0/1825764923 >> v2:172.21.15.151:3300/0 conn(0x7f7b300cf670 0x7f7b3006bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:29.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.908+0000 7f7b3481a640 1 -- 172.21.15.22:0/1825764923 >> v2:172.21.15.151:3300/0 conn(0x7f7b300cf670 msgr2=0x7f7b3006bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:29.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.908+0000 7f7b3481a640 1 --2- 172.21.15.22:0/1825764923 >> v2:172.21.15.151:3300/0 conn(0x7f7b300cf670 0x7f7b3006bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.908+0000 7f7b3481a640 1 -- 172.21.15.22:0/1825764923 >> v1:172.21.15.151:6789/0 conn(0x7f7b3015e540 legacy=0x7f7b300d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:29.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.908+0000 7f7b3481a640 1 -- 172.21.15.22:0/1825764923 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b300d3d90 con 0x7f7b300ba090 2024-03-29T04:18:29.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3481a640 1 --2- 172.21.15.22:0/1825764923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b300ba090 0x7f7b300cf130 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7b3008d720 tx=0x7f7b2001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2fb323b8774f20 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:29.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b27fff640 1 -- 172.21.15.22:0/1825764923 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b200b0070 con 0x7f7b300ba090 2024-03-29T04:18:29.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b27fff640 1 -- 172.21.15.22:0/1825764923 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b200ab600 con 0x7f7b300ba090 2024-03-29T04:18:29.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b27fff640 1 -- 172.21.15.22:0/1825764923 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b200ab9d0 con 0x7f7b300ba090 2024-03-29T04:18:29.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3727f640 1 -- 172.21.15.22:0/1825764923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b300ba090 msgr2=0x7f7b300cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:29.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3727f640 1 --2- 172.21.15.22:0/1825764923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b300ba090 0x7f7b300cf130 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7b3008d720 tx=0x7f7b2001c490 comp rx=0 tx=0).stop 2024-03-29T04:18:29.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3727f640 1 -- 172.21.15.22:0/1825764923 shutdown_connections 2024-03-29T04:18:29.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3727f640 1 --2- 172.21.15.22:0/1825764923 >> v2:172.21.15.151:3300/0 conn(0x7f7b300cf670 0x7f7b3006bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3727f640 1 --2- 172.21.15.22:0/1825764923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b300ba090 0x7f7b300cf130 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3727f640 1 -- 172.21.15.22:0/1825764923 >> 172.21.15.22:0/1825764923 conn(0x7f7b300d4360 msgr2=0x7f7b300d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:29.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3727f640 1 -- 172.21.15.22:0/1825764923 shutdown_connections 2024-03-29T04:18:29.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3727f640 1 -- 172.21.15.22:0/1825764923 wait complete. 2024-03-29T04:18:29.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3727f640 1 Processor -- start 2024-03-29T04:18:29.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3727f640 1 -- start start 2024-03-29T04:18:29.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3727f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b300ba090 0x7f7b301592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:29.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3727f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b300cf670 0x7f7b30153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:29.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3727f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7b3015e540 0x7f7b30150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:29.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3727f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3006ce60 con 0x7f7b300ba090 2024-03-29T04:18:29.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3727f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3006cce0 con 0x7f7b300cf670 2024-03-29T04:18:29.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3727f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7b3006cfe0 con 0x7f7b3015e540 2024-03-29T04:18:29.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3501b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b300ba090 0x7f7b301592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:29.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3501b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b300ba090 0x7f7b301592b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36852/0 (socket says 172.21.15.22:36852) 2024-03-29T04:18:29.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3501b640 1 -- 172.21.15.22:0/1254445037 learned_addr learned my addr 172.21.15.22:0/1254445037 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:29.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3481a640 1 --2- 172.21.15.22:0/1254445037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b300cf670 0x7f7b30153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:29.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3501b640 1 -- 172.21.15.22:0/1254445037 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7b3015e540 msgr2=0x7f7b30150870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:29.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3501b640 1 --2- 172.21.15.22:0/1254445037 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7b3015e540 0x7f7b30150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3501b640 1 -- 172.21.15.22:0/1254445037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b300cf670 msgr2=0x7f7b30153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:29.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3501b640 1 --2- 172.21.15.22:0/1254445037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b300cf670 0x7f7b30153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:29.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3501b640 1 -- 172.21.15.22:0/1254445037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b30157f30 con 0x7f7b300ba090 2024-03-29T04:18:29.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3501b640 1 --2- 172.21.15.22:0/1254445037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b300ba090 0x7f7b301592b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00c500 tx=0x7f7b2c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:29.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b25ffb640 1 -- 172.21.15.22:0/1254445037 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c016020 con 0x7f7b300ba090 2024-03-29T04:18:29.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b25ffb640 1 -- 172.21.15.22:0/1254445037 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b2c005220 con 0x7f7b300ba090 2024-03-29T04:18:29.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3727f640 1 -- 172.21.15.22:0/1254445037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b30158210 con 0x7f7b300ba090 2024-03-29T04:18:29.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b25ffb640 1 -- 172.21.15.22:0/1254445037 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c007030 con 0x7f7b300ba090 2024-03-29T04:18:29.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.912+0000 7f7b3727f640 1 -- 172.21.15.22:0/1254445037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b301587d0 con 0x7f7b300ba090 2024-03-29T04:18:29.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.916+0000 7f7b3727f640 1 -- 172.21.15.22:0/1254445037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b30068900 con 0x7f7b300ba090 2024-03-29T04:18:29.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.916+0000 7f7b25ffb640 1 -- 172.21.15.22:0/1254445037 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f7b2c0071f0 con 0x7f7b300ba090 2024-03-29T04:18:29.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.916+0000 7f7b25ffb640 1 --2- 172.21.15.22:0/1254445037 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7b14067fc0 0x7f7b1406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:29.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.916+0000 7f7b25ffb640 1 -- 172.21.15.22:0/1254445037 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f7b2c086330 con 0x7f7b300ba090 2024-03-29T04:18:29.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.916+0000 7f7b3481a640 1 --2- 172.21.15.22:0/1254445037 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7b14067fc0 0x7f7b1406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:29.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.916+0000 7f7b3481a640 1 --2- 172.21.15.22:0/1254445037 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7b14067fc0 0x7f7b1406a480 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7b200ad000 tx=0x7f7b20008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:29.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:29.920+0000 7f7b25ffb640 1 -- 172.21.15.22:0/1254445037 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f7b2c020020 con 0x7f7b300ba090 2024-03-29T04:18:30.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.076+0000 7f7b3727f640 1 -- 172.21.15.22:0/1254445037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7b3015d4c0 con 0x7f7b300ba090 2024-03-29T04:18:30.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.076+0000 7f7b25ffb640 1 -- 172.21.15.22:0/1254445037 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f7b2c0566f0 con 0x7f7b300ba090 2024-03-29T04:18:30.058 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:18:30.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.080+0000 7f7b3727f640 1 -- 172.21.15.22:0/1254445037 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7b14067fc0 msgr2=0x7f7b1406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:30.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.080+0000 7f7b3727f640 1 --2- 172.21.15.22:0/1254445037 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7b14067fc0 0x7f7b1406a480 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7b200ad000 tx=0x7f7b20008040 comp rx=0 tx=0).stop 2024-03-29T04:18:30.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.080+0000 7f7b3727f640 1 -- 172.21.15.22:0/1254445037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b300ba090 msgr2=0x7f7b301592b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:30.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.080+0000 7f7b3727f640 1 --2- 172.21.15.22:0/1254445037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b300ba090 0x7f7b301592b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00c500 tx=0x7f7b2c00c9c0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.080+0000 7f7b3727f640 1 -- 172.21.15.22:0/1254445037 shutdown_connections 2024-03-29T04:18:30.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.080+0000 7f7b3727f640 1 --2- 172.21.15.22:0/1254445037 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7b3015e540 0x7f7b30150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.080+0000 7f7b3727f640 1 --2- 172.21.15.22:0/1254445037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b300cf670 0x7f7b30153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.080+0000 7f7b3727f640 1 --2- 172.21.15.22:0/1254445037 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7b14067fc0 0x7f7b1406a480 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.080+0000 7f7b3727f640 1 --2- 172.21.15.22:0/1254445037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b300ba090 0x7f7b301592b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.080+0000 7f7b3727f640 1 -- 172.21.15.22:0/1254445037 >> 172.21.15.22:0/1254445037 conn(0x7f7b300d4360 msgr2=0x7f7b3015eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:30.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.080+0000 7f7b3727f640 1 -- 172.21.15.22:0/1254445037 shutdown_connections 2024-03-29T04:18:30.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.080+0000 7f7b3727f640 1 -- 172.21.15.22:0/1254445037 wait complete. 2024-03-29T04:18:30.076 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:18:30.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.216+0000 7ff8835e9640 1 Processor -- start 2024-03-29T04:18:30.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.216+0000 7ff8835e9640 1 -- start start 2024-03-29T04:18:30.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.220+0000 7ff8835e9640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff87c15f5e0 0x7ff87c0cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:30.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.220+0000 7ff8835e9640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff87c0b56d0 con 0x7ff87c15f200 2024-03-29T04:18:30.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.220+0000 7ff8835e9640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff87c0cfe50 con 0x7ff87c0ccd00 2024-03-29T04:18:30.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.220+0000 7ff8835e9640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7ff87c0cd170 con 0x7ff87c15f5e0 2024-03-29T04:18:30.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.220+0000 7ff880b84640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff87c15f5e0 0x7ff87c0cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:30.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.220+0000 7ff880b84640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff87c15f5e0 0x7ff87c0cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36862/0 (socket says 172.21.15.22:36862) 2024-03-29T04:18:30.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.220+0000 7ff880b84640 1 -- 172.21.15.22:0/2447613908 learned_addr learned my addr 172.21.15.22:0/2447613908 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:30.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.220+0000 7ff880b84640 1 -- 172.21.15.22:0/2447613908 >> v1:172.21.15.151:6789/0 conn(0x7ff87c0ccd00 legacy=0x7ff87c154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:30.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.220+0000 7ff880b84640 1 -- 172.21.15.22:0/2447613908 >> v1:172.21.15.22:6789/0 conn(0x7ff87c15f200 legacy=0x7ff87c0b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:30.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.220+0000 7ff880b84640 1 -- 172.21.15.22:0/2447613908 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87c0d0130 con 0x7ff87c15f5e0 2024-03-29T04:18:30.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.220+0000 7ff880b84640 1 --2- 172.21.15.22:0/2447613908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff87c15f5e0 0x7ff87c0cc7c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff86c009870 tx=0x7ff86c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1e3d63642a15f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:30.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.220+0000 7ff873fff640 1 -- 172.21.15.22:0/2447613908 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0b2070 con 0x7ff87c15f5e0 2024-03-29T04:18:30.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.220+0000 7ff873fff640 1 -- 172.21.15.22:0/2447613908 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff86c0a9070 con 0x7ff87c15f5e0 2024-03-29T04:18:30.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.220+0000 7ff873fff640 1 -- 172.21.15.22:0/2447613908 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0ad880 con 0x7ff87c15f5e0 2024-03-29T04:18:30.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.220+0000 7ff8835e9640 1 -- 172.21.15.22:0/2447613908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff87c15f5e0 msgr2=0x7ff87c0cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:30.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.220+0000 7ff8835e9640 1 --2- 172.21.15.22:0/2447613908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff87c15f5e0 0x7ff87c0cc7c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff86c009870 tx=0x7ff86c01c560 comp rx=0 tx=0).stop 2024-03-29T04:18:30.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.220+0000 7ff8835e9640 1 -- 172.21.15.22:0/2447613908 shutdown_connections 2024-03-29T04:18:30.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.220+0000 7ff8835e9640 1 --2- 172.21.15.22:0/2447613908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff87c15f5e0 0x7ff87c0cc7c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.220+0000 7ff8835e9640 1 -- 172.21.15.22:0/2447613908 >> 172.21.15.22:0/2447613908 conn(0x7ff87c0b3c80 msgr2=0x7ff87c0b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:30.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.220+0000 7ff8835e9640 1 -- 172.21.15.22:0/2447613908 shutdown_connections 2024-03-29T04:18:30.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.220+0000 7ff8835e9640 1 -- 172.21.15.22:0/2447613908 wait complete. 2024-03-29T04:18:30.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff8835e9640 1 Processor -- start 2024-03-29T04:18:30.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff8835e9640 1 -- start start 2024-03-29T04:18:30.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff8835e9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff87c0ccd00 0x7ff87c14c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:30.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff8835e9640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff87c15f200 0x7ff87c149ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:30.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff8835e9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff87c15f5e0 0x7ff87c14a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:30.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff8835e9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c16cc90 con 0x7ff87c0ccd00 2024-03-29T04:18:30.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff8835e9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c15fa70 con 0x7ff87c15f5e0 2024-03-29T04:18:30.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff8835e9640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff87c16ce10 con 0x7ff87c15f200 2024-03-29T04:18:30.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff881385640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff87c0ccd00 0x7ff87c14c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:30.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff881385640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff87c0ccd00 0x7ff87c14c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36876/0 (socket says 172.21.15.22:36876) 2024-03-29T04:18:30.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff881385640 1 -- 172.21.15.22:0/63892974 learned_addr learned my addr 172.21.15.22:0/63892974 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:30.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff881b86640 1 --2- 172.21.15.22:0/63892974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff87c15f5e0 0x7ff87c14a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:30.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff880b84640 1 --2- 172.21.15.22:0/63892974 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff87c15f200 0x7ff87c149ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:30.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff881385640 1 -- 172.21.15.22:0/63892974 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff87c15f200 msgr2=0x7ff87c149ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:30.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff881385640 1 --2- 172.21.15.22:0/63892974 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff87c15f200 0x7ff87c149ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff881385640 1 -- 172.21.15.22:0/63892974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff87c15f5e0 msgr2=0x7ff87c14a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:30.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff881385640 1 --2- 172.21.15.22:0/63892974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff87c15f5e0 0x7ff87c14a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff881385640 1 -- 172.21.15.22:0/63892974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87c0c8fe0 con 0x7ff87c0ccd00 2024-03-29T04:18:30.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff881385640 1 --2- 172.21.15.22:0/63892974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff87c0ccd00 0x7ff87c14c920 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff87800c500 tx=0x7ff87800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:30.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff871ffb640 1 -- 172.21.15.22:0/63892974 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff878016020 con 0x7ff87c0ccd00 2024-03-29T04:18:30.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff871ffb640 1 -- 172.21.15.22:0/63892974 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff878005220 con 0x7ff87c0ccd00 2024-03-29T04:18:30.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff8835e9640 1 -- 172.21.15.22:0/63892974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff87c151140 con 0x7ff87c0ccd00 2024-03-29T04:18:30.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff871ffb640 1 -- 172.21.15.22:0/63892974 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff878007030 con 0x7ff87c0ccd00 2024-03-29T04:18:30.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff8835e9640 1 -- 172.21.15.22:0/63892974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff87c0c7030 con 0x7ff87c0ccd00 2024-03-29T04:18:30.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.224+0000 7ff8835e9640 1 -- 172.21.15.22:0/63892974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff87c075980 con 0x7ff87c0ccd00 2024-03-29T04:18:30.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.228+0000 7ff871ffb640 1 -- 172.21.15.22:0/63892974 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff8780071f0 con 0x7ff87c0ccd00 2024-03-29T04:18:30.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.228+0000 7ff871ffb640 1 --2- 172.21.15.22:0/63892974 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff860067fc0 0x7ff86006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:30.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.228+0000 7ff871ffb640 1 -- 172.21.15.22:0/63892974 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7ff878086300 con 0x7ff87c0ccd00 2024-03-29T04:18:30.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.228+0000 7ff880b84640 1 --2- 172.21.15.22:0/63892974 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff860067fc0 0x7ff86006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:30.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.228+0000 7ff880b84640 1 --2- 172.21.15.22:0/63892974 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff860067fc0 0x7ff86006a480 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0af000 tx=0x7ff86c0a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:30.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.232+0000 7ff871ffb640 1 -- 172.21.15.22:0/63892974 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff878020020 con 0x7ff87c0ccd00 2024-03-29T04:18:30.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.396+0000 7ff8835e9640 1 -- 172.21.15.22:0/63892974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff87c068900 con 0x7ff87c0ccd00 2024-03-29T04:18:30.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.396+0000 7ff871ffb640 1 -- 172.21.15.22:0/63892974 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7ff8780566c0 con 0x7ff87c0ccd00 2024-03-29T04:18:30.377 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:30.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.400+0000 7ff8835e9640 1 -- 172.21.15.22:0/63892974 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff860067fc0 msgr2=0x7ff86006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:30.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.400+0000 7ff8835e9640 1 --2- 172.21.15.22:0/63892974 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff860067fc0 0x7ff86006a480 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0af000 tx=0x7ff86c0a7000 comp rx=0 tx=0).stop 2024-03-29T04:18:30.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.400+0000 7ff8835e9640 1 -- 172.21.15.22:0/63892974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff87c0ccd00 msgr2=0x7ff87c14c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:30.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.400+0000 7ff8835e9640 1 --2- 172.21.15.22:0/63892974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff87c0ccd00 0x7ff87c14c920 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff87800c500 tx=0x7ff87800c9c0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.400+0000 7ff8835e9640 1 -- 172.21.15.22:0/63892974 shutdown_connections 2024-03-29T04:18:30.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.400+0000 7ff8835e9640 1 --2- 172.21.15.22:0/63892974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff87c15f5e0 0x7ff87c14a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.400+0000 7ff8835e9640 1 --2- 172.21.15.22:0/63892974 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff87c15f200 0x7ff87c149ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.400+0000 7ff8835e9640 1 --2- 172.21.15.22:0/63892974 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff860067fc0 0x7ff86006a480 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.400+0000 7ff8835e9640 1 --2- 172.21.15.22:0/63892974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff87c0ccd00 0x7ff87c14c920 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.400+0000 7ff8835e9640 1 -- 172.21.15.22:0/63892974 >> 172.21.15.22:0/63892974 conn(0x7ff87c0b3c80 msgr2=0x7ff87c14d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:30.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.400+0000 7ff8835e9640 1 -- 172.21.15.22:0/63892974 shutdown_connections 2024-03-29T04:18:30.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.400+0000 7ff8835e9640 1 -- 172.21.15.22:0/63892974 wait complete. 2024-03-29T04:18:30.397 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:18:30.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.532+0000 7f791724f640 1 Processor -- start 2024-03-29T04:18:30.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.532+0000 7f791724f640 1 -- start start 2024-03-29T04:18:30.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.532+0000 7f791724f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79100d0720 0x7f791006f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:30.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.532+0000 7f791724f640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f79100ce3c0 0x7f79100ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:30.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.532+0000 7f791724f640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79100d0bb0 con 0x7f791015c250 2024-03-29T04:18:30.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.532+0000 7f791724f640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f79100b4fc0 con 0x7f79100ce3c0 2024-03-29T04:18:30.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.532+0000 7f791724f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f79100cc7c0 con 0x7f79100d0720 2024-03-29T04:18:30.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.532+0000 7f7914feb640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f791015c250 0x7f791006e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.22:35696/0 (socket says 172.21.15.22:35696) 2024-03-29T04:18:30.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.532+0000 7f7914feb640 1 -- 172.21.15.22:0/2909205403 learned_addr learned my addr 172.21.15.22:0/2909205403 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:30.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.532+0000 7f790ffff640 1 --2- 172.21.15.22:0/2909205403 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79100d0720 0x7f791006f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:30.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.532+0000 7f79157ec640 1 --2- 172.21.15.22:0/2909205403 >> v2:172.21.15.151:3300/0 conn(0x7f79100ce3c0 0x7f79100ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:30.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.532+0000 7f790f7fe640 1 -- 172.21.15.22:0/2909205403 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2590325019 0 0) 0x7f79100d0bb0 con 0x7f791015c250 2024-03-29T04:18:30.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.532+0000 7f790f7fe640 1 -- 172.21.15.22:0/2909205403 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78fc003530 con 0x7f791015c250 2024-03-29T04:18:30.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.532+0000 7f790ffff640 1 -- 172.21.15.22:0/2909205403 >> v2:172.21.15.151:3300/0 conn(0x7f79100ce3c0 msgr2=0x7f79100ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:30.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.532+0000 7f790ffff640 1 --2- 172.21.15.22:0/2909205403 >> v2:172.21.15.151:3300/0 conn(0x7f79100ce3c0 0x7f79100ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.532+0000 7f790ffff640 1 -- 172.21.15.22:0/2909205403 >> v1:172.21.15.151:6789/0 conn(0x7f791015c250 legacy=0x7f791006e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:30.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.532+0000 7f790ffff640 1 -- 172.21.15.22:0/2909205403 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79100ccf30 con 0x7f79100d0720 2024-03-29T04:18:30.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.536+0000 7f790ffff640 1 --2- 172.21.15.22:0/2909205403 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79100d0720 0x7f791006f090 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f78f80097c0 tx=0x7f78f801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8036bc88124e665f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:30.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.536+0000 7f790f7fe640 1 -- 172.21.15.22:0/2909205403 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f80ab430 con 0x7f79100d0720 2024-03-29T04:18:30.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.536+0000 7f790f7fe640 1 -- 172.21.15.22:0/2909205403 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78f80b0070 con 0x7f79100d0720 2024-03-29T04:18:30.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.536+0000 7f790f7fe640 1 -- 172.21.15.22:0/2909205403 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f80ab7f0 con 0x7f79100d0720 2024-03-29T04:18:30.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.536+0000 7f791724f640 1 -- 172.21.15.22:0/2909205403 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79100d0720 msgr2=0x7f791006f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:30.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.536+0000 7f791724f640 1 --2- 172.21.15.22:0/2909205403 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79100d0720 0x7f791006f090 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f78f80097c0 tx=0x7f78f801c410 comp rx=0 tx=0).stop 2024-03-29T04:18:30.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.536+0000 7f791724f640 1 -- 172.21.15.22:0/2909205403 shutdown_connections 2024-03-29T04:18:30.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.536+0000 7f791724f640 1 --2- 172.21.15.22:0/2909205403 >> v2:172.21.15.151:3300/0 conn(0x7f79100ce3c0 0x7f79100ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.536+0000 7f791724f640 1 --2- 172.21.15.22:0/2909205403 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79100d0720 0x7f791006f090 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.536+0000 7f791724f640 1 -- 172.21.15.22:0/2909205403 >> 172.21.15.22:0/2909205403 conn(0x7f79100b3c80 msgr2=0x7f79100b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:30.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.536+0000 7f791724f640 1 -- 172.21.15.22:0/2909205403 shutdown_connections 2024-03-29T04:18:30.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.536+0000 7f791724f640 1 -- 172.21.15.22:0/2909205403 wait complete. 2024-03-29T04:18:30.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.536+0000 7f791724f640 1 Processor -- start 2024-03-29T04:18:30.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.536+0000 7f791724f640 1 -- start start 2024-03-29T04:18:30.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f791724f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79100ce3c0 0x7f7910154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:30.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f791724f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79100d0720 0x7f79101fafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:30.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f791724f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f791015c250 0x7f79101fb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:30.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f791724f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f791006f5d0 con 0x7f79100d0720 2024-03-29T04:18:30.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f791724f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f791015c630 con 0x7f791015c250 2024-03-29T04:18:30.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f791724f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f791006f750 con 0x7f79100ce3c0 2024-03-29T04:18:30.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f790ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79100d0720 0x7f79101fafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:30.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f790ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79100d0720 0x7f79101fafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36890/0 (socket says 172.21.15.22:36890) 2024-03-29T04:18:30.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f790ffff640 1 -- 172.21.15.22:0/3288370105 learned_addr learned my addr 172.21.15.22:0/3288370105 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:30.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f79157ec640 1 --2- 172.21.15.22:0/3288370105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f791015c250 0x7f79101fb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:30.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f7914feb640 1 --2- 172.21.15.22:0/3288370105 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79100ce3c0 0x7f7910154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:30.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f790ffff640 1 -- 172.21.15.22:0/3288370105 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79100ce3c0 msgr2=0x7f7910154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:30.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f790ffff640 1 --2- 172.21.15.22:0/3288370105 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79100ce3c0 0x7f7910154700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f790ffff640 1 -- 172.21.15.22:0/3288370105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f791015c250 msgr2=0x7f79101fb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:30.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f790ffff640 1 --2- 172.21.15.22:0/3288370105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f791015c250 0x7f79101fb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f790ffff640 1 -- 172.21.15.22:0/3288370105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79100c9370 con 0x7f79100d0720 2024-03-29T04:18:30.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f790ffff640 1 --2- 172.21.15.22:0/3288370105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79100d0720 0x7f79101fafd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f78f801aac0 tx=0x7f78f80191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:30.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f790d7fa640 1 -- 172.21.15.22:0/3288370105 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f80abdd0 con 0x7f79100d0720 2024-03-29T04:18:30.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f790d7fa640 1 -- 172.21.15.22:0/3288370105 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78f80bd480 con 0x7f79100d0720 2024-03-29T04:18:30.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f791724f640 1 -- 172.21.15.22:0/3288370105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79100c9650 con 0x7f79100d0720 2024-03-29T04:18:30.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f790d7fa640 1 -- 172.21.15.22:0/3288370105 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f80bd870 con 0x7f79100d0720 2024-03-29T04:18:30.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f791724f640 1 -- 172.21.15.22:0/3288370105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7910154c40 con 0x7f79100d0720 2024-03-29T04:18:30.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f791724f640 1 -- 172.21.15.22:0/3288370105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7910068e60 con 0x7f79100d0720 2024-03-29T04:18:30.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f790d7fa640 1 -- 172.21.15.22:0/3288370105 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f78f80c63f0 con 0x7f79100d0720 2024-03-29T04:18:30.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f790d7fa640 1 --2- 172.21.15.22:0/3288370105 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f78f0067ff0 0x7f78f006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:30.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f790d7fa640 1 -- 172.21.15.22:0/3288370105 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f78f8120eb0 con 0x7f79100d0720 2024-03-29T04:18:30.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.540+0000 7f7914feb640 1 --2- 172.21.15.22:0/3288370105 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f78f0067ff0 0x7f78f006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:30.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.544+0000 7f7914feb640 1 --2- 172.21.15.22:0/3288370105 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f78f0067ff0 0x7f78f006a4b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7908005890 tx=0x7f7908009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:30.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.544+0000 7f790d7fa640 1 -- 172.21.15.22:0/3288370105 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f78f8016090 con 0x7f79100d0720 2024-03-29T04:18:30.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.708+0000 7f791724f640 1 -- 172.21.15.22:0/3288370105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7910166c30 con 0x7f79100d0720 2024-03-29T04:18:30.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.708+0000 7f790d7fa640 1 -- 172.21.15.22:0/3288370105 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f78f80f1410 con 0x7f79100d0720 2024-03-29T04:18:30.688 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-29T04:18:30.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.708+0000 7f791724f640 1 -- 172.21.15.22:0/3288370105 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f78f0067ff0 msgr2=0x7f78f006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:30.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.708+0000 7f791724f640 1 --2- 172.21.15.22:0/3288370105 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f78f0067ff0 0x7f78f006a4b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7908005890 tx=0x7f7908009040 comp rx=0 tx=0).stop 2024-03-29T04:18:30.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.708+0000 7f791724f640 1 -- 172.21.15.22:0/3288370105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79100d0720 msgr2=0x7f79101fafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:30.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.708+0000 7f791724f640 1 --2- 172.21.15.22:0/3288370105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79100d0720 0x7f79101fafd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f78f801aac0 tx=0x7f78f80191b0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.708+0000 7f791724f640 1 -- 172.21.15.22:0/3288370105 shutdown_connections 2024-03-29T04:18:30.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.708+0000 7f791724f640 1 --2- 172.21.15.22:0/3288370105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f791015c250 0x7f79101fb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.708+0000 7f791724f640 1 --2- 172.21.15.22:0/3288370105 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f78f0067ff0 0x7f78f006a4b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.708+0000 7f791724f640 1 --2- 172.21.15.22:0/3288370105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79100d0720 0x7f79101fafd0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.708+0000 7f791724f640 1 --2- 172.21.15.22:0/3288370105 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f79100ce3c0 0x7f7910154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.708+0000 7f791724f640 1 -- 172.21.15.22:0/3288370105 >> 172.21.15.22:0/3288370105 conn(0x7f79100b3c80 msgr2=0x7f791006e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:30.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.708+0000 7f791724f640 1 -- 172.21.15.22:0/3288370105 shutdown_connections 2024-03-29T04:18:30.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.708+0000 7f791724f640 1 -- 172.21.15.22:0/3288370105 wait complete. 2024-03-29T04:18:30.707 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:18:30.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.856+0000 7f2512254640 1 Processor -- start 2024-03-29T04:18:30.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.856+0000 7f2512254640 1 -- start start 2024-03-29T04:18:30.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.856+0000 7f2512254640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f250c06f4b0 0x7f250c1654d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:30.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.856+0000 7f2512254640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f250c15ace0 con 0x7f250c06c070 2024-03-29T04:18:30.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.856+0000 7f2512254640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f250c06c570 con 0x7f250c15a900 2024-03-29T04:18:30.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.856+0000 7f2512254640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f250c06c850 con 0x7f250c06f4b0 2024-03-29T04:18:30.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.856+0000 7f250affd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f250c06c070 0x7f250c06eda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57044/0 (socket says 172.21.15.22:57044) 2024-03-29T04:18:30.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.856+0000 7f250affd640 1 -- 172.21.15.22:0/2428337199 learned_addr learned my addr 172.21.15.22:0/2428337199 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:30.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.856+0000 7f250bfff640 1 --2- 172.21.15.22:0/2428337199 >> v2:172.21.15.22:3300/0 conn(0x7f250c06f4b0 0x7f250c1654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:30.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.860+0000 7f250bfff640 1 -- 172.21.15.22:0/2428337199 >> v1:172.21.15.151:6789/0 conn(0x7f250c15a900 legacy=0x7f250c0afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:30.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.860+0000 7f250bfff640 1 -- 172.21.15.22:0/2428337199 >> v1:172.21.15.22:6789/0 conn(0x7f250c06c070 legacy=0x7f250c06eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:30.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.860+0000 7f250bfff640 1 -- 172.21.15.22:0/2428337199 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250c06cb30 con 0x7f250c06f4b0 2024-03-29T04:18:30.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.860+0000 7f250bfff640 1 --2- 172.21.15.22:0/2428337199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f250c06f4b0 0x7f250c1654d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f25040097c0 tx=0x7f250401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24623a71484d6048 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:30.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.860+0000 7f250a7fc640 1 -- 172.21.15.22:0/2428337199 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25040b1070 con 0x7f250c06f4b0 2024-03-29T04:18:30.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.860+0000 7f250a7fc640 1 -- 172.21.15.22:0/2428337199 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25040ac410 con 0x7f250c06f4b0 2024-03-29T04:18:30.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.860+0000 7f250a7fc640 1 -- 172.21.15.22:0/2428337199 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25040ac7e0 con 0x7f250c06f4b0 2024-03-29T04:18:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.860+0000 7f2512254640 1 -- 172.21.15.22:0/2428337199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f250c06f4b0 msgr2=0x7f250c1654d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.860+0000 7f2512254640 1 --2- 172.21.15.22:0/2428337199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f250c06f4b0 0x7f250c1654d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f25040097c0 tx=0x7f250401c4a0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.860+0000 7f2512254640 1 -- 172.21.15.22:0/2428337199 shutdown_connections 2024-03-29T04:18:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.860+0000 7f2512254640 1 --2- 172.21.15.22:0/2428337199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f250c06f4b0 0x7f250c1654d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.860+0000 7f2512254640 1 -- 172.21.15.22:0/2428337199 >> 172.21.15.22:0/2428337199 conn(0x7f250c0d0c90 msgr2=0x7f250c0d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.860+0000 7f2512254640 1 -- 172.21.15.22:0/2428337199 shutdown_connections 2024-03-29T04:18:30.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.860+0000 7f2512254640 1 -- 172.21.15.22:0/2428337199 wait complete. 2024-03-29T04:18:30.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f2512254640 1 Processor -- start 2024-03-29T04:18:30.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f2512254640 1 -- start start 2024-03-29T04:18:30.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f2512254640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f250c06c070 0x7f250c13dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:30.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f2512254640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f250c06f4b0 0x7f250c157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:30.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f2512254640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f250c15a900 0x7f250c159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:30.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f2512254640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f250c1689d0 con 0x7f250c06f4b0 2024-03-29T04:18:30.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f2512254640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f250c168850 con 0x7f250c06c070 2024-03-29T04:18:30.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f2512254640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f250c168b50 con 0x7f250c15a900 2024-03-29T04:18:30.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f250affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f250c06f4b0 0x7f250c157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:30.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f250affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f250c06f4b0 0x7f250c157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36906/0 (socket says 172.21.15.22:36906) 2024-03-29T04:18:30.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f250affd640 1 -- 172.21.15.22:0/2967314635 learned_addr learned my addr 172.21.15.22:0/2967314635 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:30.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f250b7fe640 1 --2- 172.21.15.22:0/2967314635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f250c06c070 0x7f250c13dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:30.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f250bfff640 1 --2- 172.21.15.22:0/2967314635 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f250c15a900 0x7f250c159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:30.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f250affd640 1 -- 172.21.15.22:0/2967314635 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f250c15a900 msgr2=0x7f250c159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:30.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f250affd640 1 --2- 172.21.15.22:0/2967314635 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f250c15a900 0x7f250c159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f250affd640 1 -- 172.21.15.22:0/2967314635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f250c06c070 msgr2=0x7f250c13dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:30.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f250affd640 1 --2- 172.21.15.22:0/2967314635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f250c06c070 0x7f250c13dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:30.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f250affd640 1 -- 172.21.15.22:0/2967314635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250c1582c0 con 0x7f250c06f4b0 2024-03-29T04:18:30.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f250b7fe640 1 --2- 172.21.15.22:0/2967314635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f250c06c070 0x7f250c13dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:30.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f250affd640 1 --2- 172.21.15.22:0/2967314635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f250c06f4b0 0x7f250c157d80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f24f400c450 tx=0x7f24f400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:30.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f2503fff640 1 -- 172.21.15.22:0/2967314635 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f4017070 con 0x7f250c06f4b0 2024-03-29T04:18:30.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f2503fff640 1 -- 172.21.15.22:0/2967314635 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f4005cf0 con 0x7f250c06f4b0 2024-03-29T04:18:30.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f2512254640 1 -- 172.21.15.22:0/2967314635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250c153c30 con 0x7f250c06f4b0 2024-03-29T04:18:30.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f2503fff640 1 -- 172.21.15.22:0/2967314635 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f4007270 con 0x7f250c06f4b0 2024-03-29T04:18:30.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f2512254640 1 -- 172.21.15.22:0/2967314635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f250c1541f0 con 0x7f250c06f4b0 2024-03-29T04:18:30.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f2512254640 1 -- 172.21.15.22:0/2967314635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f250c068900 con 0x7f250c06f4b0 2024-03-29T04:18:30.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.864+0000 7f2503fff640 1 -- 172.21.15.22:0/2967314635 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f24f401a070 con 0x7f250c06f4b0 2024-03-29T04:18:30.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.868+0000 7f2503fff640 1 --2- 172.21.15.22:0/2967314635 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f24ec067fc0 0x7f24ec06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:30.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.868+0000 7f2503fff640 1 -- 172.21.15.22:0/2967314635 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f24f4085b10 con 0x7f250c06f4b0 2024-03-29T04:18:30.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.868+0000 7f250b7fe640 1 --2- 172.21.15.22:0/2967314635 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f24ec067fc0 0x7f24ec06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:30.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.868+0000 7f250b7fe640 1 --2- 172.21.15.22:0/2967314635 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f24ec067fc0 0x7f24ec06a480 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f24fc001d70 tx=0x7f24fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:30.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:30.872+0000 7f2503fff640 1 -- 172.21.15.22:0/2967314635 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f24f4055870 con 0x7f250c06f4b0 2024-03-29T04:18:31.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.036+0000 7f2512254640 1 -- 172.21.15.22:0/2967314635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f250c0cccc0 con 0x7f250c06f4b0 2024-03-29T04:18:31.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.036+0000 7f2503fff640 1 -- 172.21.15.22:0/2967314635 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f24f4059b70 con 0x7f250c06f4b0 2024-03-29T04:18:31.019 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:31.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.040+0000 7f2512254640 1 -- 172.21.15.22:0/2967314635 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f24ec067fc0 msgr2=0x7f24ec06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:31.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.040+0000 7f2512254640 1 --2- 172.21.15.22:0/2967314635 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f24ec067fc0 0x7f24ec06a480 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f24fc001d70 tx=0x7f24fc009040 comp rx=0 tx=0).stop 2024-03-29T04:18:31.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.040+0000 7f2512254640 1 -- 172.21.15.22:0/2967314635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f250c06f4b0 msgr2=0x7f250c157d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:31.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.040+0000 7f2512254640 1 --2- 172.21.15.22:0/2967314635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f250c06f4b0 0x7f250c157d80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f24f400c450 tx=0x7f24f400c910 comp rx=0 tx=0).stop 2024-03-29T04:18:31.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.040+0000 7f2512254640 1 -- 172.21.15.22:0/2967314635 shutdown_connections 2024-03-29T04:18:31.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.040+0000 7f2512254640 1 --2- 172.21.15.22:0/2967314635 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f250c15a900 0x7f250c159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.040+0000 7f2512254640 1 --2- 172.21.15.22:0/2967314635 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f24ec067fc0 0x7f24ec06a480 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.040+0000 7f2512254640 1 --2- 172.21.15.22:0/2967314635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f250c06f4b0 0x7f250c157d80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.040+0000 7f2512254640 1 --2- 172.21.15.22:0/2967314635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f250c06c070 0x7f250c13dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.040+0000 7f2512254640 1 -- 172.21.15.22:0/2967314635 >> 172.21.15.22:0/2967314635 conn(0x7f250c0d0c90 msgr2=0x7f250c0d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:31.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.040+0000 7f2512254640 1 -- 172.21.15.22:0/2967314635 shutdown_connections 2024-03-29T04:18:31.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.040+0000 7f2512254640 1 -- 172.21.15.22:0/2967314635 wait complete. 2024-03-29T04:18:31.039 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:18:31.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.184+0000 7f590cf3a640 1 Processor -- start 2024-03-29T04:18:31.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.184+0000 7f590cf3a640 1 -- start start 2024-03-29T04:18:31.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.184+0000 7f590cf3a640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f59080d1840 0x7f59080ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.184+0000 7f590cf3a640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59080cfd20 con 0x7f59080d1cd0 2024-03-29T04:18:31.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.184+0000 7f590cf3a640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59080d0000 con 0x7f59080b7e10 2024-03-29T04:18:31.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.184+0000 7f590cf3a640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f59080d0b80 con 0x7f59080d1840 2024-03-29T04:18:31.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.184+0000 7f5906d9d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f59080d1cd0 0x7f59081666f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57052/0 (socket says 172.21.15.22:57052) 2024-03-29T04:18:31.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.184+0000 7f5906d9d640 1 -- 172.21.15.22:0/1590255499 learned_addr learned my addr 172.21.15.22:0/1590255499 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:31.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.184+0000 7f5905d9b640 1 --2- 172.21.15.22:0/1590255499 >> v2:172.21.15.151:3300/0 conn(0x7f59080d1840 0x7f59080ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.184+0000 7f590559a640 1 -- 172.21.15.22:0/1590255499 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1815680507 0 0) 0x7f59080cfd20 con 0x7f59080d1cd0 2024-03-29T04:18:31.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.184+0000 7f590559a640 1 -- 172.21.15.22:0/1590255499 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58f0003530 con 0x7f59080d1cd0 2024-03-29T04:18:31.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.184+0000 7f590559a640 1 -- 172.21.15.22:0/1590255499 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3128602096 0 0) 0x7f58f0003530 con 0x7f59080d1cd0 2024-03-29T04:18:31.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.184+0000 7f590559a640 1 -- 172.21.15.22:0/1590255499 >> v2:172.21.15.151:3300/0 conn(0x7f59080d1840 msgr2=0x7f59080ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:31.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.184+0000 7f590559a640 1 --2- 172.21.15.22:0/1590255499 >> v2:172.21.15.151:3300/0 conn(0x7f59080d1840 0x7f59080ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.184+0000 7f590559a640 1 -- 172.21.15.22:0/1590255499 >> v1:172.21.15.151:6789/0 conn(0x7f59080b7e10 legacy=0x7f59080b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:31.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.184+0000 7f590559a640 1 -- 172.21.15.22:0/1590255499 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59080d02e0 con 0x7f59080d1cd0 2024-03-29T04:18:31.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.184+0000 7f590559a640 1 -- 172.21.15.22:0/1590255499 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f5900001a00 con 0x7f59080d1cd0 2024-03-29T04:18:31.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590559a640 1 -- 172.21.15.22:0/1590255499 >> v1:172.21.15.22:6789/0 conn(0x7f59080d1cd0 legacy=0x7f59081666f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:31.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590559a640 1 --2- 172.21.15.22:0/1590255499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f00039c0 0x7f58f0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590559a640 1 --2- 172.21.15.22:0/1590255499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58f0004810 0x7f58f0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590559a640 1 --2- 172.21.15.22:0/1590255499 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f00051d0 0x7f58f000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590559a640 1 -- 172.21.15.22:0/1590255499 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f59080d0b80 con 0x7f58f00039c0 2024-03-29T04:18:31.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590559a640 1 -- 172.21.15.22:0/1590255499 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f58f000c0d0 con 0x7f58f0004810 2024-03-29T04:18:31.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590559a640 1 -- 172.21.15.22:0/1590255499 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f58f000c2d0 con 0x7f58f00051d0 2024-03-29T04:18:31.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590659c640 1 --2- 172.21.15.22:0/1590255499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f00039c0 0x7f58f0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f5905d9b640 1 --2- 172.21.15.22:0/1590255499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58f0004810 0x7f58f0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f5906d9d640 1 --2- 172.21.15.22:0/1590255499 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f00051d0 0x7f58f000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590659c640 1 -- 172.21.15.22:0/1590255499 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f00051d0 msgr2=0x7f58f000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:31.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590659c640 1 --2- 172.21.15.22:0/1590255499 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f00051d0 0x7f58f000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590659c640 1 -- 172.21.15.22:0/1590255499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58f0004810 msgr2=0x7f58f0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:31.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590659c640 1 --2- 172.21.15.22:0/1590255499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58f0004810 0x7f58f0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590659c640 1 -- 172.21.15.22:0/1590255499 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59080d02e0 con 0x7f58f00039c0 2024-03-29T04:18:31.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f5906d9d640 1 --2- 172.21.15.22:0/1590255499 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f00051d0 0x7f58f000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:31.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f5906d9d640 1 -- 172.21.15.22:0/1590255499 reap_dead start 2024-03-29T04:18:31.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590659c640 1 --2- 172.21.15.22:0/1590255499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f00039c0 0x7f58f0003de0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f58f8001780 tx=0x7f58f801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57557413f9528e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:31.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590559a640 1 -- 172.21.15.22:0/1590255499 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f80a8050 con 0x7f58f00039c0 2024-03-29T04:18:31.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590559a640 1 -- 172.21.15.22:0/1590255499 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f80052f0 con 0x7f58f00039c0 2024-03-29T04:18:31.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590559a640 1 -- 172.21.15.22:0/1590255499 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f80013a0 con 0x7f58f00039c0 2024-03-29T04:18:31.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590cf3a640 1 -- 172.21.15.22:0/1590255499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f00039c0 msgr2=0x7f58f0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:31.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590cf3a640 1 --2- 172.21.15.22:0/1590255499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f00039c0 0x7f58f0003de0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f58f8001780 tx=0x7f58f801c400 comp rx=0 tx=0).stop 2024-03-29T04:18:31.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590cf3a640 1 -- 172.21.15.22:0/1590255499 shutdown_connections 2024-03-29T04:18:31.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590cf3a640 1 --2- 172.21.15.22:0/1590255499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f00039c0 0x7f58f0003de0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590cf3a640 1 -- 172.21.15.22:0/1590255499 >> 172.21.15.22:0/1590255499 conn(0x7f59080b3c80 msgr2=0x7f59080b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:31.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590cf3a640 1 -- 172.21.15.22:0/1590255499 shutdown_connections 2024-03-29T04:18:31.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.188+0000 7f590cf3a640 1 -- 172.21.15.22:0/1590255499 wait complete. 2024-03-29T04:18:31.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f590cf3a640 1 Processor -- start 2024-03-29T04:18:31.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f590cf3a640 1 -- start start 2024-03-29T04:18:31.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f590cf3a640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f00039c0 0x7f59081524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f590cf3a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5908162350 0x7f59081627b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f590cf3a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f590806dfa0 0x7f590806e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f590cf3a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f59080ceb00 con 0x7f5908162350 2024-03-29T04:18:31.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f590cf3a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f59080d08a0 con 0x7f590806dfa0 2024-03-29T04:18:31.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f590cf3a640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f59080cec80 con 0x7f58f00039c0 2024-03-29T04:18:31.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f5905d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5908162350 0x7f59081627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f5905d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5908162350 0x7f59081627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36924/0 (socket says 172.21.15.22:36924) 2024-03-29T04:18:31.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f5905d9b640 1 -- 172.21.15.22:0/2969239304 learned_addr learned my addr 172.21.15.22:0/2969239304 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:31.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f590659c640 1 --2- 172.21.15.22:0/2969239304 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f00039c0 0x7f59081524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f5906d9d640 1 --2- 172.21.15.22:0/2969239304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f590806dfa0 0x7f590806e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f5905d9b640 1 -- 172.21.15.22:0/2969239304 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f00039c0 msgr2=0x7f59081524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:31.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f5905d9b640 1 --2- 172.21.15.22:0/2969239304 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f00039c0 0x7f59081524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f5905d9b640 1 -- 172.21.15.22:0/2969239304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f590806dfa0 msgr2=0x7f590806e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:31.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f5905d9b640 1 --2- 172.21.15.22:0/2969239304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f590806dfa0 0x7f590806e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f5905d9b640 1 -- 172.21.15.22:0/2969239304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59081608e0 con 0x7f5908162350 2024-03-29T04:18:31.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f590659c640 1 --2- 172.21.15.22:0/2969239304 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f00039c0 0x7f59081524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:31.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f5905d9b640 1 --2- 172.21.15.22:0/2969239304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5908162350 0x7f59081627b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f58fc011690 tx=0x7f58fc011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:31.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f58f6ffd640 1 -- 172.21.15.22:0/2969239304 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc00f960 con 0x7f5908162350 2024-03-29T04:18:31.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f58f6ffd640 1 -- 172.21.15.22:0/2969239304 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58fc00fb00 con 0x7f5908162350 2024-03-29T04:18:31.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f590cf3a640 1 -- 172.21.15.22:0/2969239304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5908160bc0 con 0x7f5908162350 2024-03-29T04:18:31.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f58f6ffd640 1 -- 172.21.15.22:0/2969239304 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc015460 con 0x7f5908162350 2024-03-29T04:18:31.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f590cf3a640 1 -- 172.21.15.22:0/2969239304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5908161180 con 0x7f5908162350 2024-03-29T04:18:31.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f590cf3a640 1 -- 172.21.15.22:0/2969239304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5908068890 con 0x7f5908162350 2024-03-29T04:18:31.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.192+0000 7f58f6ffd640 1 -- 172.21.15.22:0/2969239304 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f58fc015600 con 0x7f5908162350 2024-03-29T04:18:31.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.196+0000 7f58f6ffd640 1 --2- 172.21.15.22:0/2969239304 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f58e8067fc0 0x7f58e806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.196+0000 7f58f6ffd640 1 -- 172.21.15.22:0/2969239304 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f58fc0871b0 con 0x7f5908162350 2024-03-29T04:18:31.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.196+0000 7f590659c640 1 --2- 172.21.15.22:0/2969239304 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f58e8067fc0 0x7f58e806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.196+0000 7f590659c640 1 --2- 172.21.15.22:0/2969239304 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f58e8067fc0 0x7f58e806a480 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f58f8001750 tx=0x7f58f80011a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:31.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.200+0000 7f58f6ffd640 1 -- 172.21.15.22:0/2969239304 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f58fc019030 con 0x7f5908162350 2024-03-29T04:18:31.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.372+0000 7f590cf3a640 1 -- 172.21.15.22:0/2969239304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f59080d1620 con 0x7f5908162350 2024-03-29T04:18:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.372+0000 7f58f6ffd640 1 -- 172.21.15.22:0/2969239304 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f58fc057560 con 0x7f5908162350 2024-03-29T04:18:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-03-29T04:18:31.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.376+0000 7f590cf3a640 1 -- 172.21.15.22:0/2969239304 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f58e8067fc0 msgr2=0x7f58e806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:31.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.376+0000 7f590cf3a640 1 --2- 172.21.15.22:0/2969239304 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f58e8067fc0 0x7f58e806a480 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f58f8001750 tx=0x7f58f80011a0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.376+0000 7f590cf3a640 1 -- 172.21.15.22:0/2969239304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5908162350 msgr2=0x7f59081627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:31.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.376+0000 7f590cf3a640 1 --2- 172.21.15.22:0/2969239304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5908162350 0x7f59081627b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f58fc011690 tx=0x7f58fc011b50 comp rx=0 tx=0).stop 2024-03-29T04:18:31.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.376+0000 7f590cf3a640 1 -- 172.21.15.22:0/2969239304 shutdown_connections 2024-03-29T04:18:31.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.376+0000 7f590cf3a640 1 --2- 172.21.15.22:0/2969239304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f590806dfa0 0x7f590806e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.376+0000 7f590cf3a640 1 --2- 172.21.15.22:0/2969239304 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f58e8067fc0 0x7f58e806a480 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.376+0000 7f590cf3a640 1 --2- 172.21.15.22:0/2969239304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5908162350 0x7f59081627b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.376+0000 7f590cf3a640 1 --2- 172.21.15.22:0/2969239304 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58f00039c0 0x7f59081524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.376+0000 7f590cf3a640 1 -- 172.21.15.22:0/2969239304 >> 172.21.15.22:0/2969239304 conn(0x7f59080b3c80 msgr2=0x7f59080c69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:31.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.376+0000 7f590cf3a640 1 -- 172.21.15.22:0/2969239304 shutdown_connections 2024-03-29T04:18:31.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.376+0000 7f590cf3a640 1 -- 172.21.15.22:0/2969239304 wait complete. 2024-03-29T04:18:31.373 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi022.front.sepia.ceph.com... 2024-03-29T04:18:31.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:31.374 DEBUG:teuthology.orchestra.run.smithi022:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T04:18:31.384 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:18:31.384 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-03-29T04:18:31.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:31.385 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T04:18:31.406 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:18:31.430 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:18:31.432 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:18:31.432 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:18:31.438 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:18:31.438 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:18:31.526 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.676+0000 7f1e51735640 1 Processor -- start 2024-03-29T04:18:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.676+0000 7f1e51735640 1 -- start start 2024-03-29T04:18:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.680+0000 7f1e51735640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4c0afa50 0x7f1e4c0cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.680+0000 7f1e51735640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1e4c1418d0 0x7f1e4c141d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.680+0000 7f1e51735640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e4c0ccb60 con 0x7f1e4c067fd0 2024-03-29T04:18:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.680+0000 7f1e51735640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f1e4c0cc910 con 0x7f1e4c1418d0 2024-03-29T04:18:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.680+0000 7f1e51735640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1e4c0cce40 con 0x7f1e4c0afa50 2024-03-29T04:18:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.680+0000 7f1e4b7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1e4c1418d0 0x7f1e4c141d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.680+0000 7f1e4affd640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4c0afa50 0x7f1e4c0cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.680+0000 7f1e4affd640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4c0afa50 0x7f1e4c0cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.22:36908/0 (socket says 172.21.15.22:36908) 2024-03-29T04:18:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.680+0000 7f1e4b7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1e4c1418d0 0x7f1e4c141d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36932/0 (socket says 172.21.15.22:36932) 2024-03-29T04:18:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.680+0000 7f1e4affd640 1 -- 172.21.15.22:0/2492357788 learned_addr learned my addr 172.21.15.22:0/2492357788 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:31.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.680+0000 7f1e4b7fe640 1 -- 172.21.15.22:0/2492357788 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4c0afa50 msgr2=0x7f1e4c0cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:31.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.680+0000 7f1e4b7fe640 1 --2- 172.21.15.22:0/2492357788 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4c0afa50 0x7f1e4c0cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.680+0000 7f1e4b7fe640 1 -- 172.21.15.22:0/2492357788 >> v1:172.21.15.151:6789/0 conn(0x7f1e4c067fd0 legacy=0x7f1e4c0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:31.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.680+0000 7f1e4b7fe640 1 -- 172.21.15.22:0/2492357788 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4c0cd120 con 0x7f1e4c1418d0 2024-03-29T04:18:31.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.680+0000 7f1e4b7fe640 1 --2- 172.21.15.22:0/2492357788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4c1418d0 0x7f1e4c141d60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009d90 tx=0x7f1e4401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19582e5ef338601c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:31.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.680+0000 7f1e4a7fc640 1 -- 172.21.15.22:0/2492357788 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e440b2070 con 0x7f1e4c1418d0 2024-03-29T04:18:31.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.680+0000 7f1e4a7fc640 1 -- 172.21.15.22:0/2492357788 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e440a57a0 con 0x7f1e4c1418d0 2024-03-29T04:18:31.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.680+0000 7f1e4a7fc640 1 -- 172.21.15.22:0/2492357788 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e440a5b70 con 0x7f1e4c1418d0 2024-03-29T04:18:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e51735640 1 -- 172.21.15.22:0/2492357788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4c1418d0 msgr2=0x7f1e4c141d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e51735640 1 --2- 172.21.15.22:0/2492357788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4c1418d0 0x7f1e4c141d60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009d90 tx=0x7f1e4401c8e0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e51735640 1 -- 172.21.15.22:0/2492357788 shutdown_connections 2024-03-29T04:18:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e51735640 1 --2- 172.21.15.22:0/2492357788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4c1418d0 0x7f1e4c141d60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e51735640 1 --2- 172.21.15.22:0/2492357788 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4c0afa50 0x7f1e4c0cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e51735640 1 -- 172.21.15.22:0/2492357788 >> 172.21.15.22:0/2492357788 conn(0x7f1e4c0d0710 msgr2=0x7f1e4c0d0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e51735640 1 -- 172.21.15.22:0/2492357788 shutdown_connections 2024-03-29T04:18:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e51735640 1 -- 172.21.15.22:0/2492357788 wait complete. 2024-03-29T04:18:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e51735640 1 Processor -- start 2024-03-29T04:18:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e51735640 1 -- start start 2024-03-29T04:18:31.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e51735640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4c067fd0 0x7f1e4c142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e51735640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4c0afa50 0x7f1e4c1442d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e51735640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e4c1418d0 0x7f1e4c15de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e51735640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c06c200 con 0x7f1e4c0afa50 2024-03-29T04:18:31.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e51735640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c06c080 con 0x7f1e4c1418d0 2024-03-29T04:18:31.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e51735640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1e4c06c380 con 0x7f1e4c067fd0 2024-03-29T04:18:31.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e43fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4c0afa50 0x7f1e4c1442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e43fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4c0afa50 0x7f1e4c1442d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36938/0 (socket says 172.21.15.22:36938) 2024-03-29T04:18:31.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e43fff640 1 -- 172.21.15.22:0/344079186 learned_addr learned my addr 172.21.15.22:0/344079186 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:31.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e4affd640 1 --2- 172.21.15.22:0/344079186 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4c067fd0 0x7f1e4c142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e4b7fe640 1 --2- 172.21.15.22:0/344079186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e4c1418d0 0x7f1e4c15de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e43fff640 1 -- 172.21.15.22:0/344079186 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4c067fd0 msgr2=0x7f1e4c142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e43fff640 1 --2- 172.21.15.22:0/344079186 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4c067fd0 0x7f1e4c142e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e43fff640 1 -- 172.21.15.22:0/344079186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e4c1418d0 msgr2=0x7f1e4c15de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e43fff640 1 --2- 172.21.15.22:0/344079186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e4c1418d0 0x7f1e4c15de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e43fff640 1 -- 172.21.15.22:0/344079186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4c139b10 con 0x7f1e4c0afa50 2024-03-29T04:18:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e4affd640 1 --2- 172.21.15.22:0/344079186 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4c067fd0 0x7f1e4c142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e43fff640 1 --2- 172.21.15.22:0/344079186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4c0afa50 0x7f1e4c1442d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400c390 tx=0x7f1e3400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e437fe640 1 -- 172.21.15.22:0/344079186 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34017070 con 0x7f1e4c0afa50 2024-03-29T04:18:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e437fe640 1 -- 172.21.15.22:0/344079186 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e340052d0 con 0x7f1e4c0afa50 2024-03-29T04:18:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e51735640 1 -- 172.21.15.22:0/344079186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4c1434e0 con 0x7f1e4c0afa50 2024-03-29T04:18:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e437fe640 1 -- 172.21.15.22:0/344079186 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34003820 con 0x7f1e4c0afa50 2024-03-29T04:18:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.684+0000 7f1e51735640 1 -- 172.21.15.22:0/344079186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e4c143d20 con 0x7f1e4c0afa50 2024-03-29T04:18:31.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.688+0000 7f1e51735640 1 -- 172.21.15.22:0/344079186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e4c074550 con 0x7f1e4c0afa50 2024-03-29T04:18:31.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.688+0000 7f1e437fe640 1 -- 172.21.15.22:0/344079186 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1e3401a070 con 0x7f1e4c0afa50 2024-03-29T04:18:31.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.688+0000 7f1e437fe640 1 --2- 172.21.15.22:0/344079186 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1e2c067fc0 0x7f1e2c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.688+0000 7f1e437fe640 1 -- 172.21.15.22:0/344079186 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f1e34085b90 con 0x7f1e4c0afa50 2024-03-29T04:18:31.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.688+0000 7f1e4affd640 1 --2- 172.21.15.22:0/344079186 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1e2c067fc0 0x7f1e2c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.688+0000 7f1e4affd640 1 --2- 172.21.15.22:0/344079186 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1e2c067fc0 0x7f1e2c06a480 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c009b00 tx=0x7f1e3c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:31.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.692+0000 7f1e437fe640 1 -- 172.21.15.22:0/344079186 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1e4c1434e0 con 0x7f1e4c0afa50 2024-03-29T04:18:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.852+0000 7f1e51735640 1 -- 172.21.15.22:0/344079186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e4c166750 con 0x7f1e4c0afa50 2024-03-29T04:18:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.856+0000 7f1e437fe640 1 -- 172.21.15.22:0/344079186 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18847 (secure 0 0 0) 0x7f1e34005480 con 0x7f1e4c0afa50 2024-03-29T04:18:31.836 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:31.837 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:18:16.893439+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.856+0000 7f1e51735640 1 -- 172.21.15.22:0/344079186 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1e2c067fc0 msgr2=0x7f1e2c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.856+0000 7f1e51735640 1 --2- 172.21.15.22:0/344079186 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1e2c067fc0 0x7f1e2c06a480 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c009b00 tx=0x7f1e3c008040 comp rx=0 tx=0).stop 2024-03-29T04:18:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.856+0000 7f1e51735640 1 -- 172.21.15.22:0/344079186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4c0afa50 msgr2=0x7f1e4c1442d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.856+0000 7f1e51735640 1 --2- 172.21.15.22:0/344079186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4c0afa50 0x7f1e4c1442d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400c390 tx=0x7f1e3400c850 comp rx=0 tx=0).stop 2024-03-29T04:18:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.856+0000 7f1e51735640 1 -- 172.21.15.22:0/344079186 shutdown_connections 2024-03-29T04:18:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.856+0000 7f1e51735640 1 --2- 172.21.15.22:0/344079186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e4c1418d0 0x7f1e4c15de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.856+0000 7f1e51735640 1 --2- 172.21.15.22:0/344079186 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1e2c067fc0 0x7f1e2c06a480 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.856+0000 7f1e51735640 1 --2- 172.21.15.22:0/344079186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4c0afa50 0x7f1e4c1442d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.856+0000 7f1e51735640 1 --2- 172.21.15.22:0/344079186 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4c067fd0 0x7f1e4c142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.856+0000 7f1e51735640 1 -- 172.21.15.22:0/344079186 >> 172.21.15.22:0/344079186 conn(0x7f1e4c0d0710 msgr2=0x7f1e4c0b1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.856+0000 7f1e51735640 1 -- 172.21.15.22:0/344079186 shutdown_connections 2024-03-29T04:18:31.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.856+0000 7f1e51735640 1 -- 172.21.15.22:0/344079186 wait complete. 2024-03-29T04:18:31.855 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:17:34.034142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:17:34.886659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:17:38.628632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:18:31.856 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:18:31.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.988+0000 7fc13e6f1640 1 Processor -- start 2024-03-29T04:18:31.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.988+0000 7fc13e6f1640 1 -- start start 2024-03-29T04:18:31.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.988+0000 7fc13e6f1640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc13806f4b0 0x7fc1381654d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.988+0000 7fc13e6f1640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc13815ace0 con 0x7fc13806c070 2024-03-29T04:18:31.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc13e6f1640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc13806c570 con 0x7fc13815a900 2024-03-29T04:18:31.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc13e6f1640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc13806c850 con 0x7fc13806f4b0 2024-03-29T04:18:31.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc1377fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fc13806c070 0x7fc13806eda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57054/0 (socket says 172.21.15.22:57054) 2024-03-29T04:18:31.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc1377fe640 1 -- 172.21.15.22:0/1767202577 learned_addr learned my addr 172.21.15.22:0/1767202577 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:31.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc13cc8e640 1 --2- 172.21.15.22:0/1767202577 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc13806f4b0 0x7fc1381654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc136ffd640 1 -- 172.21.15.22:0/1767202577 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2348029568 0 0) 0x7fc13815ace0 con 0x7fc13806c070 2024-03-29T04:18:31.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc136ffd640 1 -- 172.21.15.22:0/1767202577 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc120003530 con 0x7fc13806c070 2024-03-29T04:18:31.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc136ffd640 1 -- 172.21.15.22:0/1767202577 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1819859718 0 0) 0x7fc120003530 con 0x7fc13806c070 2024-03-29T04:18:31.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc136ffd640 1 -- 172.21.15.22:0/1767202577 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc13806f4b0 msgr2=0x7fc1381654d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:31.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc136ffd640 1 --2- 172.21.15.22:0/1767202577 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc13806f4b0 0x7fc1381654d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc136ffd640 1 -- 172.21.15.22:0/1767202577 >> v1:172.21.15.151:6789/0 conn(0x7fc13815a900 legacy=0x7fc1380afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:31.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc136ffd640 1 -- 172.21.15.22:0/1767202577 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13806cb30 con 0x7fc13806c070 2024-03-29T04:18:31.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc136ffd640 1 -- 172.21.15.22:0/1767202577 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7fc12c001680 con 0x7fc13806c070 2024-03-29T04:18:31.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc136ffd640 1 -- 172.21.15.22:0/1767202577 >> v1:172.21.15.22:6789/0 conn(0x7fc13806c070 legacy=0x7fc13806eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:31.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc136ffd640 1 --2- 172.21.15.22:0/1767202577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1200039f0 0x7fc120003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc136ffd640 1 --2- 172.21.15.22:0/1767202577 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc13806f4b0 0x7fc120004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc136ffd640 1 --2- 172.21.15.22:0/1767202577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc120004d50 0x7fc1200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc136ffd640 1 -- 172.21.15.22:0/1767202577 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc13806c850 con 0x7fc1200039f0 2024-03-29T04:18:31.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc136ffd640 1 -- 172.21.15.22:0/1767202577 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc12000ba80 con 0x7fc120004d50 2024-03-29T04:18:31.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc136ffd640 1 -- 172.21.15.22:0/1767202577 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc12000bc80 con 0x7fc13806f4b0 2024-03-29T04:18:31.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc137fff640 1 --2- 172.21.15.22:0/1767202577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1200039f0 0x7fc120003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc1377fe640 1 --2- 172.21.15.22:0/1767202577 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc13806f4b0 0x7fc120004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc13cc8e640 1 --2- 172.21.15.22:0/1767202577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc120004d50 0x7fc1200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc137fff640 1 -- 172.21.15.22:0/1767202577 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc13806f4b0 msgr2=0x7fc120004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:31.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc137fff640 1 --2- 172.21.15.22:0/1767202577 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc13806f4b0 0x7fc120004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc137fff640 1 -- 172.21.15.22:0/1767202577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc120004d50 msgr2=0x7fc1200051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:31.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc137fff640 1 --2- 172.21.15.22:0/1767202577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc120004d50 0x7fc1200051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc137fff640 1 -- 172.21.15.22:0/1767202577 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc120003530 con 0x7fc1200039f0 2024-03-29T04:18:31.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc13cc8e640 1 --2- 172.21.15.22:0/1767202577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc120004d50 0x7fc1200051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:31.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc137fff640 1 --2- 172.21.15.22:0/1767202577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1200039f0 0x7fc120003e10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc128001750 tx=0x7fc12801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc07eae307b0acf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:31.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc136ffd640 1 -- 172.21.15.22:0/1767202577 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1280013a0 con 0x7fc1200039f0 2024-03-29T04:18:31.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc136ffd640 1 -- 172.21.15.22:0/1767202577 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1280052f0 con 0x7fc1200039f0 2024-03-29T04:18:31.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc136ffd640 1 -- 172.21.15.22:0/1767202577 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128003840 con 0x7fc1200039f0 2024-03-29T04:18:31.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc13e6f1640 1 -- 172.21.15.22:0/1767202577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1200039f0 msgr2=0x7fc120003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:31.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc13e6f1640 1 --2- 172.21.15.22:0/1767202577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1200039f0 0x7fc120003e10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc128001750 tx=0x7fc12801c3e0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.992+0000 7fc13cc8e640 1 -- 172.21.15.22:0/1767202577 reap_dead start 2024-03-29T04:18:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc13e6f1640 1 -- 172.21.15.22:0/1767202577 shutdown_connections 2024-03-29T04:18:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc13e6f1640 1 -- 172.21.15.22:0/1767202577 >> 172.21.15.22:0/1767202577 conn(0x7fc1380d0c90 msgr2=0x7fc1380d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc13e6f1640 1 -- 172.21.15.22:0/1767202577 shutdown_connections 2024-03-29T04:18:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc13e6f1640 1 -- 172.21.15.22:0/1767202577 wait complete. 2024-03-29T04:18:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc13e6f1640 1 Processor -- start 2024-03-29T04:18:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc13e6f1640 1 -- start start 2024-03-29T04:18:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc13e6f1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1200039f0 0x7fc138146c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc13e6f1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc138149d20 0x7fc1381fa780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc13e6f1640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc138159080 0x7fc138159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc13e6f1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc138165860 con 0x7fc1200039f0 2024-03-29T04:18:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc13e6f1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc1381656e0 con 0x7fc138149d20 2024-03-29T04:18:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc13e6f1640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc1381659e0 con 0x7fc138159080 2024-03-29T04:18:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc137fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1200039f0 0x7fc138146c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc137fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1200039f0 0x7fc138146c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36964/0 (socket says 172.21.15.22:36964) 2024-03-29T04:18:31.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc137fff640 1 -- 172.21.15.22:0/4089770296 learned_addr learned my addr 172.21.15.22:0/4089770296 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:31.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc13cc8e640 1 --2- 172.21.15.22:0/4089770296 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc138159080 0x7fc138159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc1377fe640 1 --2- 172.21.15.22:0/4089770296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc138149d20 0x7fc1381fa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc137fff640 1 -- 172.21.15.22:0/4089770296 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc138159080 msgr2=0x7fc138159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:31.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc137fff640 1 --2- 172.21.15.22:0/4089770296 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc138159080 0x7fc138159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc137fff640 1 -- 172.21.15.22:0/4089770296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc138149d20 msgr2=0x7fc1381fa780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:31.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc137fff640 1 --2- 172.21.15.22:0/4089770296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc138149d20 0x7fc1381fa780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:31.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc137fff640 1 -- 172.21.15.22:0/4089770296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1381fb0f0 con 0x7fc1200039f0 2024-03-29T04:18:31.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc137fff640 1 --2- 172.21.15.22:0/4089770296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1200039f0 0x7fc138146c90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc12801c8c0 tx=0x7fc120005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:31.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc134ff9640 1 -- 172.21.15.22:0/4089770296 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128007a40 con 0x7fc1200039f0 2024-03-29T04:18:31.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc134ff9640 1 -- 172.21.15.22:0/4089770296 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1280bb070 con 0x7fc1200039f0 2024-03-29T04:18:31.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc13e6f1640 1 -- 172.21.15.22:0/4089770296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1381fb3d0 con 0x7fc1200039f0 2024-03-29T04:18:31.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc134ff9640 1 -- 172.21.15.22:0/4089770296 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128000ee0 con 0x7fc1200039f0 2024-03-29T04:18:31.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc13e6f1640 1 -- 172.21.15.22:0/4089770296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc138149800 con 0x7fc1200039f0 2024-03-29T04:18:31.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:31.996+0000 7fc13e6f1640 1 -- 172.21.15.22:0/4089770296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc138068900 con 0x7fc1200039f0 2024-03-29T04:18:31.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.000+0000 7fc134ff9640 1 -- 172.21.15.22:0/4089770296 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fc1280b4040 con 0x7fc1200039f0 2024-03-29T04:18:31.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.000+0000 7fc134ff9640 1 --2- 172.21.15.22:0/4089770296 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc124068160 0x7fc12406a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:31.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.000+0000 7fc134ff9640 1 -- 172.21.15.22:0/4089770296 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fc128120760 con 0x7fc1200039f0 2024-03-29T04:18:31.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.000+0000 7fc1377fe640 1 --2- 172.21.15.22:0/4089770296 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc124068160 0x7fc12406a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:31.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.000+0000 7fc1377fe640 1 --2- 172.21.15.22:0/4089770296 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc124068160 0x7fc12406a620 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc12c001ec0 tx=0x7fc12c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:31.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.004+0000 7fc134ff9640 1 -- 172.21.15.22:0/4089770296 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fc1280a4e20 con 0x7fc1200039f0 2024-03-29T04:18:32.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.164+0000 7fc13e6f1640 1 -- 172.21.15.22:0/4089770296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc1380d1240 con 0x7fc1200039f0 2024-03-29T04:18:32.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.164+0000 7fc134ff9640 1 -- 172.21.15.22:0/4089770296 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7fc1280160f0 con 0x7fc1200039f0 2024-03-29T04:18:32.145 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:18:32.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.164+0000 7fc13e6f1640 1 -- 172.21.15.22:0/4089770296 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc124068160 msgr2=0x7fc12406a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:32.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.164+0000 7fc13e6f1640 1 --2- 172.21.15.22:0/4089770296 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc124068160 0x7fc12406a620 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc12c001ec0 tx=0x7fc12c009040 comp rx=0 tx=0).stop 2024-03-29T04:18:32.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.164+0000 7fc13e6f1640 1 -- 172.21.15.22:0/4089770296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1200039f0 msgr2=0x7fc138146c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:32.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.164+0000 7fc13e6f1640 1 --2- 172.21.15.22:0/4089770296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1200039f0 0x7fc138146c90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc12801c8c0 tx=0x7fc120005840 comp rx=0 tx=0).stop 2024-03-29T04:18:32.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.164+0000 7fc13e6f1640 1 -- 172.21.15.22:0/4089770296 shutdown_connections 2024-03-29T04:18:32.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.164+0000 7fc13e6f1640 1 --2- 172.21.15.22:0/4089770296 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc138159080 0x7fc138159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.164+0000 7fc13e6f1640 1 --2- 172.21.15.22:0/4089770296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc138149d20 0x7fc1381fa780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.164+0000 7fc13e6f1640 1 --2- 172.21.15.22:0/4089770296 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc124068160 0x7fc12406a620 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.164+0000 7fc13e6f1640 1 --2- 172.21.15.22:0/4089770296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1200039f0 0x7fc138146c90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.164+0000 7fc13e6f1640 1 -- 172.21.15.22:0/4089770296 >> 172.21.15.22:0/4089770296 conn(0x7fc1380d0c90 msgr2=0x7fc1380ce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:32.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.164+0000 7fc13e6f1640 1 -- 172.21.15.22:0/4089770296 shutdown_connections 2024-03-29T04:18:32.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.164+0000 7fc13e6f1640 1 -- 172.21.15.22:0/4089770296 wait complete. 2024-03-29T04:18:32.163 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:18:32.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82e68f4640 1 Processor -- start 2024-03-29T04:18:32.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82e68f4640 1 -- start start 2024-03-29T04:18:32.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82e68f4640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f82e015c390 0x7f82e00d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82e68f4640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e006baf0 con 0x7f82e006f9e0 2024-03-29T04:18:32.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82e68f4640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e006bdd0 con 0x7f82e006b660 2024-03-29T04:18:32.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82e68f4640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f82e00cfcf0 con 0x7f82e015c390 2024-03-29T04:18:32.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82e4e91640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f82e006f9e0 0x7f82e006cb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57068/0 (socket says 172.21.15.22:57068) 2024-03-29T04:18:32.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82e4e91640 1 -- 172.21.15.22:0/2187452108 learned_addr learned my addr 172.21.15.22:0/2187452108 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:32.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82dffff640 1 --2- 172.21.15.22:0/2187452108 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f82e015c390 0x7f82e00d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82deffd640 1 -- 172.21.15.22:0/2187452108 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3079056738 0 0) 0x7f82e006baf0 con 0x7f82e006f9e0 2024-03-29T04:18:32.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82deffd640 1 -- 172.21.15.22:0/2187452108 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82c8003530 con 0x7f82e006f9e0 2024-03-29T04:18:32.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82deffd640 1 -- 172.21.15.22:0/2187452108 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1494232531 0 0) 0x7f82c8003530 con 0x7f82e006f9e0 2024-03-29T04:18:32.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82deffd640 1 -- 172.21.15.22:0/2187452108 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f82e015c390 msgr2=0x7f82e00d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:32.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82deffd640 1 --2- 172.21.15.22:0/2187452108 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f82e015c390 0x7f82e00d08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82deffd640 1 -- 172.21.15.22:0/2187452108 >> v1:172.21.15.151:6789/0 conn(0x7f82e006b660 legacy=0x7f82e015c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:32.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82deffd640 1 -- 172.21.15.22:0/2187452108 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e006c0b0 con 0x7f82e006f9e0 2024-03-29T04:18:32.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82deffd640 1 -- 172.21.15.22:0/2187452108 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f82d8001a00 con 0x7f82e006f9e0 2024-03-29T04:18:32.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82deffd640 1 -- 172.21.15.22:0/2187452108 >> v1:172.21.15.22:6789/0 conn(0x7f82e006f9e0 legacy=0x7f82e006cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:32.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82deffd640 1 --2- 172.21.15.22:0/2187452108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82c80039f0 0x7f82c8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82deffd640 1 --2- 172.21.15.22:0/2187452108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82c8004810 0x7f82c8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82deffd640 1 --2- 172.21.15.22:0/2187452108 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f82e015c390 0x7f82c80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82deffd640 1 -- 172.21.15.22:0/2187452108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f82e00cfcf0 con 0x7f82c80039f0 2024-03-29T04:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82deffd640 1 -- 172.21.15.22:0/2187452108 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f82c800ba80 con 0x7f82c8004810 2024-03-29T04:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82deffd640 1 -- 172.21.15.22:0/2187452108 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f82c800bc80 con 0x7f82e015c390 2024-03-29T04:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82dffff640 1 --2- 172.21.15.22:0/2187452108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82c80039f0 0x7f82c8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82e4e91640 1 --2- 172.21.15.22:0/2187452108 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f82e015c390 0x7f82c80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82df7fe640 1 --2- 172.21.15.22:0/2187452108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82c8004810 0x7f82c8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82dffff640 1 -- 172.21.15.22:0/2187452108 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f82e015c390 msgr2=0x7f82c80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82dffff640 1 --2- 172.21.15.22:0/2187452108 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f82e015c390 0x7f82c80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82dffff640 1 -- 172.21.15.22:0/2187452108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82c8004810 msgr2=0x7f82c8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82dffff640 1 --2- 172.21.15.22:0/2187452108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82c8004810 0x7f82c8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.300+0000 7f82dffff640 1 -- 172.21.15.22:0/2187452108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c8003530 con 0x7f82c80039f0 2024-03-29T04:18:32.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.304+0000 7f82dffff640 1 --2- 172.21.15.22:0/2187452108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82c80039f0 0x7f82c8003e10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009ab0 tx=0x7f82d001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33487cd6ead6d30d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:32.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.304+0000 7f82deffd640 1 -- 172.21.15.22:0/2187452108 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d00b2070 con 0x7f82c80039f0 2024-03-29T04:18:32.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.304+0000 7f82deffd640 1 -- 172.21.15.22:0/2187452108 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82d00a55d0 con 0x7f82c80039f0 2024-03-29T04:18:32.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.304+0000 7f82deffd640 1 -- 172.21.15.22:0/2187452108 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d00096b0 con 0x7f82c80039f0 2024-03-29T04:18:32.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.304+0000 7f82e68f4640 1 -- 172.21.15.22:0/2187452108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82c80039f0 msgr2=0x7f82c8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:32.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.304+0000 7f82e68f4640 1 --2- 172.21.15.22:0/2187452108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82c80039f0 0x7f82c8003e10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009ab0 tx=0x7f82d001ca00 comp rx=0 tx=0).stop 2024-03-29T04:18:32.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.304+0000 7f82e4e91640 1 -- 172.21.15.22:0/2187452108 reap_dead start 2024-03-29T04:18:32.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.304+0000 7f82e68f4640 1 -- 172.21.15.22:0/2187452108 shutdown_connections 2024-03-29T04:18:32.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.304+0000 7f82e68f4640 1 -- 172.21.15.22:0/2187452108 >> 172.21.15.22:0/2187452108 conn(0x7f82e00d1070 msgr2=0x7f82e00d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:32.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.304+0000 7f82e68f4640 1 -- 172.21.15.22:0/2187452108 shutdown_connections 2024-03-29T04:18:32.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.304+0000 7f82e68f4640 1 -- 172.21.15.22:0/2187452108 wait complete. 2024-03-29T04:18:32.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82e68f4640 1 Processor -- start 2024-03-29T04:18:32.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82e68f4640 1 -- start start 2024-03-29T04:18:32.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82e68f4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82c80039f0 0x7f82e01f7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82e68f4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82e0154530 0x7f82e0153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82e68f4640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f82e0156710 0x7f82e0156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82e68f4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f82e006d830 con 0x7f82e0154530 2024-03-29T04:18:32.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82e68f4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f82e006d6b0 con 0x7f82c80039f0 2024-03-29T04:18:32.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82e68f4640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f82e00d08e0 con 0x7f82e0156710 2024-03-29T04:18:32.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82df7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82e0154530 0x7f82e0153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82df7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82e0154530 0x7f82e0153400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36990/0 (socket says 172.21.15.22:36990) 2024-03-29T04:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82df7fe640 1 -- 172.21.15.22:0/2830195950 learned_addr learned my addr 172.21.15.22:0/2830195950 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82e4e91640 1 --2- 172.21.15.22:0/2830195950 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f82e0156710 0x7f82e0156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82dffff640 1 --2- 172.21.15.22:0/2830195950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82c80039f0 0x7f82e01f7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82df7fe640 1 -- 172.21.15.22:0/2830195950 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f82e0156710 msgr2=0x7f82e0156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82df7fe640 1 --2- 172.21.15.22:0/2830195950 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f82e0156710 0x7f82e0156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82df7fe640 1 -- 172.21.15.22:0/2830195950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82c80039f0 msgr2=0x7f82e01f7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82df7fe640 1 --2- 172.21.15.22:0/2830195950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82c80039f0 0x7f82e01f7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82df7fe640 1 -- 172.21.15.22:0/2830195950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e01f8010 con 0x7f82e0154530 2024-03-29T04:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82dffff640 1 --2- 172.21.15.22:0/2830195950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82c80039f0 0x7f82e01f7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82df7fe640 1 --2- 172.21.15.22:0/2830195950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82e0154530 0x7f82e0153400 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f82d400e4b0 tx=0x7f82d400e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82dcff9640 1 -- 172.21.15.22:0/2830195950 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d40052e0 con 0x7f82e0154530 2024-03-29T04:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82dcff9640 1 -- 172.21.15.22:0/2830195950 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82d4018070 con 0x7f82e0154530 2024-03-29T04:18:32.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82e68f4640 1 -- 172.21.15.22:0/2830195950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e01f82f0 con 0x7f82e0154530 2024-03-29T04:18:32.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82dcff9640 1 -- 172.21.15.22:0/2830195950 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d4005480 con 0x7f82e0154530 2024-03-29T04:18:32.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82e68f4640 1 -- 172.21.15.22:0/2830195950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e01f88b0 con 0x7f82e0154530 2024-03-29T04:18:32.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82e68f4640 1 -- 172.21.15.22:0/2830195950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82e0068900 con 0x7f82e0154530 2024-03-29T04:18:32.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82dcff9640 1 -- 172.21.15.22:0/2830195950 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f82d4003840 con 0x7f82e0154530 2024-03-29T04:18:32.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82dcff9640 1 --2- 172.21.15.22:0/2830195950 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f82cc067fc0 0x7f82cc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82dcff9640 1 -- 172.21.15.22:0/2830195950 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f82d40865b0 con 0x7f82e0154530 2024-03-29T04:18:32.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.308+0000 7f82dffff640 1 --2- 172.21.15.22:0/2830195950 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f82cc067fc0 0x7f82cc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.312+0000 7f82dffff640 1 --2- 172.21.15.22:0/2830195950 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f82cc067fc0 0x7f82cc06a480 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f82d00af000 tx=0x7f82d00046d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:32.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.316+0000 7f82dcff9640 1 -- 172.21.15.22:0/2830195950 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f82d4056960 con 0x7f82e0154530 2024-03-29T04:18:32.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.468+0000 7f82e68f4640 1 -- 172.21.15.22:0/2830195950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f82e00d1f80 con 0x7f82e0154530 2024-03-29T04:18:32.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.468+0000 7f82dcff9640 1 -- 172.21.15.22:0/2830195950 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f82d4056260 con 0x7f82e0154530 2024-03-29T04:18:32.450 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:32.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.472+0000 7f82e68f4640 1 -- 172.21.15.22:0/2830195950 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f82cc067fc0 msgr2=0x7f82cc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:32.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.472+0000 7f82e68f4640 1 --2- 172.21.15.22:0/2830195950 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f82cc067fc0 0x7f82cc06a480 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f82d00af000 tx=0x7f82d00046d0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.472+0000 7f82e68f4640 1 -- 172.21.15.22:0/2830195950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82e0154530 msgr2=0x7f82e0153400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:32.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.472+0000 7f82e68f4640 1 --2- 172.21.15.22:0/2830195950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82e0154530 0x7f82e0153400 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f82d400e4b0 tx=0x7f82d400e970 comp rx=0 tx=0).stop 2024-03-29T04:18:32.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.472+0000 7f82e68f4640 1 -- 172.21.15.22:0/2830195950 shutdown_connections 2024-03-29T04:18:32.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.472+0000 7f82e68f4640 1 --2- 172.21.15.22:0/2830195950 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f82e0156710 0x7f82e0156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.472+0000 7f82e68f4640 1 --2- 172.21.15.22:0/2830195950 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f82cc067fc0 0x7f82cc06a480 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.472+0000 7f82e68f4640 1 --2- 172.21.15.22:0/2830195950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82e0154530 0x7f82e0153400 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.472+0000 7f82e68f4640 1 --2- 172.21.15.22:0/2830195950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82c80039f0 0x7f82e01f7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.472+0000 7f82e68f4640 1 -- 172.21.15.22:0/2830195950 >> 172.21.15.22:0/2830195950 conn(0x7f82e00d1070 msgr2=0x7f82e00b5770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:32.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.472+0000 7f82e68f4640 1 -- 172.21.15.22:0/2830195950 shutdown_connections 2024-03-29T04:18:32.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.472+0000 7f82e68f4640 1 -- 172.21.15.22:0/2830195950 wait complete. 2024-03-29T04:18:32.468 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:18:32.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.608+0000 7fdf081fc640 1 Processor -- start 2024-03-29T04:18:32.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.608+0000 7fdf081fc640 1 -- start start 2024-03-29T04:18:32.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf081fc640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fdf00070270 0x7fdf00161f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf081fc640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdf000ce2c0 0x7fdf0015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf081fc640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf00162de0 con 0x7fdf000c9a00 2024-03-29T04:18:32.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf081fc640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fdf00162540 con 0x7fdf00070270 2024-03-29T04:18:32.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf081fc640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fdf00162820 con 0x7fdf000ce2c0 2024-03-29T04:18:32.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf05f98640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fdf000c9a00 0x7fdf000c9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57076/0 (socket says 172.21.15.22:57076) 2024-03-29T04:18:32.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf05f98640 1 -- 172.21.15.22:0/4202392589 learned_addr learned my addr 172.21.15.22:0/4202392589 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:32.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf06799640 1 --2- 172.21.15.22:0/4202392589 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdf000ce2c0 0x7fdf0015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf05797640 1 --2- 172.21.15.22:0/4202392589 >> v2:172.21.15.151:3300/0 conn(0x7fdf00070270 0x7fdf00161f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf04f96640 1 -- 172.21.15.22:0/4202392589 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3010424908 0 0) 0x7fdf00162de0 con 0x7fdf000c9a00 2024-03-29T04:18:32.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf04f96640 1 -- 172.21.15.22:0/4202392589 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdeec003530 con 0x7fdf000c9a00 2024-03-29T04:18:32.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf04f96640 1 -- 172.21.15.22:0/4202392589 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1267610469 0 0) 0x7fdeec003530 con 0x7fdf000c9a00 2024-03-29T04:18:32.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf04f96640 1 -- 172.21.15.22:0/4202392589 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdf000ce2c0 msgr2=0x7fdf0015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:32.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf04f96640 1 --2- 172.21.15.22:0/4202392589 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdf000ce2c0 0x7fdf0015efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf04f96640 1 -- 172.21.15.22:0/4202392589 >> v2:172.21.15.151:3300/0 conn(0x7fdf00070270 msgr2=0x7fdf00161f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:32.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf04f96640 1 --2- 172.21.15.22:0/4202392589 >> v2:172.21.15.151:3300/0 conn(0x7fdf00070270 0x7fdf00161f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf04f96640 1 -- 172.21.15.22:0/4202392589 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf00162b00 con 0x7fdf000c9a00 2024-03-29T04:18:32.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf04f96640 1 -- 172.21.15.22:0/4202392589 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7fdefc001680 con 0x7fdf000c9a00 2024-03-29T04:18:32.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf04f96640 1 -- 172.21.15.22:0/4202392589 >> v1:172.21.15.22:6789/0 conn(0x7fdf000c9a00 legacy=0x7fdf000c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:32.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf04f96640 1 --2- 172.21.15.22:0/4202392589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdeec0039c0 0x7fdeec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf04f96640 1 --2- 172.21.15.22:0/4202392589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdeec0043b0 0x7fdeec004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf04f96640 1 --2- 172.21.15.22:0/4202392589 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdf000ce2c0 0x7fdeec004f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf04f96640 1 -- 172.21.15.22:0/4202392589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdf00162540 con 0x7fdeec0043b0 2024-03-29T04:18:32.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf04f96640 1 -- 172.21.15.22:0/4202392589 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdf00162820 con 0x7fdeec0039c0 2024-03-29T04:18:32.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf04f96640 1 -- 172.21.15.22:0/4202392589 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fdeec00b9d0 con 0x7fdf000ce2c0 2024-03-29T04:18:32.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf05797640 1 --2- 172.21.15.22:0/4202392589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdeec0043b0 0x7fdeec004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf06799640 1 --2- 172.21.15.22:0/4202392589 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdf000ce2c0 0x7fdeec004f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf05797640 1 -- 172.21.15.22:0/4202392589 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdf000ce2c0 msgr2=0x7fdeec004f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:32.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf05797640 1 --2- 172.21.15.22:0/4202392589 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdf000ce2c0 0x7fdeec004f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf05797640 1 -- 172.21.15.22:0/4202392589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdeec0039c0 msgr2=0x7fdeec003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:32.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf05797640 1 --2- 172.21.15.22:0/4202392589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdeec0039c0 0x7fdeec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.612+0000 7fdf05797640 1 -- 172.21.15.22:0/4202392589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf00162b00 con 0x7fdeec0043b0 2024-03-29T04:18:32.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf05797640 1 --2- 172.21.15.22:0/4202392589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdeec0043b0 0x7fdeec004890 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009ae0 tx=0x7fdef001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c60d410a08204688 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:32.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf04f96640 1 -- 172.21.15.22:0/4202392589 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef00b2070 con 0x7fdeec0043b0 2024-03-29T04:18:32.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf04f96640 1 -- 172.21.15.22:0/4202392589 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdef00a56a0 con 0x7fdeec0043b0 2024-03-29T04:18:32.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf04f96640 1 -- 172.21.15.22:0/4202392589 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef00096b0 con 0x7fdeec0043b0 2024-03-29T04:18:32.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf081fc640 1 -- 172.21.15.22:0/4202392589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdeec0043b0 msgr2=0x7fdeec004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:32.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf081fc640 1 --2- 172.21.15.22:0/4202392589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdeec0043b0 0x7fdeec004890 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009ae0 tx=0x7fdef001cab0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf06799640 1 -- 172.21.15.22:0/4202392589 reap_dead start 2024-03-29T04:18:32.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf081fc640 1 -- 172.21.15.22:0/4202392589 shutdown_connections 2024-03-29T04:18:32.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf081fc640 1 -- 172.21.15.22:0/4202392589 >> 172.21.15.22:0/4202392589 conn(0x7fdf000cd170 msgr2=0x7fdf000cd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:32.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf081fc640 1 -- 172.21.15.22:0/4202392589 shutdown_connections 2024-03-29T04:18:32.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf081fc640 1 -- 172.21.15.22:0/4202392589 wait complete. 2024-03-29T04:18:32.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf081fc640 1 Processor -- start 2024-03-29T04:18:32.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf081fc640 1 -- start start 2024-03-29T04:18:32.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf081fc640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdeec0043b0 0x7fdf0014f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf081fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf00151500 0x7fdf0006eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf081fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf0006f060 0x7fdf0014f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf081fc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0015e7e0 con 0x7fdf0006f060 2024-03-29T04:18:32.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf081fc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0015e660 con 0x7fdf00151500 2024-03-29T04:18:32.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf081fc640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fdf000d0260 con 0x7fdeec0043b0 2024-03-29T04:18:32.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf06799640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf0006f060 0x7fdf0014f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf06799640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf0006f060 0x7fdf0014f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36998/0 (socket says 172.21.15.22:36998) 2024-03-29T04:18:32.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf06799640 1 -- 172.21.15.22:0/3078488280 learned_addr learned my addr 172.21.15.22:0/3078488280 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:32.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf05797640 1 --2- 172.21.15.22:0/3078488280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf00151500 0x7fdf0006eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.616+0000 7fdf05f98640 1 --2- 172.21.15.22:0/3078488280 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdeec0043b0 0x7fdf0014f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.620+0000 7fdf06799640 1 -- 172.21.15.22:0/3078488280 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdeec0043b0 msgr2=0x7fdf0014f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:32.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.620+0000 7fdf06799640 1 --2- 172.21.15.22:0/3078488280 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdeec0043b0 0x7fdf0014f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.620+0000 7fdf06799640 1 -- 172.21.15.22:0/3078488280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf00151500 msgr2=0x7fdf0006eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:32.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.620+0000 7fdf06799640 1 --2- 172.21.15.22:0/3078488280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf00151500 0x7fdf0006eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.620+0000 7fdf06799640 1 -- 172.21.15.22:0/3078488280 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf000b5140 con 0x7fdf0006f060 2024-03-29T04:18:32.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.620+0000 7fdf05f98640 1 --2- 172.21.15.22:0/3078488280 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdeec0043b0 0x7fdf0014f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:32.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.620+0000 7fdf06799640 1 --2- 172.21.15.22:0/3078488280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf0006f060 0x7fdf0014f9b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdf000ce6a0 tx=0x7fdef80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:32.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.620+0000 7fdef67fc640 1 -- 172.21.15.22:0/3078488280 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8011e50 con 0x7fdf0006f060 2024-03-29T04:18:32.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.620+0000 7fdef67fc640 1 -- 172.21.15.22:0/3078488280 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdef801b070 con 0x7fdf0006f060 2024-03-29T04:18:32.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.620+0000 7fdf081fc640 1 -- 172.21.15.22:0/3078488280 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf000b5420 con 0x7fdf0006f060 2024-03-29T04:18:32.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.620+0000 7fdef67fc640 1 -- 172.21.15.22:0/3078488280 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef80039a0 con 0x7fdf0006f060 2024-03-29T04:18:32.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.620+0000 7fdf081fc640 1 -- 172.21.15.22:0/3078488280 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf000b59e0 con 0x7fdf0006f060 2024-03-29T04:18:32.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.620+0000 7fdf081fc640 1 -- 172.21.15.22:0/3078488280 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf00068900 con 0x7fdf0006f060 2024-03-29T04:18:32.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.620+0000 7fdef67fc640 1 -- 172.21.15.22:0/3078488280 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fdef8003b60 con 0x7fdf0006f060 2024-03-29T04:18:32.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.620+0000 7fdef67fc640 1 --2- 172.21.15.22:0/3078488280 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdee4067fc0 0x7fdee406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.620+0000 7fdef67fc640 1 -- 172.21.15.22:0/3078488280 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fdef8086fc0 con 0x7fdf0006f060 2024-03-29T04:18:32.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.620+0000 7fdf05f98640 1 --2- 172.21.15.22:0/3078488280 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdee4067fc0 0x7fdee406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.620+0000 7fdf05f98640 1 --2- 172.21.15.22:0/3078488280 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdee4067fc0 0x7fdee406a480 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdefc001030 tx=0x7fdefc0016c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:32.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.624+0000 7fdef67fc640 1 -- 172.21.15.22:0/3078488280 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fdef8025020 con 0x7fdf0006f060 2024-03-29T04:18:32.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.780+0000 7fdf081fc640 1 -- 172.21.15.22:0/3078488280 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdf00067700 con 0x7fdf0006f060 2024-03-29T04:18:32.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.780+0000 7fdef67fc640 1 -- 172.21.15.22:0/3078488280 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fdef8057400 con 0x7fdf0006f060 2024-03-29T04:18:32.762 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-29T04:18:32.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.784+0000 7fdf081fc640 1 -- 172.21.15.22:0/3078488280 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdee4067fc0 msgr2=0x7fdee406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:32.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.784+0000 7fdf081fc640 1 --2- 172.21.15.22:0/3078488280 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdee4067fc0 0x7fdee406a480 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdefc001030 tx=0x7fdefc0016c0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.784+0000 7fdf081fc640 1 -- 172.21.15.22:0/3078488280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf0006f060 msgr2=0x7fdf0014f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:32.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.784+0000 7fdf081fc640 1 --2- 172.21.15.22:0/3078488280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf0006f060 0x7fdf0014f9b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdf000ce6a0 tx=0x7fdef80115a0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.784+0000 7fdf081fc640 1 -- 172.21.15.22:0/3078488280 shutdown_connections 2024-03-29T04:18:32.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.784+0000 7fdf081fc640 1 --2- 172.21.15.22:0/3078488280 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdee4067fc0 0x7fdee406a480 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.784+0000 7fdf081fc640 1 --2- 172.21.15.22:0/3078488280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf0006f060 0x7fdf0014f9b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.784+0000 7fdf081fc640 1 --2- 172.21.15.22:0/3078488280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf00151500 0x7fdf0006eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.784+0000 7fdf081fc640 1 --2- 172.21.15.22:0/3078488280 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fdeec0043b0 0x7fdf0014f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.784+0000 7fdf081fc640 1 -- 172.21.15.22:0/3078488280 >> 172.21.15.22:0/3078488280 conn(0x7fdf000cd170 msgr2=0x7fdf00153480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:32.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.784+0000 7fdf081fc640 1 -- 172.21.15.22:0/3078488280 shutdown_connections 2024-03-29T04:18:32.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.784+0000 7fdf081fc640 1 -- 172.21.15.22:0/3078488280 wait complete. 2024-03-29T04:18:32.780 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:18:32.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.928+0000 7f53de8b0640 1 Processor -- start 2024-03-29T04:18:32.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53de8b0640 1 -- start start 2024-03-29T04:18:32.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53de8b0640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f53d815ff40 0x7f53d806c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53de8b0640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53d806d710 con 0x7f53d806cbb0 2024-03-29T04:18:32.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53de8b0640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53d806d9f0 con 0x7f53d813f140 2024-03-29T04:18:32.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53de8b0640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f53d806e320 con 0x7f53d815ff40 2024-03-29T04:18:32.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53dce4d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f53d806cbb0 0x7f53d80ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57092/0 (socket says 172.21.15.22:57092) 2024-03-29T04:18:32.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53dce4d640 1 -- 172.21.15.22:0/3791568810 learned_addr learned my addr 172.21.15.22:0/3791568810 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:32.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d77fe640 1 --2- 172.21.15.22:0/3791568810 >> v2:172.21.15.151:3300/0 conn(0x7f53d815ff40 0x7f53d806c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d6ffd640 1 -- 172.21.15.22:0/3791568810 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 880456615 0 0) 0x7f53d806d710 con 0x7f53d806cbb0 2024-03-29T04:18:32.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d6ffd640 1 -- 172.21.15.22:0/3791568810 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53c4003530 con 0x7f53d806cbb0 2024-03-29T04:18:32.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d6ffd640 1 -- 172.21.15.22:0/3791568810 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1842221527 0 0) 0x7f53c4003530 con 0x7f53d806cbb0 2024-03-29T04:18:32.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d6ffd640 1 -- 172.21.15.22:0/3791568810 >> v2:172.21.15.151:3300/0 conn(0x7f53d815ff40 msgr2=0x7f53d806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:32.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d6ffd640 1 --2- 172.21.15.22:0/3791568810 >> v2:172.21.15.151:3300/0 conn(0x7f53d815ff40 0x7f53d806c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d6ffd640 1 -- 172.21.15.22:0/3791568810 >> v1:172.21.15.151:6789/0 conn(0x7f53d813f140 legacy=0x7f53d81606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:32.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d6ffd640 1 -- 172.21.15.22:0/3791568810 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d806dcd0 con 0x7f53d806cbb0 2024-03-29T04:18:32.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d6ffd640 1 -- 172.21.15.22:0/3791568810 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f53d0001a00 con 0x7f53d806cbb0 2024-03-29T04:18:32.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d6ffd640 1 -- 172.21.15.22:0/3791568810 >> v1:172.21.15.22:6789/0 conn(0x7f53d806cbb0 legacy=0x7f53d80ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:32.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d6ffd640 1 --2- 172.21.15.22:0/3791568810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53c40039c0 0x7f53c4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d6ffd640 1 --2- 172.21.15.22:0/3791568810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53c4004810 0x7f53c4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d6ffd640 1 --2- 172.21.15.22:0/3791568810 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53c40051d0 0x7f53c400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d6ffd640 1 -- 172.21.15.22:0/3791568810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f53d806e320 con 0x7f53c4004810 2024-03-29T04:18:32.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d6ffd640 1 -- 172.21.15.22:0/3791568810 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f53c400c070 con 0x7f53c40039c0 2024-03-29T04:18:32.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d6ffd640 1 -- 172.21.15.22:0/3791568810 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f53c400c270 con 0x7f53c40051d0 2024-03-29T04:18:32.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d77fe640 1 --2- 172.21.15.22:0/3791568810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53c4004810 0x7f53c4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d7fff640 1 --2- 172.21.15.22:0/3791568810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53c40039c0 0x7f53c4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53dce4d640 1 --2- 172.21.15.22:0/3791568810 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53c40051d0 0x7f53c400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d77fe640 1 -- 172.21.15.22:0/3791568810 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53c40051d0 msgr2=0x7f53c400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:32.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d77fe640 1 --2- 172.21.15.22:0/3791568810 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53c40051d0 0x7f53c400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d77fe640 1 -- 172.21.15.22:0/3791568810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53c40039c0 msgr2=0x7f53c4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:32.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d77fe640 1 --2- 172.21.15.22:0/3791568810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53c40039c0 0x7f53c4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d77fe640 1 -- 172.21.15.22:0/3791568810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c4003530 con 0x7f53c4004810 2024-03-29T04:18:32.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53d7fff640 1 --2- 172.21.15.22:0/3791568810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53c40039c0 0x7f53c4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:32.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.932+0000 7f53dce4d640 1 -- 172.21.15.22:0/3791568810 reap_dead start 2024-03-29T04:18:32.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53d77fe640 1 --2- 172.21.15.22:0/3791568810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53c4004810 0x7f53c4004c90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009ab0 tx=0x7f53c001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278011c8f41faaeb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:32.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53d6ffd640 1 -- 172.21.15.22:0/3791568810 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c00b2070 con 0x7f53c4004810 2024-03-29T04:18:32.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53d6ffd640 1 -- 172.21.15.22:0/3791568810 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53c00a5640 con 0x7f53c4004810 2024-03-29T04:18:32.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53d6ffd640 1 -- 172.21.15.22:0/3791568810 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c00096b0 con 0x7f53c4004810 2024-03-29T04:18:32.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53de8b0640 1 -- 172.21.15.22:0/3791568810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53c4004810 msgr2=0x7f53c4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:32.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53de8b0640 1 --2- 172.21.15.22:0/3791568810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53c4004810 0x7f53c4004c90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009ab0 tx=0x7f53c001ca50 comp rx=0 tx=0).stop 2024-03-29T04:18:32.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53de8b0640 1 -- 172.21.15.22:0/3791568810 shutdown_connections 2024-03-29T04:18:32.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53de8b0640 1 --2- 172.21.15.22:0/3791568810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53c4004810 0x7f53c4004c90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53de8b0640 1 -- 172.21.15.22:0/3791568810 >> 172.21.15.22:0/3791568810 conn(0x7f53d80d00a0 msgr2=0x7f53d80d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:32.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53de8b0640 1 -- 172.21.15.22:0/3791568810 shutdown_connections 2024-03-29T04:18:32.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53de8b0640 1 -- 172.21.15.22:0/3791568810 wait complete. 2024-03-29T04:18:32.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53de8b0640 1 Processor -- start 2024-03-29T04:18:32.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53de8b0640 1 -- start start 2024-03-29T04:18:32.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53de8b0640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53c4004810 0x7f53d813aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53de8b0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53d813b3f0 0x7f53d813c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53de8b0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53d813d6b0 0x7f53d813db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53de8b0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f53d813f6a0 con 0x7f53d813d6b0 2024-03-29T04:18:32.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53de8b0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f53d813f520 con 0x7f53d813b3f0 2024-03-29T04:18:32.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53de8b0640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f53d806c670 con 0x7f53c4004810 2024-03-29T04:18:32.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53dce4d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53d813d6b0 0x7f53d813db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53dce4d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53d813d6b0 0x7f53d813db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37020/0 (socket says 172.21.15.22:37020) 2024-03-29T04:18:32.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53dce4d640 1 -- 172.21.15.22:0/2502339243 learned_addr learned my addr 172.21.15.22:0/2502339243 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:32.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53d7fff640 1 --2- 172.21.15.22:0/2502339243 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53c4004810 0x7f53d813aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53d77fe640 1 --2- 172.21.15.22:0/2502339243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53d813b3f0 0x7f53d813c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53dce4d640 1 -- 172.21.15.22:0/2502339243 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53c4004810 msgr2=0x7f53d813aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:32.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53dce4d640 1 --2- 172.21.15.22:0/2502339243 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53c4004810 0x7f53d813aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53dce4d640 1 -- 172.21.15.22:0/2502339243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53d813b3f0 msgr2=0x7f53d813c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:32.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53dce4d640 1 --2- 172.21.15.22:0/2502339243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53d813b3f0 0x7f53d813c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:32.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53dce4d640 1 -- 172.21.15.22:0/2502339243 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d813e470 con 0x7f53d813d6b0 2024-03-29T04:18:32.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.936+0000 7f53d7fff640 1 --2- 172.21.15.22:0/2502339243 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53c4004810 0x7f53d813aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:32.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.940+0000 7f53dce4d640 1 --2- 172.21.15.22:0/2502339243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53d813d6b0 0x7f53d813db30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f53d806d230 tx=0x7f53d0010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:32.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.940+0000 7f53d4ff9640 1 -- 172.21.15.22:0/2502339243 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d001a070 con 0x7f53d813d6b0 2024-03-29T04:18:32.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.940+0000 7f53d4ff9640 1 -- 172.21.15.22:0/2502339243 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d0011040 con 0x7f53d813d6b0 2024-03-29T04:18:32.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.940+0000 7f53d4ff9640 1 -- 172.21.15.22:0/2502339243 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d000ec20 con 0x7f53d813d6b0 2024-03-29T04:18:32.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.940+0000 7f53de8b0640 1 -- 172.21.15.22:0/2502339243 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d813c9f0 con 0x7f53d813d6b0 2024-03-29T04:18:32.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.940+0000 7f53de8b0640 1 -- 172.21.15.22:0/2502339243 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d80ad070 con 0x7f53d813d6b0 2024-03-29T04:18:32.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.940+0000 7f53de8b0640 1 -- 172.21.15.22:0/2502339243 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d8068900 con 0x7f53d813d6b0 2024-03-29T04:18:32.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.940+0000 7f53d4ff9640 1 -- 172.21.15.22:0/2502339243 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f53d001d070 con 0x7f53d813d6b0 2024-03-29T04:18:32.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.940+0000 7f53d4ff9640 1 --2- 172.21.15.22:0/2502339243 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f53b8067fc0 0x7f53b806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:32.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.940+0000 7f53d4ff9640 1 -- 172.21.15.22:0/2502339243 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f53d0088fa0 con 0x7f53d813d6b0 2024-03-29T04:18:32.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.940+0000 7f53d7fff640 1 --2- 172.21.15.22:0/2502339243 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f53b8067fc0 0x7f53b806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:32.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.940+0000 7f53d7fff640 1 --2- 172.21.15.22:0/2502339243 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f53b8067fc0 0x7f53b806a480 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f53c8003600 tx=0x7f53c8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:32.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:32.944+0000 7f53d4ff9640 1 -- 172.21.15.22:0/2502339243 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f53d813e470 con 0x7f53d813d6b0 2024-03-29T04:18:33.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.108+0000 7f53de8b0640 1 -- 172.21.15.22:0/2502339243 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f53d815ee80 con 0x7f53d813d6b0 2024-03-29T04:18:33.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.108+0000 7f53d4ff9640 1 -- 172.21.15.22:0/2502339243 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f53d0059350 con 0x7f53d813d6b0 2024-03-29T04:18:33.090 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:33.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.112+0000 7f53de8b0640 1 -- 172.21.15.22:0/2502339243 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f53b8067fc0 msgr2=0x7f53b806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:33.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.112+0000 7f53de8b0640 1 --2- 172.21.15.22:0/2502339243 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f53b8067fc0 0x7f53b806a480 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f53c8003600 tx=0x7f53c8009040 comp rx=0 tx=0).stop 2024-03-29T04:18:33.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.112+0000 7f53de8b0640 1 -- 172.21.15.22:0/2502339243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53d813d6b0 msgr2=0x7f53d813db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:33.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.112+0000 7f53de8b0640 1 --2- 172.21.15.22:0/2502339243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53d813d6b0 0x7f53d813db30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f53d806d230 tx=0x7f53d0010cf0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.112+0000 7f53de8b0640 1 -- 172.21.15.22:0/2502339243 shutdown_connections 2024-03-29T04:18:33.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.112+0000 7f53de8b0640 1 --2- 172.21.15.22:0/2502339243 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f53b8067fc0 0x7f53b806a480 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.112+0000 7f53de8b0640 1 --2- 172.21.15.22:0/2502339243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53d813d6b0 0x7f53d813db30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.112+0000 7f53de8b0640 1 --2- 172.21.15.22:0/2502339243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53d813b3f0 0x7f53d813c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.112+0000 7f53de8b0640 1 --2- 172.21.15.22:0/2502339243 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f53c4004810 0x7f53d813aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.112+0000 7f53de8b0640 1 -- 172.21.15.22:0/2502339243 >> 172.21.15.22:0/2502339243 conn(0x7f53d80d00a0 msgr2=0x7f53d80ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:33.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.112+0000 7f53de8b0640 1 -- 172.21.15.22:0/2502339243 shutdown_connections 2024-03-29T04:18:33.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.112+0000 7f53de8b0640 1 -- 172.21.15.22:0/2502339243 wait complete. 2024-03-29T04:18:33.110 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:18:33.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.248+0000 7f0f3254b640 1 Processor -- start 2024-03-29T04:18:33.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.248+0000 7f0f3254b640 1 -- start start 2024-03-29T04:18:33.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.248+0000 7f0f3254b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0f2c0cc910 0x7f0f2c166ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:33.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.248+0000 7f0f3254b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f2c0ccef0 con 0x7f0f2c0b41f0 2024-03-29T04:18:33.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.248+0000 7f0f3254b640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f2c0b3710 con 0x7f0f2c06c670 2024-03-29T04:18:33.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.248+0000 7f0f3254b640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f0f2c0b34c0 con 0x7f0f2c0cc910 2024-03-29T04:18:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.248+0000 7f0f2b7fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f0f2c0b41f0 0x7f0f2c06ca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57106/0 (socket says 172.21.15.22:57106) 2024-03-29T04:18:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.248+0000 7f0f2b7fe640 1 -- 172.21.15.22:0/627274899 learned_addr learned my addr 172.21.15.22:0/627274899 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.248+0000 7f0f30ae8640 1 --2- 172.21.15.22:0/627274899 >> v2:172.21.15.22:3300/0 conn(0x7f0f2c0cc910 0x7f0f2c166ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.248+0000 7f0f2affd640 1 -- 172.21.15.22:0/627274899 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 930509718 0 0) 0x7f0f2c0ccef0 con 0x7f0f2c0b41f0 2024-03-29T04:18:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.248+0000 7f0f2affd640 1 -- 172.21.15.22:0/627274899 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f14003530 con 0x7f0f2c0b41f0 2024-03-29T04:18:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.248+0000 7f0f30ae8640 1 -- 172.21.15.22:0/627274899 >> v1:172.21.15.151:6789/0 conn(0x7f0f2c06c670 legacy=0x7f0f2c06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.248+0000 7f0f30ae8640 1 -- 172.21.15.22:0/627274899 >> v1:172.21.15.22:6789/0 conn(0x7f0f2c0b41f0 legacy=0x7f0f2c06ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.248+0000 7f0f30ae8640 1 -- 172.21.15.22:0/627274899 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2c0b39f0 con 0x7f0f2c0cc910 2024-03-29T04:18:33.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f30ae8640 1 --2- 172.21.15.22:0/627274899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f2c0cc910 0x7f0f2c166ee0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009d90 tx=0x7f0f2401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81aa08887d86a181 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:33.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f2affd640 1 -- 172.21.15.22:0/627274899 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f240b2070 con 0x7f0f2c0cc910 2024-03-29T04:18:33.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f2affd640 1 -- 172.21.15.22:0/627274899 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f240a5830 con 0x7f0f2c0cc910 2024-03-29T04:18:33.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f2affd640 1 -- 172.21.15.22:0/627274899 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f240a5c00 con 0x7f0f2c0cc910 2024-03-29T04:18:33.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f3254b640 1 -- 172.21.15.22:0/627274899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f2c0cc910 msgr2=0x7f0f2c166ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:33.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f3254b640 1 --2- 172.21.15.22:0/627274899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f2c0cc910 0x7f0f2c166ee0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009d90 tx=0x7f0f2401c970 comp rx=0 tx=0).stop 2024-03-29T04:18:33.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f3254b640 1 -- 172.21.15.22:0/627274899 shutdown_connections 2024-03-29T04:18:33.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f3254b640 1 --2- 172.21.15.22:0/627274899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f2c0cc910 0x7f0f2c166ee0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f3254b640 1 -- 172.21.15.22:0/627274899 >> 172.21.15.22:0/627274899 conn(0x7f0f2c0cfee0 msgr2=0x7f0f2c0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:33.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f3254b640 1 -- 172.21.15.22:0/627274899 shutdown_connections 2024-03-29T04:18:33.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f3254b640 1 -- 172.21.15.22:0/627274899 wait complete. 2024-03-29T04:18:33.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f3254b640 1 Processor -- start 2024-03-29T04:18:33.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f3254b640 1 -- start start 2024-03-29T04:18:33.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f3254b640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0f2c06c670 0x7f0f2c15cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:33.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f3254b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f2c0b41f0 0x7f0f2c139af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:33.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f3254b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f2c0cc910 0x7f0f2c13a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:33.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f3254b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c169fd0 con 0x7f0f2c0b41f0 2024-03-29T04:18:33.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f3254b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c169e50 con 0x7f0f2c0cc910 2024-03-29T04:18:33.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f3254b640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0f2c16a150 con 0x7f0f2c06c670 2024-03-29T04:18:33.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f2b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f2c0b41f0 0x7f0f2c139af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:33.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f2b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f2c0b41f0 0x7f0f2c139af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37040/0 (socket says 172.21.15.22:37040) 2024-03-29T04:18:33.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f2b7fe640 1 -- 172.21.15.22:0/1401321734 learned_addr learned my addr 172.21.15.22:0/1401321734 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:33.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f2bfff640 1 --2- 172.21.15.22:0/1401321734 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0f2c06c670 0x7f0f2c15cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:33.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f30ae8640 1 --2- 172.21.15.22:0/1401321734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f2c0cc910 0x7f0f2c13a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:33.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f2b7fe640 1 -- 172.21.15.22:0/1401321734 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0f2c06c670 msgr2=0x7f0f2c15cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:33.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f2b7fe640 1 --2- 172.21.15.22:0/1401321734 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0f2c06c670 0x7f0f2c15cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f2b7fe640 1 -- 172.21.15.22:0/1401321734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f2c0cc910 msgr2=0x7f0f2c13a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:33.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f2b7fe640 1 --2- 172.21.15.22:0/1401321734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f2c0cc910 0x7f0f2c13a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f2b7fe640 1 -- 172.21.15.22:0/1401321734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2c159fc0 con 0x7f0f2c0b41f0 2024-03-29T04:18:33.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f2bfff640 1 --2- 172.21.15.22:0/1401321734 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0f2c06c670 0x7f0f2c15cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:33.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.252+0000 7f0f2b7fe640 1 --2- 172.21.15.22:0/1401321734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f2c0b41f0 0x7f0f2c139af0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000c420 tx=0x7f0f2000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:33.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.256+0000 7f0f28ff9640 1 -- 172.21.15.22:0/1401321734 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20017070 con 0x7f0f2c0b41f0 2024-03-29T04:18:33.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.256+0000 7f0f28ff9640 1 -- 172.21.15.22:0/1401321734 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f20005cf0 con 0x7f0f2c0b41f0 2024-03-29T04:18:33.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.256+0000 7f0f28ff9640 1 -- 172.21.15.22:0/1401321734 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f200072d0 con 0x7f0f2c0b41f0 2024-03-29T04:18:33.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.256+0000 7f0f3254b640 1 -- 172.21.15.22:0/1401321734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2c154c00 con 0x7f0f2c0b41f0 2024-03-29T04:18:33.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.256+0000 7f0f3254b640 1 -- 172.21.15.22:0/1401321734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f2c15a520 con 0x7f0f2c0b41f0 2024-03-29T04:18:33.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.256+0000 7f0f3254b640 1 -- 172.21.15.22:0/1401321734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f2c068890 con 0x7f0f2c0b41f0 2024-03-29T04:18:33.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.256+0000 7f0f28ff9640 1 -- 172.21.15.22:0/1401321734 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f0f2001a070 con 0x7f0f2c0b41f0 2024-03-29T04:18:33.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.256+0000 7f0f28ff9640 1 --2- 172.21.15.22:0/1401321734 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0f18067ff0 0x7f0f1806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:33.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.256+0000 7f0f28ff9640 1 -- 172.21.15.22:0/1401321734 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f0f20085b90 con 0x7f0f2c0b41f0 2024-03-29T04:18:33.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.256+0000 7f0f2bfff640 1 --2- 172.21.15.22:0/1401321734 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0f18067ff0 0x7f0f1806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:33.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.256+0000 7f0f2bfff640 1 --2- 172.21.15.22:0/1401321734 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0f18067ff0 0x7f0f1806a4b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c06f6a0 tx=0x7f0f1c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.260+0000 7f0f28ff9640 1 -- 172.21.15.22:0/1401321734 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f0f2c159fc0 con 0x7f0f2c0b41f0 2024-03-29T04:18:33.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.436+0000 7f0f3254b640 1 -- 172.21.15.22:0/1401321734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0f2c06f310 con 0x7f0f2c0b41f0 2024-03-29T04:18:33.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.436+0000 7f0f28ff9640 1 -- 172.21.15.22:0/1401321734 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f0f20055980 con 0x7f0f2c0b41f0 2024-03-29T04:18:33.417 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-03-29T04:18:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.436+0000 7f0f3254b640 1 -- 172.21.15.22:0/1401321734 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0f18067ff0 msgr2=0x7f0f1806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.436+0000 7f0f3254b640 1 --2- 172.21.15.22:0/1401321734 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0f18067ff0 0x7f0f1806a4b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c06f6a0 tx=0x7f0f1c009040 comp rx=0 tx=0).stop 2024-03-29T04:18:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.436+0000 7f0f3254b640 1 -- 172.21.15.22:0/1401321734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f2c0b41f0 msgr2=0x7f0f2c139af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.436+0000 7f0f3254b640 1 --2- 172.21.15.22:0/1401321734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f2c0b41f0 0x7f0f2c139af0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000c420 tx=0x7f0f2000c8e0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.436+0000 7f0f3254b640 1 -- 172.21.15.22:0/1401321734 shutdown_connections 2024-03-29T04:18:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.436+0000 7f0f3254b640 1 --2- 172.21.15.22:0/1401321734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f2c0cc910 0x7f0f2c13a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.436+0000 7f0f3254b640 1 --2- 172.21.15.22:0/1401321734 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0f18067ff0 0x7f0f1806a4b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.436+0000 7f0f3254b640 1 --2- 172.21.15.22:0/1401321734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f2c0b41f0 0x7f0f2c139af0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.436+0000 7f0f3254b640 1 --2- 172.21.15.22:0/1401321734 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0f2c06c670 0x7f0f2c15cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.436+0000 7f0f3254b640 1 -- 172.21.15.22:0/1401321734 >> 172.21.15.22:0/1401321734 conn(0x7f0f2c0cfee0 msgr2=0x7f0f2c0cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:33.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.436+0000 7f0f3254b640 1 -- 172.21.15.22:0/1401321734 shutdown_connections 2024-03-29T04:18:33.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.436+0000 7f0f3254b640 1 -- 172.21.15.22:0/1401321734 wait complete. 2024-03-29T04:18:33.437 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:18:33.437 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:18:33.443 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:33.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.584+0000 7f7373141640 1 Processor -- start 2024-03-29T04:18:33.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.584+0000 7f7373141640 1 -- start start 2024-03-29T04:18:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.584+0000 7f7373141640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f736c0d0470 0x7f736c15fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.584+0000 7f7373141640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f736c0d0900 0x7f736c1603e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.584+0000 7f7373141640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f736c0ccba0 con 0x7f736c15c1c0 2024-03-29T04:18:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.584+0000 7f7373141640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f736c0ce3a0 con 0x7f736c0d0900 2024-03-29T04:18:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.584+0000 7f7373141640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f736c0cce80 con 0x7f736c0d0470 2024-03-29T04:18:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.584+0000 7f73716de640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f736c0d0900 0x7f736c1603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.584+0000 7f73716de640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f736c0d0900 0x7f736c1603e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37042/0 (socket says 172.21.15.22:37042) 2024-03-29T04:18:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.584+0000 7f73716de640 1 -- 172.21.15.22:0/2870184973 learned_addr learned my addr 172.21.15.22:0/2870184973 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.584+0000 7f736bfff640 1 --2- 172.21.15.22:0/2870184973 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f736c0d0470 0x7f736c15fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.584+0000 7f73716de640 1 -- 172.21.15.22:0/2870184973 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f736c0d0470 msgr2=0x7f736c15fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.584+0000 7f73716de640 1 --2- 172.21.15.22:0/2870184973 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f736c0d0470 0x7f736c15fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.584+0000 7f73716de640 1 -- 172.21.15.22:0/2870184973 >> v1:172.21.15.151:6789/0 conn(0x7f736c15c1c0 legacy=0x7f736c0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.584+0000 7f73716de640 1 -- 172.21.15.22:0/2870184973 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f736c0b26f0 con 0x7f736c0d0900 2024-03-29T04:18:33.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.584+0000 7f73716de640 1 --2- 172.21.15.22:0/2870184973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f736c0d0900 0x7f736c1603e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7364009e50 tx=0x7f736401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b07f7afa40e6f52 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:33.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f736b7fe640 1 -- 172.21.15.22:0/2870184973 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73640a5700 con 0x7f736c0d0900 2024-03-29T04:18:33.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f736b7fe640 1 -- 172.21.15.22:0/2870184973 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73640a58a0 con 0x7f736c0d0900 2024-03-29T04:18:33.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f736b7fe640 1 -- 172.21.15.22:0/2870184973 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73640a5c70 con 0x7f736c0d0900 2024-03-29T04:18:33.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7373141640 1 -- 172.21.15.22:0/2870184973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f736c0d0900 msgr2=0x7f736c1603e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:33.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7373141640 1 --2- 172.21.15.22:0/2870184973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f736c0d0900 0x7f736c1603e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7364009e50 tx=0x7f736401ca30 comp rx=0 tx=0).stop 2024-03-29T04:18:33.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7373141640 1 -- 172.21.15.22:0/2870184973 shutdown_connections 2024-03-29T04:18:33.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7373141640 1 --2- 172.21.15.22:0/2870184973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f736c0d0900 0x7f736c1603e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7373141640 1 --2- 172.21.15.22:0/2870184973 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f736c0d0470 0x7f736c15fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7373141640 1 -- 172.21.15.22:0/2870184973 >> 172.21.15.22:0/2870184973 conn(0x7f736c0b3d60 msgr2=0x7f736c0b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:33.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7373141640 1 -- 172.21.15.22:0/2870184973 shutdown_connections 2024-03-29T04:18:33.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7373141640 1 -- 172.21.15.22:0/2870184973 wait complete. 2024-03-29T04:18:33.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7373141640 1 Processor -- start 2024-03-29T04:18:33.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7373141640 1 -- start start 2024-03-29T04:18:33.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7373141640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f736c0d0470 0x7f736c1555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:33.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7373141640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f736c0d0900 0x7f736c156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:33.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7373141640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f736c15c1c0 0x7f736c1586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:33.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7373141640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f736c160aa0 con 0x7f736c0d0470 2024-03-29T04:18:33.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7373141640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f736c160920 con 0x7f736c0d0900 2024-03-29T04:18:33.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7373141640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f736c160c20 con 0x7f736c15c1c0 2024-03-29T04:18:33.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7370edd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f736c0d0470 0x7f736c1555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:33.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7370edd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f736c0d0470 0x7f736c1555b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37058/0 (socket says 172.21.15.22:37058) 2024-03-29T04:18:33.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7370edd640 1 -- 172.21.15.22:0/4185290825 learned_addr learned my addr 172.21.15.22:0/4185290825 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:33.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f736bfff640 1 --2- 172.21.15.22:0/4185290825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f736c0d0900 0x7f736c156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:33.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f73716de640 1 --2- 172.21.15.22:0/4185290825 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f736c15c1c0 0x7f736c1586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:33.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7370edd640 1 -- 172.21.15.22:0/4185290825 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f736c15c1c0 msgr2=0x7f736c1586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:33.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7370edd640 1 --2- 172.21.15.22:0/4185290825 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f736c15c1c0 0x7f736c1586d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7370edd640 1 -- 172.21.15.22:0/4185290825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f736c0d0900 msgr2=0x7f736c156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:33.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7370edd640 1 --2- 172.21.15.22:0/4185290825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f736c0d0900 0x7f736c156c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7370edd640 1 -- 172.21.15.22:0/4185290825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f736c154b10 con 0x7f736c0d0470 2024-03-29T04:18:33.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7370edd640 1 --2- 172.21.15.22:0/4185290825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f736c0d0470 0x7f736c1555b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f736000c3c0 tx=0x7f736000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:33.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f73697fa640 1 -- 172.21.15.22:0/4185290825 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360017070 con 0x7f736c0d0470 2024-03-29T04:18:33.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f73697fa640 1 -- 172.21.15.22:0/4185290825 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73600052d0 con 0x7f736c0d0470 2024-03-29T04:18:33.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7373141640 1 -- 172.21.15.22:0/4185290825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f736c156550 con 0x7f736c0d0470 2024-03-29T04:18:33.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f73697fa640 1 -- 172.21.15.22:0/4185290825 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360003840 con 0x7f736c0d0470 2024-03-29T04:18:33.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.588+0000 7f7373141640 1 -- 172.21.15.22:0/4185290825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f736c154db0 con 0x7f736c0d0470 2024-03-29T04:18:33.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.592+0000 7f7373141640 1 -- 172.21.15.22:0/4185290825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f736c068890 con 0x7f736c0d0470 2024-03-29T04:18:33.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.592+0000 7f73697fa640 1 -- 172.21.15.22:0/4185290825 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f736001a070 con 0x7f736c0d0470 2024-03-29T04:18:33.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.592+0000 7f73697fa640 1 --2- 172.21.15.22:0/4185290825 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7350067fc0 0x7f735006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:33.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.592+0000 7f73697fa640 1 -- 172.21.15.22:0/4185290825 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f7360085cd0 con 0x7f736c0d0470 2024-03-29T04:18:33.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.592+0000 7f736bfff640 1 --2- 172.21.15.22:0/4185290825 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7350067fc0 0x7f735006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:33.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.592+0000 7f736bfff640 1 --2- 172.21.15.22:0/4185290825 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7350067fc0 0x7f735006a480 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f736c14fa10 tx=0x7f7358008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:33.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.596+0000 7f73697fa640 1 -- 172.21.15.22:0/4185290825 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f7360005480 con 0x7f736c0d0470 2024-03-29T04:18:33.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.756+0000 7f7373141640 1 -- 172.21.15.22:0/4185290825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f736c153520 con 0x7f736c0d0470 2024-03-29T04:18:33.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.760+0000 7f73697fa640 1 -- 172.21.15.22:0/4185290825 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18847 (secure 0 0 0) 0x7f736005ae20 con 0x7f736c0d0470 2024-03-29T04:18:33.741 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:33.741 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:18:16.893439+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:33.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.760+0000 7f7373141640 1 -- 172.21.15.22:0/4185290825 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7350067fc0 msgr2=0x7f735006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:33.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.760+0000 7f7373141640 1 --2- 172.21.15.22:0/4185290825 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7350067fc0 0x7f735006a480 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f736c14fa10 tx=0x7f7358008040 comp rx=0 tx=0).stop 2024-03-29T04:18:33.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.760+0000 7f7373141640 1 -- 172.21.15.22:0/4185290825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f736c0d0470 msgr2=0x7f736c1555b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:33.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.760+0000 7f7373141640 1 --2- 172.21.15.22:0/4185290825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f736c0d0470 0x7f736c1555b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f736000c3c0 tx=0x7f736000c880 comp rx=0 tx=0).stop 2024-03-29T04:18:33.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.760+0000 7f7373141640 1 -- 172.21.15.22:0/4185290825 shutdown_connections 2024-03-29T04:18:33.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.760+0000 7f7373141640 1 --2- 172.21.15.22:0/4185290825 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f736c15c1c0 0x7f736c1586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.760+0000 7f7373141640 1 --2- 172.21.15.22:0/4185290825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f736c0d0900 0x7f736c156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.760+0000 7f7373141640 1 --2- 172.21.15.22:0/4185290825 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7350067fc0 0x7f735006a480 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.760+0000 7f7373141640 1 --2- 172.21.15.22:0/4185290825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f736c0d0470 0x7f736c1555b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.760+0000 7f7373141640 1 -- 172.21.15.22:0/4185290825 >> 172.21.15.22:0/4185290825 conn(0x7f736c0b3d60 msgr2=0x7f736c06e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:33.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.760+0000 7f7373141640 1 -- 172.21.15.22:0/4185290825 shutdown_connections 2024-03-29T04:18:33.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.760+0000 7f7373141640 1 -- 172.21.15.22:0/4185290825 wait complete. 2024-03-29T04:18:33.758 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:17:34.034142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:17:34.886659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:17:38.628632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:18:33.758 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:18:33.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.896+0000 7f51e40c1640 1 Processor -- start 2024-03-29T04:18:33.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.896+0000 7f51e40c1640 1 -- start start 2024-03-29T04:18:33.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.896+0000 7f51e40c1640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f51dc15e4c0 0x7f51dc0b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:33.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.896+0000 7f51e40c1640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51dc06c070 0x7f51dc0cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:33.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.896+0000 7f51e40c1640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51dc06c850 con 0x7f51dc0cf4f0 2024-03-29T04:18:33.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.896+0000 7f51e40c1640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f51dc15e8a0 con 0x7f51dc15e4c0 2024-03-29T04:18:33.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.896+0000 7f51e40c1640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f51dc06cb30 con 0x7f51dc06c070 2024-03-29T04:18:33.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.896+0000 7f51e1e5d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f51dc15e4c0 0x7f51dc0b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:33.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.896+0000 7f51e1e5d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f51dc15e4c0 0x7f51dc0b9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37066/0 (socket says 172.21.15.22:37066) 2024-03-29T04:18:33.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.896+0000 7f51e1e5d640 1 -- 172.21.15.22:0/942872996 learned_addr learned my addr 172.21.15.22:0/942872996 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:33.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.896+0000 7f51e165c640 1 --2- 172.21.15.22:0/942872996 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51dc06c070 0x7f51dc0cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:33.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.896+0000 7f51e1e5d640 1 -- 172.21.15.22:0/942872996 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51dc06c070 msgr2=0x7f51dc0cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:33.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.896+0000 7f51e1e5d640 1 --2- 172.21.15.22:0/942872996 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51dc06c070 0x7f51dc0cefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.896+0000 7f51e1e5d640 1 -- 172.21.15.22:0/942872996 >> v1:172.21.15.151:6789/0 conn(0x7f51dc0cf4f0 legacy=0x7f51dc161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:33.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.896+0000 7f51e1e5d640 1 -- 172.21.15.22:0/942872996 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51dc06ce10 con 0x7f51dc15e4c0 2024-03-29T04:18:33.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.896+0000 7f51e165c640 1 --2- 172.21.15.22:0/942872996 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51dc06c070 0x7f51dc0cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:33.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e1e5d640 1 --2- 172.21.15.22:0/942872996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51dc15e4c0 0x7f51dc0b9e30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009870 tx=0x7f51d801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80a0c81b751a215 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:33.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e0e5b640 1 -- 172.21.15.22:0/942872996 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d80b2070 con 0x7f51dc15e4c0 2024-03-29T04:18:33.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e0e5b640 1 -- 172.21.15.22:0/942872996 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51d80a9070 con 0x7f51dc15e4c0 2024-03-29T04:18:33.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e0e5b640 1 -- 172.21.15.22:0/942872996 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d80ad880 con 0x7f51dc15e4c0 2024-03-29T04:18:33.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e40c1640 1 -- 172.21.15.22:0/942872996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51dc15e4c0 msgr2=0x7f51dc0b9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:33.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e40c1640 1 --2- 172.21.15.22:0/942872996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51dc15e4c0 0x7f51dc0b9e30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009870 tx=0x7f51d801c560 comp rx=0 tx=0).stop 2024-03-29T04:18:33.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e40c1640 1 -- 172.21.15.22:0/942872996 shutdown_connections 2024-03-29T04:18:33.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e40c1640 1 --2- 172.21.15.22:0/942872996 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51dc06c070 0x7f51dc0cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e40c1640 1 --2- 172.21.15.22:0/942872996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51dc15e4c0 0x7f51dc0b9e30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e40c1640 1 -- 172.21.15.22:0/942872996 >> 172.21.15.22:0/942872996 conn(0x7f51dc0b6410 msgr2=0x7f51dc0d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:33.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e40c1640 1 -- 172.21.15.22:0/942872996 shutdown_connections 2024-03-29T04:18:33.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e40c1640 1 -- 172.21.15.22:0/942872996 wait complete. 2024-03-29T04:18:33.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e40c1640 1 Processor -- start 2024-03-29T04:18:33.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e40c1640 1 -- start start 2024-03-29T04:18:33.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e40c1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51dc06c070 0x7f51dc0b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:33.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e40c1640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51dc0cf4f0 0x7f51dc156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:33.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e40c1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51dc15e4c0 0x7f51dc14d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:33.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e40c1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc16b250 con 0x7f51dc06c070 2024-03-29T04:18:33.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e40c1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc0cf900 con 0x7f51dc15e4c0 2024-03-29T04:18:33.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e40c1640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f51dc16b3d0 con 0x7f51dc0cf4f0 2024-03-29T04:18:33.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e1e5d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51dc06c070 0x7f51dc0b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:33.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e1e5d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51dc06c070 0x7f51dc0b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37082/0 (socket says 172.21.15.22:37082) 2024-03-29T04:18:33.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e1e5d640 1 -- 172.21.15.22:0/2429079704 learned_addr learned my addr 172.21.15.22:0/2429079704 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:33.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e165c640 1 --2- 172.21.15.22:0/2429079704 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51dc0cf4f0 0x7f51dc156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:33.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e265e640 1 --2- 172.21.15.22:0/2429079704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51dc15e4c0 0x7f51dc14d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:33.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e1e5d640 1 -- 172.21.15.22:0/2429079704 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51dc0cf4f0 msgr2=0x7f51dc156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:33.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e1e5d640 1 --2- 172.21.15.22:0/2429079704 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51dc0cf4f0 0x7f51dc156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e1e5d640 1 -- 172.21.15.22:0/2429079704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51dc15e4c0 msgr2=0x7f51dc14d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:33.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e1e5d640 1 --2- 172.21.15.22:0/2429079704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51dc15e4c0 0x7f51dc14d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:33.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e1e5d640 1 -- 172.21.15.22:0/2429079704 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51dc0d4220 con 0x7f51dc06c070 2024-03-29T04:18:33.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e265e640 1 --2- 172.21.15.22:0/2429079704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51dc15e4c0 0x7f51dc14d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:33.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e1e5d640 1 --2- 172.21.15.22:0/2429079704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51dc06c070 0x7f51dc0b48d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f51d80b0040 tx=0x7f51d801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:33.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51d27fc640 1 -- 172.21.15.22:0/2429079704 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d80b2040 con 0x7f51dc06c070 2024-03-29T04:18:33.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51d27fc640 1 -- 172.21.15.22:0/2429079704 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51d80be070 con 0x7f51dc06c070 2024-03-29T04:18:33.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e40c1640 1 -- 172.21.15.22:0/2429079704 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51dc15b4d0 con 0x7f51dc06c070 2024-03-29T04:18:33.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51d27fc640 1 -- 172.21.15.22:0/2429079704 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d80b8880 con 0x7f51dc06c070 2024-03-29T04:18:33.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.900+0000 7f51e40c1640 1 -- 172.21.15.22:0/2429079704 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51dc0b4eb0 con 0x7f51dc06c070 2024-03-29T04:18:33.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.904+0000 7f51e40c1640 1 -- 172.21.15.22:0/2429079704 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51dc068900 con 0x7f51dc06c070 2024-03-29T04:18:33.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.904+0000 7f51d27fc640 1 -- 172.21.15.22:0/2429079704 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f51d80ab070 con 0x7f51dc06c070 2024-03-29T04:18:33.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.904+0000 7f51d27fc640 1 --2- 172.21.15.22:0/2429079704 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f51c0067fc0 0x7f51c006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:33.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.904+0000 7f51d27fc640 1 -- 172.21.15.22:0/2429079704 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f51d8120c10 con 0x7f51dc06c070 2024-03-29T04:18:33.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.904+0000 7f51e165c640 1 --2- 172.21.15.22:0/2429079704 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f51c0067fc0 0x7f51c006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:33.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.904+0000 7f51e165c640 1 --2- 172.21.15.22:0/2429079704 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f51c0067fc0 0x7f51c006a480 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f51cc006fd0 tx=0x7f51cc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:33.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:33.908+0000 7f51d27fc640 1 -- 172.21.15.22:0/2429079704 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f51d8016150 con 0x7f51dc06c070 2024-03-29T04:18:34.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.076+0000 7f51e40c1640 1 -- 172.21.15.22:0/2429079704 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f51dc0b74e0 con 0x7f51dc06c070 2024-03-29T04:18:34.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.076+0000 7f51d27fc640 1 -- 172.21.15.22:0/2429079704 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f51d81240a0 con 0x7f51dc06c070 2024-03-29T04:18:34.058 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:18:34.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.080+0000 7f51e40c1640 1 -- 172.21.15.22:0/2429079704 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f51c0067fc0 msgr2=0x7f51c006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:34.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.080+0000 7f51e40c1640 1 --2- 172.21.15.22:0/2429079704 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f51c0067fc0 0x7f51c006a480 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f51cc006fd0 tx=0x7f51cc008040 comp rx=0 tx=0).stop 2024-03-29T04:18:34.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.080+0000 7f51e40c1640 1 -- 172.21.15.22:0/2429079704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51dc06c070 msgr2=0x7f51dc0b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:34.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.080+0000 7f51e40c1640 1 --2- 172.21.15.22:0/2429079704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51dc06c070 0x7f51dc0b48d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f51d80b0040 tx=0x7f51d801ada0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.080+0000 7f51e40c1640 1 -- 172.21.15.22:0/2429079704 shutdown_connections 2024-03-29T04:18:34.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.080+0000 7f51e40c1640 1 --2- 172.21.15.22:0/2429079704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51dc15e4c0 0x7f51dc14d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.080+0000 7f51e40c1640 1 --2- 172.21.15.22:0/2429079704 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f51dc0cf4f0 0x7f51dc156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.080+0000 7f51e40c1640 1 --2- 172.21.15.22:0/2429079704 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f51c0067fc0 0x7f51c006a480 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.080+0000 7f51e40c1640 1 --2- 172.21.15.22:0/2429079704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51dc06c070 0x7f51dc0b48d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.080+0000 7f51e40c1640 1 -- 172.21.15.22:0/2429079704 >> 172.21.15.22:0/2429079704 conn(0x7f51dc0b6410 msgr2=0x7f51dc0b9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:34.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.080+0000 7f51e40c1640 1 -- 172.21.15.22:0/2429079704 shutdown_connections 2024-03-29T04:18:34.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.080+0000 7f51e40c1640 1 -- 172.21.15.22:0/2429079704 wait complete. 2024-03-29T04:18:34.077 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:18:34.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.224+0000 7f5cc7b0c640 1 Processor -- start 2024-03-29T04:18:34.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.224+0000 7f5cc7b0c640 1 -- start start 2024-03-29T04:18:34.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.224+0000 7f5cc7b0c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5cc00b4ff0 0x7f5cc0070640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:34.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.224+0000 7f5cc7b0c640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cc00ce4d0 con 0x7f5cc006f090 2024-03-29T04:18:34.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.224+0000 7f5cc7b0c640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cc00cc7f0 con 0x7f5cc015f880 2024-03-29T04:18:34.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.224+0000 7f5cc7b0c640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f5cc00b5400 con 0x7f5cc00b4ff0 2024-03-29T04:18:34.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.224+0000 7f5cc50a7640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f5cc006f090 0x7f5cc00ce9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57114/0 (socket says 172.21.15.22:57114) 2024-03-29T04:18:34.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.224+0000 7f5cc50a7640 1 -- 172.21.15.22:0/3251609771 learned_addr learned my addr 172.21.15.22:0/3251609771 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:34.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.224+0000 7f5cc60a9640 1 --2- 172.21.15.22:0/3251609771 >> v2:172.21.15.22:3300/0 conn(0x7f5cc00b4ff0 0x7f5cc0070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:34.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.224+0000 7f5cc60a9640 1 -- 172.21.15.22:0/3251609771 >> v1:172.21.15.151:6789/0 conn(0x7f5cc015f880 legacy=0x7f5cc006e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:34.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.224+0000 7f5cc60a9640 1 -- 172.21.15.22:0/3251609771 >> v1:172.21.15.22:6789/0 conn(0x7f5cc006f090 legacy=0x7f5cc00ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:34.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.224+0000 7f5cc60a9640 1 -- 172.21.15.22:0/3251609771 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc00ccad0 con 0x7f5cc00b4ff0 2024-03-29T04:18:34.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.228+0000 7f5cc60a9640 1 --2- 172.21.15.22:0/3251609771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cc00b4ff0 0x7f5cc0070640 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009d90 tx=0x7f5cb801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde5970680a95f11 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:34.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.228+0000 7f5cc48a6640 1 -- 172.21.15.22:0/3251609771 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb80b2070 con 0x7f5cc00b4ff0 2024-03-29T04:18:34.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.228+0000 7f5cc48a6640 1 -- 172.21.15.22:0/3251609771 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cb80a58b0 con 0x7f5cc00b4ff0 2024-03-29T04:18:34.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.228+0000 7f5cc48a6640 1 -- 172.21.15.22:0/3251609771 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb80a5c80 con 0x7f5cc00b4ff0 2024-03-29T04:18:34.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.228+0000 7f5cc7b0c640 1 -- 172.21.15.22:0/3251609771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cc00b4ff0 msgr2=0x7f5cc0070640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:34.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.228+0000 7f5cc7b0c640 1 --2- 172.21.15.22:0/3251609771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cc00b4ff0 0x7f5cc0070640 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009d90 tx=0x7f5cb801c8e0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.228+0000 7f5cc7b0c640 1 -- 172.21.15.22:0/3251609771 shutdown_connections 2024-03-29T04:18:34.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.228+0000 7f5cc7b0c640 1 --2- 172.21.15.22:0/3251609771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cc00b4ff0 0x7f5cc0070640 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.228+0000 7f5cc7b0c640 1 -- 172.21.15.22:0/3251609771 >> 172.21.15.22:0/3251609771 conn(0x7f5cc00b3cf0 msgr2=0x7f5cc00b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:34.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.228+0000 7f5cc7b0c640 1 -- 172.21.15.22:0/3251609771 shutdown_connections 2024-03-29T04:18:34.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.228+0000 7f5cc7b0c640 1 -- 172.21.15.22:0/3251609771 wait complete. 2024-03-29T04:18:34.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.228+0000 7f5cc7b0c640 1 Processor -- start 2024-03-29T04:18:34.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.228+0000 7f5cc7b0c640 1 -- start start 2024-03-29T04:18:34.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cc7b0c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cc006f090 0x7f5cc01591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:34.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cc7b0c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cc00b4ff0 0x7f5cc0159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:34.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cc7b0c640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5cc015f880 0x7f5cc0157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:34.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cc7b0c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc00cf650 con 0x7f5cc00b4ff0 2024-03-29T04:18:34.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cc7b0c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc00cf4d0 con 0x7f5cc006f090 2024-03-29T04:18:34.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cc7b0c640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5cc0160850 con 0x7f5cc015f880 2024-03-29T04:18:34.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cc50a7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cc00b4ff0 0x7f5cc0159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:34.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cc50a7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cc00b4ff0 0x7f5cc0159720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37090/0 (socket says 172.21.15.22:37090) 2024-03-29T04:18:34.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cc50a7640 1 -- 172.21.15.22:0/2389977487 learned_addr learned my addr 172.21.15.22:0/2389977487 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:34.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cc60a9640 1 --2- 172.21.15.22:0/2389977487 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5cc015f880 0x7f5cc0157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:34.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cc50a7640 1 -- 172.21.15.22:0/2389977487 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5cc015f880 msgr2=0x7f5cc0157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:34.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cc50a7640 1 --2- 172.21.15.22:0/2389977487 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5cc015f880 0x7f5cc0157650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cc58a8640 1 --2- 172.21.15.22:0/2389977487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cc006f090 0x7f5cc01591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:34.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cc50a7640 1 -- 172.21.15.22:0/2389977487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cc006f090 msgr2=0x7f5cc01591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:34.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cc50a7640 1 --2- 172.21.15.22:0/2389977487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cc006f090 0x7f5cc01591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cc50a7640 1 -- 172.21.15.22:0/2389977487 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc00b3270 con 0x7f5cc00b4ff0 2024-03-29T04:18:34.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cc58a8640 1 --2- 172.21.15.22:0/2389977487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cc006f090 0x7f5cc01591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:34.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cc50a7640 1 --2- 172.21.15.22:0/2389977487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cc00b4ff0 0x7f5cc0159720 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0007fd0 tx=0x7f5cb000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:34.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cb67fc640 1 -- 172.21.15.22:0/2389977487 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0017070 con 0x7f5cc00b4ff0 2024-03-29T04:18:34.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cb67fc640 1 -- 172.21.15.22:0/2389977487 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cb0005d10 con 0x7f5cc00b4ff0 2024-03-29T04:18:34.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cc7b0c640 1 -- 172.21.15.22:0/2389977487 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc00cb440 con 0x7f5cc00b4ff0 2024-03-29T04:18:34.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cb67fc640 1 -- 172.21.15.22:0/2389977487 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0007310 con 0x7f5cc00b4ff0 2024-03-29T04:18:34.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cc7b0c640 1 -- 172.21.15.22:0/2389977487 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc00cba00 con 0x7f5cc00b4ff0 2024-03-29T04:18:34.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cc7b0c640 1 -- 172.21.15.22:0/2389977487 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc0068900 con 0x7f5cc00b4ff0 2024-03-29T04:18:34.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cb67fc640 1 -- 172.21.15.22:0/2389977487 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f5cb001a070 con 0x7f5cc00b4ff0 2024-03-29T04:18:34.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.232+0000 7f5cb67fc640 1 --2- 172.21.15.22:0/2389977487 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5ca4067fc0 0x7f5ca406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:34.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.236+0000 7f5cb67fc640 1 -- 172.21.15.22:0/2389977487 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f5cb0085ff0 con 0x7f5cc00b4ff0 2024-03-29T04:18:34.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.236+0000 7f5cc58a8640 1 --2- 172.21.15.22:0/2389977487 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5ca4067fc0 0x7f5ca406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:34.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.236+0000 7f5cc58a8640 1 --2- 172.21.15.22:0/2389977487 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5ca4067fc0 0x7f5ca406a480 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5cc00cd5f0 tx=0x7f5cbc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:34.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.236+0000 7f5cb67fc640 1 -- 172.21.15.22:0/2389977487 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f5cb0021020 con 0x7f5cc00b4ff0 2024-03-29T04:18:34.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.400+0000 7f5cc7b0c640 1 -- 172.21.15.22:0/2389977487 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5cc00b7ee0 con 0x7f5cc00b4ff0 2024-03-29T04:18:34.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.400+0000 7f5cb67fc640 1 -- 172.21.15.22:0/2389977487 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f5cb0056510 con 0x7f5cc00b4ff0 2024-03-29T04:18:34.380 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:34.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.400+0000 7f5cc7b0c640 1 -- 172.21.15.22:0/2389977487 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5ca4067fc0 msgr2=0x7f5ca406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:34.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.400+0000 7f5cc7b0c640 1 --2- 172.21.15.22:0/2389977487 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5ca4067fc0 0x7f5ca406a480 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5cc00cd5f0 tx=0x7f5cbc009040 comp rx=0 tx=0).stop 2024-03-29T04:18:34.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.400+0000 7f5cc7b0c640 1 -- 172.21.15.22:0/2389977487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cc00b4ff0 msgr2=0x7f5cc0159720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:34.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.400+0000 7f5cc7b0c640 1 --2- 172.21.15.22:0/2389977487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cc00b4ff0 0x7f5cc0159720 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0007fd0 tx=0x7f5cb000c890 comp rx=0 tx=0).stop 2024-03-29T04:18:34.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.400+0000 7f5cc7b0c640 1 -- 172.21.15.22:0/2389977487 shutdown_connections 2024-03-29T04:18:34.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.400+0000 7f5cc7b0c640 1 --2- 172.21.15.22:0/2389977487 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5cc015f880 0x7f5cc0157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.400+0000 7f5cc7b0c640 1 --2- 172.21.15.22:0/2389977487 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5ca4067fc0 0x7f5ca406a480 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.400+0000 7f5cc7b0c640 1 --2- 172.21.15.22:0/2389977487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cc00b4ff0 0x7f5cc0159720 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.400+0000 7f5cc7b0c640 1 --2- 172.21.15.22:0/2389977487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cc006f090 0x7f5cc01591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.400+0000 7f5cc7b0c640 1 -- 172.21.15.22:0/2389977487 >> 172.21.15.22:0/2389977487 conn(0x7f5cc00b3cf0 msgr2=0x7f5cc00b5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:34.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.400+0000 7f5cc7b0c640 1 -- 172.21.15.22:0/2389977487 shutdown_connections 2024-03-29T04:18:34.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.400+0000 7f5cc7b0c640 1 -- 172.21.15.22:0/2389977487 wait complete. 2024-03-29T04:18:34.399 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:18:34.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.548+0000 7f5140a02640 1 Processor -- start 2024-03-29T04:18:34.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.548+0000 7f5140a02640 1 -- start start 2024-03-29T04:18:34.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.548+0000 7f5140a02640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f513c1459b0 0x7f513c0b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:34.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.548+0000 7f5140a02640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f513c06e040 con 0x7f513c160350 2024-03-29T04:18:34.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.548+0000 7f5140a02640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f513c06e990 con 0x7f513c15fec0 2024-03-29T04:18:34.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.548+0000 7f5140a02640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f513c06ec70 con 0x7f513c1459b0 2024-03-29T04:18:34.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.548+0000 7f513a59c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f513c1459b0 0x7f513c0b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:34.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.548+0000 7f513a59c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f513c1459b0 0x7f513c0b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37102/0 (socket says 172.21.15.22:37102) 2024-03-29T04:18:34.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.548+0000 7f513a59c640 1 -- 172.21.15.22:0/3208093838 learned_addr learned my addr 172.21.15.22:0/3208093838 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:34.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.548+0000 7f513a59c640 1 -- 172.21.15.22:0/3208093838 >> v1:172.21.15.151:6789/0 conn(0x7f513c15fec0 legacy=0x7f513c06f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:34.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.548+0000 7f513a59c640 1 -- 172.21.15.22:0/3208093838 >> v1:172.21.15.22:6789/0 conn(0x7f513c160350 legacy=0x7f513c06fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:34.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.548+0000 7f513a59c640 1 -- 172.21.15.22:0/3208093838 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f513c06ef50 con 0x7f513c1459b0 2024-03-29T04:18:34.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.552+0000 7f513a59c640 1 --2- 172.21.15.22:0/3208093838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f513c1459b0 0x7f513c0b3f70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5124009870 tx=0x7f512401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61da51eb014147e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:34.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.552+0000 7f513959a640 1 -- 172.21.15.22:0/3208093838 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51240b1070 con 0x7f513c1459b0 2024-03-29T04:18:34.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.552+0000 7f513959a640 1 -- 172.21.15.22:0/3208093838 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51240ac410 con 0x7f513c1459b0 2024-03-29T04:18:34.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.552+0000 7f513959a640 1 -- 172.21.15.22:0/3208093838 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51240ac7e0 con 0x7f513c1459b0 2024-03-29T04:18:34.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.552+0000 7f5140a02640 1 -- 172.21.15.22:0/3208093838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f513c1459b0 msgr2=0x7f513c0b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:34.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.552+0000 7f5140a02640 1 --2- 172.21.15.22:0/3208093838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f513c1459b0 0x7f513c0b3f70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5124009870 tx=0x7f512401c560 comp rx=0 tx=0).stop 2024-03-29T04:18:34.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.552+0000 7f5140a02640 1 -- 172.21.15.22:0/3208093838 shutdown_connections 2024-03-29T04:18:34.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.552+0000 7f5140a02640 1 --2- 172.21.15.22:0/3208093838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f513c1459b0 0x7f513c0b3f70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.552+0000 7f5140a02640 1 -- 172.21.15.22:0/3208093838 >> 172.21.15.22:0/3208093838 conn(0x7f513c0b2c70 msgr2=0x7f513c0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:34.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.552+0000 7f5140a02640 1 -- 172.21.15.22:0/3208093838 shutdown_connections 2024-03-29T04:18:34.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.552+0000 7f5140a02640 1 -- 172.21.15.22:0/3208093838 wait complete. 2024-03-29T04:18:34.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.552+0000 7f5140a02640 1 Processor -- start 2024-03-29T04:18:34.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.552+0000 7f5140a02640 1 -- start start 2024-03-29T04:18:34.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5140a02640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f513c1459b0 0x7f513c159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:34.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5140a02640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f513c15fec0 0x7f513c1421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:34.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5140a02640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f513c160350 0x7f513c15e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:34.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5140a02640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f513c145f10 con 0x7f513c15fec0 2024-03-29T04:18:34.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5140a02640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f513c145d90 con 0x7f513c160350 2024-03-29T04:18:34.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5140a02640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f513c06dbd0 con 0x7f513c1459b0 2024-03-29T04:18:34.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5139d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f513c15fec0 0x7f513c1421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:34.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5139d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f513c15fec0 0x7f513c1421e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37108/0 (socket says 172.21.15.22:37108) 2024-03-29T04:18:34.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5139d9b640 1 -- 172.21.15.22:0/1631681491 learned_addr learned my addr 172.21.15.22:0/1631681491 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:34.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f513a59c640 1 --2- 172.21.15.22:0/1631681491 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f513c1459b0 0x7f513c159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:34.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f513ad9d640 1 --2- 172.21.15.22:0/1631681491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f513c160350 0x7f513c15e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:34.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5139d9b640 1 -- 172.21.15.22:0/1631681491 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f513c1459b0 msgr2=0x7f513c159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:34.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5139d9b640 1 --2- 172.21.15.22:0/1631681491 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f513c1459b0 0x7f513c159050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5139d9b640 1 -- 172.21.15.22:0/1631681491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f513c160350 msgr2=0x7f513c15e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:34.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5139d9b640 1 --2- 172.21.15.22:0/1631681491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f513c160350 0x7f513c15e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5139d9b640 1 -- 172.21.15.22:0/1631681491 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f513c143050 con 0x7f513c15fec0 2024-03-29T04:18:34.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5139d9b640 1 --2- 172.21.15.22:0/1631681491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f513c15fec0 0x7f513c1421e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f512c00c370 tx=0x7f512c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:34.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5132ffd640 1 -- 172.21.15.22:0/1631681491 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c0052d0 con 0x7f513c15fec0 2024-03-29T04:18:34.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5132ffd640 1 -- 172.21.15.22:0/1631681491 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f512c005470 con 0x7f513c15fec0 2024-03-29T04:18:34.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5140a02640 1 -- 172.21.15.22:0/1631681491 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f513c143330 con 0x7f513c15fec0 2024-03-29T04:18:34.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5132ffd640 1 -- 172.21.15.22:0/1631681491 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c007260 con 0x7f513c15fec0 2024-03-29T04:18:34.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5140a02640 1 -- 172.21.15.22:0/1631681491 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f513c1438f0 con 0x7f513c15fec0 2024-03-29T04:18:34.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5140a02640 1 -- 172.21.15.22:0/1631681491 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f513c068900 con 0x7f513c15fec0 2024-03-29T04:18:34.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5132ffd640 1 -- 172.21.15.22:0/1631681491 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f512c007400 con 0x7f513c15fec0 2024-03-29T04:18:34.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5132ffd640 1 --2- 172.21.15.22:0/1631681491 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f511c067fc0 0x7f511c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:34.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f5132ffd640 1 -- 172.21.15.22:0/1631681491 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f512c085e50 con 0x7f513c15fec0 2024-03-29T04:18:34.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.556+0000 7f513a59c640 1 --2- 172.21.15.22:0/1631681491 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f511c067fc0 0x7f511c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:34.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.560+0000 7f513a59c640 1 --2- 172.21.15.22:0/1631681491 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f511c067fc0 0x7f511c06a480 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f51240ae000 tx=0x7f5124008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:34.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.560+0000 7f5132ffd640 1 -- 172.21.15.22:0/1631681491 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f512c056160 con 0x7f513c15fec0 2024-03-29T04:18:34.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.716+0000 7f5140a02640 1 -- 172.21.15.22:0/1631681491 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f513c15fc30 con 0x7f513c15fec0 2024-03-29T04:18:34.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.716+0000 7f5132ffd640 1 -- 172.21.15.22:0/1631681491 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f512c055a60 con 0x7f513c15fec0 2024-03-29T04:18:34.698 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-29T04:18:34.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.720+0000 7f5140a02640 1 -- 172.21.15.22:0/1631681491 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f511c067fc0 msgr2=0x7f511c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:34.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.720+0000 7f5140a02640 1 --2- 172.21.15.22:0/1631681491 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f511c067fc0 0x7f511c06a480 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f51240ae000 tx=0x7f5124008040 comp rx=0 tx=0).stop 2024-03-29T04:18:34.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.720+0000 7f5140a02640 1 -- 172.21.15.22:0/1631681491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f513c15fec0 msgr2=0x7f513c1421e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:34.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.720+0000 7f5140a02640 1 --2- 172.21.15.22:0/1631681491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f513c15fec0 0x7f513c1421e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f512c00c370 tx=0x7f512c00c830 comp rx=0 tx=0).stop 2024-03-29T04:18:34.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.720+0000 7f5140a02640 1 -- 172.21.15.22:0/1631681491 shutdown_connections 2024-03-29T04:18:34.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.720+0000 7f5140a02640 1 --2- 172.21.15.22:0/1631681491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f513c160350 0x7f513c15e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.720+0000 7f5140a02640 1 --2- 172.21.15.22:0/1631681491 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f511c067fc0 0x7f511c06a480 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.720+0000 7f5140a02640 1 --2- 172.21.15.22:0/1631681491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f513c15fec0 0x7f513c1421e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.720+0000 7f5140a02640 1 --2- 172.21.15.22:0/1631681491 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f513c1459b0 0x7f513c159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.720+0000 7f5140a02640 1 -- 172.21.15.22:0/1631681491 >> 172.21.15.22:0/1631681491 conn(0x7f513c0b2c70 msgr2=0x7f513c0cbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:34.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.720+0000 7f5140a02640 1 -- 172.21.15.22:0/1631681491 shutdown_connections 2024-03-29T04:18:34.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.720+0000 7f5140a02640 1 -- 172.21.15.22:0/1631681491 wait complete. 2024-03-29T04:18:34.716 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:18:34.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f294e912640 1 Processor -- start 2024-03-29T04:18:34.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f294e912640 1 -- start start 2024-03-29T04:18:34.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f294e912640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f294815e6d0 0x7f29480b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:34.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f294e912640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29480b58c0 con 0x7f29480ae960 2024-03-29T04:18:34.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f294e912640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29480b5ba0 con 0x7f294815e2f0 2024-03-29T04:18:34.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f294e912640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f29480cd060 con 0x7f294815e6d0 2024-03-29T04:18:34.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f29477fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f29480ae960 0x7f294806c0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57136/0 (socket says 172.21.15.22:57136) 2024-03-29T04:18:34.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f29477fe640 1 -- 172.21.15.22:0/2904464428 learned_addr learned my addr 172.21.15.22:0/2904464428 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:34.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f294ceaf640 1 --2- 172.21.15.22:0/2904464428 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f294815e6d0 0x7f29480b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:34.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f2946ffd640 1 -- 172.21.15.22:0/2904464428 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2611097825 0 0) 0x7f29480b58c0 con 0x7f29480ae960 2024-03-29T04:18:34.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f2946ffd640 1 -- 172.21.15.22:0/2904464428 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2930003530 con 0x7f29480ae960 2024-03-29T04:18:34.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f2946ffd640 1 -- 172.21.15.22:0/2904464428 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3910276719 0 0) 0x7f2930003530 con 0x7f29480ae960 2024-03-29T04:18:34.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f2946ffd640 1 -- 172.21.15.22:0/2904464428 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f294815e6d0 msgr2=0x7f29480b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:34.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f2946ffd640 1 --2- 172.21.15.22:0/2904464428 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f294815e6d0 0x7f29480b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f2946ffd640 1 -- 172.21.15.22:0/2904464428 >> v1:172.21.15.151:6789/0 conn(0x7f294815e2f0 legacy=0x7f29480ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:34.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f2946ffd640 1 -- 172.21.15.22:0/2904464428 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294806b8b0 con 0x7f29480ae960 2024-03-29T04:18:34.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f2946ffd640 1 -- 172.21.15.22:0/2904464428 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f293c001680 con 0x7f29480ae960 2024-03-29T04:18:34.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f2946ffd640 1 -- 172.21.15.22:0/2904464428 >> v1:172.21.15.22:6789/0 conn(0x7f29480ae960 legacy=0x7f294806c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:34.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f2946ffd640 1 --2- 172.21.15.22:0/2904464428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29300039f0 0x7f2930003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:34.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f2946ffd640 1 --2- 172.21.15.22:0/2904464428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2930004810 0x7f2930004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:34.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f2946ffd640 1 --2- 172.21.15.22:0/2904464428 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f294815e6d0 0x7f2930005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:34.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f2946ffd640 1 -- 172.21.15.22:0/2904464428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f293000bad0 con 0x7f29300039f0 2024-03-29T04:18:34.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f2946ffd640 1 -- 172.21.15.22:0/2904464428 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f293000bc80 con 0x7f2930004810 2024-03-29T04:18:34.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f2946ffd640 1 -- 172.21.15.22:0/2904464428 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f293000be80 con 0x7f294815e6d0 2024-03-29T04:18:34.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f2947fff640 1 --2- 172.21.15.22:0/2904464428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29300039f0 0x7f2930003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:34.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f294ceaf640 1 --2- 172.21.15.22:0/2904464428 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f294815e6d0 0x7f2930005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:34.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f29477fe640 1 --2- 172.21.15.22:0/2904464428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2930004810 0x7f2930004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:34.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f2947fff640 1 -- 172.21.15.22:0/2904464428 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f294815e6d0 msgr2=0x7f2930005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:34.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f2947fff640 1 --2- 172.21.15.22:0/2904464428 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f294815e6d0 0x7f2930005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f2947fff640 1 -- 172.21.15.22:0/2904464428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2930004810 msgr2=0x7f2930004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:34.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f2947fff640 1 --2- 172.21.15.22:0/2904464428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2930004810 0x7f2930004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.856+0000 7f2947fff640 1 -- 172.21.15.22:0/2904464428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2930003530 con 0x7f29300039f0 2024-03-29T04:18:34.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.860+0000 7f2947fff640 1 --2- 172.21.15.22:0/2904464428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29300039f0 0x7f2930003dd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2938001860 tx=0x7f293801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2cc2b41092a7e90 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:34.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.860+0000 7f2946ffd640 1 -- 172.21.15.22:0/2904464428 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29380b0070 con 0x7f29300039f0 2024-03-29T04:18:34.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.860+0000 7f2946ffd640 1 -- 172.21.15.22:0/2904464428 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29380ab620 con 0x7f29300039f0 2024-03-29T04:18:34.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.860+0000 7f2946ffd640 1 -- 172.21.15.22:0/2904464428 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938001460 con 0x7f29300039f0 2024-03-29T04:18:34.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.860+0000 7f294e912640 1 -- 172.21.15.22:0/2904464428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29300039f0 msgr2=0x7f2930003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:34.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.860+0000 7f294e912640 1 --2- 172.21.15.22:0/2904464428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29300039f0 0x7f2930003dd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2938001860 tx=0x7f293801c520 comp rx=0 tx=0).stop 2024-03-29T04:18:34.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.860+0000 7f294ceaf640 1 -- 172.21.15.22:0/2904464428 reap_dead start 2024-03-29T04:18:34.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.860+0000 7f294e912640 1 -- 172.21.15.22:0/2904464428 shutdown_connections 2024-03-29T04:18:34.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.860+0000 7f294e912640 1 -- 172.21.15.22:0/2904464428 >> 172.21.15.22:0/2904464428 conn(0x7f29480d0c90 msgr2=0x7f29480d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:34.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.860+0000 7f294e912640 1 -- 172.21.15.22:0/2904464428 shutdown_connections 2024-03-29T04:18:34.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.860+0000 7f294e912640 1 -- 172.21.15.22:0/2904464428 wait complete. 2024-03-29T04:18:34.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.860+0000 7f294e912640 1 Processor -- start 2024-03-29T04:18:34.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f294e912640 1 -- start start 2024-03-29T04:18:34.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f294e912640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29300039f0 0x7f2948142a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:34.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f294e912640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f29481590a0 0x7f29481628d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:34.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f294e912640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2948157ac0 0x7f2948157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:34.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f294e912640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f294816a480 con 0x7f29300039f0 2024-03-29T04:18:34.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f294e912640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f29480b55f0 con 0x7f2948157ac0 2024-03-29T04:18:34.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f294e912640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f294816a600 con 0x7f29481590a0 2024-03-29T04:18:34.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f2947fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29300039f0 0x7f2948142a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:34.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f2947fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29300039f0 0x7f2948142a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37134/0 (socket says 172.21.15.22:37134) 2024-03-29T04:18:34.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f2947fff640 1 -- 172.21.15.22:0/2935394687 learned_addr learned my addr 172.21.15.22:0/2935394687 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:34.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f294ceaf640 1 --2- 172.21.15.22:0/2935394687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2948157ac0 0x7f2948157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:34.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f29477fe640 1 --2- 172.21.15.22:0/2935394687 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f29481590a0 0x7f29481628d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:34.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f2947fff640 1 -- 172.21.15.22:0/2935394687 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f29481590a0 msgr2=0x7f29481628d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:34.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f2947fff640 1 --2- 172.21.15.22:0/2935394687 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f29481590a0 0x7f29481628d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f2947fff640 1 -- 172.21.15.22:0/2935394687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2948157ac0 msgr2=0x7f2948157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:34.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f2947fff640 1 --2- 172.21.15.22:0/2935394687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2948157ac0 0x7f2948157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:34.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f2947fff640 1 -- 172.21.15.22:0/2935394687 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294815fa80 con 0x7f29300039f0 2024-03-29T04:18:34.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f294ceaf640 1 --2- 172.21.15.22:0/2935394687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2948157ac0 0x7f2948157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:34.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f2947fff640 1 --2- 172.21.15.22:0/2935394687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29300039f0 0x7f2948142a40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2938001860 tx=0x7f2930005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:34.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f2944ff9640 1 -- 172.21.15.22:0/2935394687 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29380b9020 con 0x7f29300039f0 2024-03-29T04:18:34.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f2944ff9640 1 -- 172.21.15.22:0/2935394687 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2938001440 con 0x7f29300039f0 2024-03-29T04:18:34.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f294e912640 1 -- 172.21.15.22:0/2935394687 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294815ecc0 con 0x7f29300039f0 2024-03-29T04:18:34.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f2944ff9640 1 -- 172.21.15.22:0/2935394687 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29380b0040 con 0x7f29300039f0 2024-03-29T04:18:34.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f294e912640 1 -- 172.21.15.22:0/2935394687 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f294815f280 con 0x7f29300039f0 2024-03-29T04:18:34.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f294e912640 1 -- 172.21.15.22:0/2935394687 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2948068900 con 0x7f29300039f0 2024-03-29T04:18:34.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f2944ff9640 1 -- 172.21.15.22:0/2935394687 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f29380bc020 con 0x7f29300039f0 2024-03-29T04:18:34.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.864+0000 7f2944ff9640 1 --2- 172.21.15.22:0/2935394687 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2934067fc0 0x7f293406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:34.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.868+0000 7f2944ff9640 1 -- 172.21.15.22:0/2935394687 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f2938120600 con 0x7f29300039f0 2024-03-29T04:18:34.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.868+0000 7f29477fe640 1 --2- 172.21.15.22:0/2935394687 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2934067fc0 0x7f293406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:34.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.868+0000 7f29477fe640 1 --2- 172.21.15.22:0/2935394687 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2934067fc0 0x7f293406a480 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f293c001210 tx=0x7f293c007040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:34.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:34.872+0000 7f2944ff9640 1 -- 172.21.15.22:0/2935394687 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f29380c3030 con 0x7f29300039f0 2024-03-29T04:18:35.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.036+0000 7f294e912640 1 -- 172.21.15.22:0/2935394687 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2948070680 con 0x7f29300039f0 2024-03-29T04:18:35.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.036+0000 7f2944ff9640 1 -- 172.21.15.22:0/2935394687 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f2938016230 con 0x7f29300039f0 2024-03-29T04:18:35.016 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:35.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.036+0000 7f294e912640 1 -- 172.21.15.22:0/2935394687 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2934067fc0 msgr2=0x7f293406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:35.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.036+0000 7f294e912640 1 --2- 172.21.15.22:0/2935394687 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2934067fc0 0x7f293406a480 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f293c001210 tx=0x7f293c007040 comp rx=0 tx=0).stop 2024-03-29T04:18:35.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.036+0000 7f294e912640 1 -- 172.21.15.22:0/2935394687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29300039f0 msgr2=0x7f2948142a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:35.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.036+0000 7f294e912640 1 --2- 172.21.15.22:0/2935394687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29300039f0 0x7f2948142a40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2938001860 tx=0x7f2930005840 comp rx=0 tx=0).stop 2024-03-29T04:18:35.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.036+0000 7f294e912640 1 -- 172.21.15.22:0/2935394687 shutdown_connections 2024-03-29T04:18:35.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.036+0000 7f294e912640 1 --2- 172.21.15.22:0/2935394687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2948157ac0 0x7f2948157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.036+0000 7f294e912640 1 --2- 172.21.15.22:0/2935394687 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f29481590a0 0x7f29481628d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.036+0000 7f294e912640 1 --2- 172.21.15.22:0/2935394687 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2934067fc0 0x7f293406a480 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.036+0000 7f294e912640 1 --2- 172.21.15.22:0/2935394687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29300039f0 0x7f2948142a40 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.036+0000 7f294e912640 1 -- 172.21.15.22:0/2935394687 >> 172.21.15.22:0/2935394687 conn(0x7f29480d0c90 msgr2=0x7f29480b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:35.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.036+0000 7f294e912640 1 -- 172.21.15.22:0/2935394687 shutdown_connections 2024-03-29T04:18:35.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.036+0000 7f294e912640 1 -- 172.21.15.22:0/2935394687 wait complete. 2024-03-29T04:18:35.035 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:18:35.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.176+0000 7f36f1293640 1 Processor -- start 2024-03-29T04:18:35.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.176+0000 7f36f1293640 1 -- start start 2024-03-29T04:18:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.176+0000 7f36f1293640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f36ec15bf50 0x7f36ec15e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.176+0000 7f36f1293640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f36ec0b42d0 0x7f36ec0b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.176+0000 7f36f1293640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f36ec161f70 0x7f36ec0b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.176+0000 7f36f1293640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f36ec06b720 con 0x7f36ec15bf50 2024-03-29T04:18:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.176+0000 7f36f1293640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f36ec15c330 con 0x7f36ec0b42d0 2024-03-29T04:18:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.176+0000 7f36f1293640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f36ec1624e0 con 0x7f36ec161f70 2024-03-29T04:18:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.176+0000 7f36eaffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f36ec15bf50 0x7f36ec15e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.176+0000 7f36eaffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f36ec15bf50 0x7f36ec15e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51066/0 (socket says 172.21.15.22:51066) 2024-03-29T04:18:35.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.176+0000 7f36eaffd640 1 -- 172.21.15.22:0/3752539351 learned_addr learned my addr 172.21.15.22:0/3752539351 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:35.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.176+0000 7f36ea7fc640 1 --2- 172.21.15.22:0/3752539351 >> v2:172.21.15.151:3300/0 conn(0x7f36ec0b42d0 0x7f36ec0b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:35.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.176+0000 7f36eb7fe640 1 --2- 172.21.15.22:0/3752539351 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f36ec161f70 0x7f36ec0b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:35.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.176+0000 7f36eaffd640 1 -- 172.21.15.22:0/3752539351 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f36ec161f70 msgr2=0x7f36ec0b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:35.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.176+0000 7f36eaffd640 1 --2- 172.21.15.22:0/3752539351 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f36ec161f70 0x7f36ec0b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.176+0000 7f36eaffd640 1 -- 172.21.15.22:0/3752539351 >> v2:172.21.15.151:3300/0 conn(0x7f36ec0b42d0 msgr2=0x7f36ec0b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:35.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.180+0000 7f36eaffd640 1 --2- 172.21.15.22:0/3752539351 >> v2:172.21.15.151:3300/0 conn(0x7f36ec0b42d0 0x7f36ec0b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.180+0000 7f36eaffd640 1 -- 172.21.15.22:0/3752539351 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ec1627c0 con 0x7f36ec15bf50 2024-03-29T04:18:35.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.180+0000 7f36eaffd640 1 --2- 172.21.15.22:0/3752539351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36ec15bf50 0x7f36ec15e930 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0b4e70 tx=0x7f36dc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7d609feeb19737 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:35.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.180+0000 7f36e9ffb640 1 -- 172.21.15.22:0/3752539351 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc0aa430 con 0x7f36ec15bf50 2024-03-29T04:18:35.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.180+0000 7f36e9ffb640 1 -- 172.21.15.22:0/3752539351 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36dc0af070 con 0x7f36ec15bf50 2024-03-29T04:18:35.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.180+0000 7f36e9ffb640 1 -- 172.21.15.22:0/3752539351 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc0aa7f0 con 0x7f36ec15bf50 2024-03-29T04:18:35.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.180+0000 7f36f1293640 1 -- 172.21.15.22:0/3752539351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36ec15bf50 msgr2=0x7f36ec15e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:35.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.180+0000 7f36f1293640 1 --2- 172.21.15.22:0/3752539351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36ec15bf50 0x7f36ec15e930 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0b4e70 tx=0x7f36dc01c4f0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.180+0000 7f36f1293640 1 -- 172.21.15.22:0/3752539351 shutdown_connections 2024-03-29T04:18:35.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.180+0000 7f36f1293640 1 --2- 172.21.15.22:0/3752539351 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f36ec161f70 0x7f36ec0b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.180+0000 7f36f1293640 1 --2- 172.21.15.22:0/3752539351 >> v2:172.21.15.151:3300/0 conn(0x7f36ec0b42d0 0x7f36ec0b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.180+0000 7f36f1293640 1 --2- 172.21.15.22:0/3752539351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36ec15bf50 0x7f36ec15e930 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.180+0000 7f36f1293640 1 -- 172.21.15.22:0/3752539351 >> 172.21.15.22:0/3752539351 conn(0x7f36ec0b2980 msgr2=0x7f36ec0d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:35.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.180+0000 7f36f1293640 1 -- 172.21.15.22:0/3752539351 shutdown_connections 2024-03-29T04:18:35.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.180+0000 7f36f1293640 1 -- 172.21.15.22:0/3752539351 wait complete. 2024-03-29T04:18:35.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36f1293640 1 Processor -- start 2024-03-29T04:18:35.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36f1293640 1 -- start start 2024-03-29T04:18:35.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36f1293640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36ec0b42d0 0x7f36ec14db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36f1293640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36ec15bf50 0x7f36ec156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36f1293640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f36ec161f70 0x7f36ec14c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36f1293640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec06ed00 con 0x7f36ec0b42d0 2024-03-29T04:18:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36f1293640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec06eb80 con 0x7f36ec15bf50 2024-03-29T04:18:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36f1293640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f36ec06ee80 con 0x7f36ec161f70 2024-03-29T04:18:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36eaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36ec0b42d0 0x7f36ec14db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36eaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36ec0b42d0 0x7f36ec14db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51072/0 (socket says 172.21.15.22:51072) 2024-03-29T04:18:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36eaffd640 1 -- 172.21.15.22:0/2207218172 learned_addr learned my addr 172.21.15.22:0/2207218172 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36ea7fc640 1 --2- 172.21.15.22:0/2207218172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36ec15bf50 0x7f36ec156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36eb7fe640 1 --2- 172.21.15.22:0/2207218172 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f36ec161f70 0x7f36ec14c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36eaffd640 1 -- 172.21.15.22:0/2207218172 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f36ec161f70 msgr2=0x7f36ec14c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36eaffd640 1 --2- 172.21.15.22:0/2207218172 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f36ec161f70 0x7f36ec14c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36eaffd640 1 -- 172.21.15.22:0/2207218172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36ec15bf50 msgr2=0x7f36ec156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36eaffd640 1 --2- 172.21.15.22:0/2207218172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36ec15bf50 0x7f36ec156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36eaffd640 1 -- 172.21.15.22:0/2207218172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ec160d70 con 0x7f36ec0b42d0 2024-03-29T04:18:35.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36eaffd640 1 --2- 172.21.15.22:0/2207218172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36ec0b42d0 0x7f36ec14db30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f36dc01aba0 tx=0x7f36dc01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:35.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36d37fe640 1 -- 172.21.15.22:0/2207218172 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc0bd5f0 con 0x7f36ec0b42d0 2024-03-29T04:18:35.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36d37fe640 1 -- 172.21.15.22:0/2207218172 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36dc0bd790 con 0x7f36ec0b42d0 2024-03-29T04:18:35.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36d37fe640 1 -- 172.21.15.22:0/2207218172 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc0bdb80 con 0x7f36ec0b42d0 2024-03-29T04:18:35.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36f1293640 1 -- 172.21.15.22:0/2207218172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ec14cd70 con 0x7f36ec0b42d0 2024-03-29T04:18:35.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.184+0000 7f36f1293640 1 -- 172.21.15.22:0/2207218172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ec14d330 con 0x7f36ec0b42d0 2024-03-29T04:18:35.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.188+0000 7f36f1293640 1 -- 172.21.15.22:0/2207218172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36ec068890 con 0x7f36ec0b42d0 2024-03-29T04:18:35.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.188+0000 7f36d37fe640 1 -- 172.21.15.22:0/2207218172 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f36dc0b33f0 con 0x7f36ec0b42d0 2024-03-29T04:18:35.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.188+0000 7f36d37fe640 1 --2- 172.21.15.22:0/2207218172 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f36d8067fc0 0x7f36d806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:35.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.188+0000 7f36d37fe640 1 -- 172.21.15.22:0/2207218172 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f36dc1210d0 con 0x7f36ec0b42d0 2024-03-29T04:18:35.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.188+0000 7f36ea7fc640 1 --2- 172.21.15.22:0/2207218172 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f36d8067fc0 0x7f36d806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:35.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.188+0000 7f36ea7fc640 1 --2- 172.21.15.22:0/2207218172 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f36d8067fc0 0x7f36d806a480 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f36ec15e0c0 tx=0x7f36e0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:35.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.192+0000 7f36d37fe640 1 -- 172.21.15.22:0/2207218172 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f36dc016110 con 0x7f36ec0b42d0 2024-03-29T04:18:35.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.360+0000 7f36f1293640 1 -- 172.21.15.22:0/2207218172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f36ec0d1340 con 0x7f36ec0b42d0 2024-03-29T04:18:35.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.360+0000 7f36d37fe640 1 -- 172.21.15.22:0/2207218172 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f36dc0f0ec0 con 0x7f36ec0b42d0 2024-03-29T04:18:35.341 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-03-29T04:18:35.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.360+0000 7f36f1293640 1 -- 172.21.15.22:0/2207218172 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f36d8067fc0 msgr2=0x7f36d806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:35.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.364+0000 7f36f1293640 1 --2- 172.21.15.22:0/2207218172 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f36d8067fc0 0x7f36d806a480 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f36ec15e0c0 tx=0x7f36e0008040 comp rx=0 tx=0).stop 2024-03-29T04:18:35.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.364+0000 7f36f1293640 1 -- 172.21.15.22:0/2207218172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36ec0b42d0 msgr2=0x7f36ec14db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:35.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.364+0000 7f36f1293640 1 --2- 172.21.15.22:0/2207218172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36ec0b42d0 0x7f36ec14db30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f36dc01aba0 tx=0x7f36dc01ad30 comp rx=0 tx=0).stop 2024-03-29T04:18:35.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.364+0000 7f36f1293640 1 -- 172.21.15.22:0/2207218172 shutdown_connections 2024-03-29T04:18:35.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.364+0000 7f36f1293640 1 --2- 172.21.15.22:0/2207218172 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f36ec161f70 0x7f36ec14c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.364+0000 7f36f1293640 1 --2- 172.21.15.22:0/2207218172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36ec15bf50 0x7f36ec156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.364+0000 7f36f1293640 1 --2- 172.21.15.22:0/2207218172 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f36d8067fc0 0x7f36d806a480 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.364+0000 7f36f1293640 1 --2- 172.21.15.22:0/2207218172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36ec0b42d0 0x7f36ec14db30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.364+0000 7f36f1293640 1 -- 172.21.15.22:0/2207218172 >> 172.21.15.22:0/2207218172 conn(0x7f36ec0b2980 msgr2=0x7f36ec154b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:35.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.364+0000 7f36f1293640 1 -- 172.21.15.22:0/2207218172 shutdown_connections 2024-03-29T04:18:35.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.364+0000 7f36f1293640 1 -- 172.21.15.22:0/2207218172 wait complete. 2024-03-29T04:18:35.359 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi151.front.sepia.ceph.com... 2024-03-29T04:18:35.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:35.359 DEBUG:teuthology.orchestra.run.smithi151:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-03-29T04:18:35.382 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:18:35.382 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-03-29T04:18:35.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:35.382 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-29T04:18:35.439 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:18:35.439 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:18:35.445 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:18:35.445 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:18:35.492 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:35.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.672+0000 7f0509b67640 1 Processor -- start 2024-03-29T04:18:35.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f0509b67640 1 -- start start 2024-03-29T04:18:35.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f0509b67640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f050406ea60 0x7f0504165790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:35.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f0509b67640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0504072cb0 con 0x7f050415a840 2024-03-29T04:18:35.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f0509b67640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0504072f90 con 0x7f050415eb10 2024-03-29T04:18:35.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f0509b67640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0504073270 con 0x7f050406ea60 2024-03-29T04:18:35.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f05037fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f050415a840 0x7f050415e400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58466/0 (socket says 172.21.15.22:58466) 2024-03-29T04:18:35.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f05037fe640 1 -- 172.21.15.22:0/2162305428 learned_addr learned my addr 172.21.15.22:0/2162305428 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:35.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f0503fff640 1 --2- 172.21.15.22:0/2162305428 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f050406ea60 0x7f0504165790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:35.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f05027fc640 1 -- 172.21.15.22:0/2162305428 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2089889594 0 0) 0x7f0504072cb0 con 0x7f050415a840 2024-03-29T04:18:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f05027fc640 1 -- 172.21.15.22:0/2162305428 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04ec003530 con 0x7f050415a840 2024-03-29T04:18:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f05027fc640 1 -- 172.21.15.22:0/2162305428 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 225066106 0 0) 0x7f04ec003530 con 0x7f050415a840 2024-03-29T04:18:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f05027fc640 1 -- 172.21.15.22:0/2162305428 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f050406ea60 msgr2=0x7f0504165790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f05027fc640 1 --2- 172.21.15.22:0/2162305428 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f050406ea60 0x7f0504165790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f05027fc640 1 -- 172.21.15.22:0/2162305428 >> v1:172.21.15.151:6789/0 conn(0x7f050415eb10 legacy=0x7f05040b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f05027fc640 1 -- 172.21.15.22:0/2162305428 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0504073550 con 0x7f050415a840 2024-03-29T04:18:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f05027fc640 1 -- 172.21.15.22:0/2162305428 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f04f4001680 con 0x7f050415a840 2024-03-29T04:18:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f0503fff640 1 --2- 172.21.15.22:0/2162305428 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f050406ea60 0x7f0504165790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f05027fc640 1 -- 172.21.15.22:0/2162305428 >> v1:172.21.15.22:6789/0 conn(0x7f050415a840 legacy=0x7f050415e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f05027fc640 1 --2- 172.21.15.22:0/2162305428 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f050406ea60 0x7f04ec0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f05027fc640 1 --2- 172.21.15.22:0/2162305428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04ec003fe0 0x7f04ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f05027fc640 1 --2- 172.21.15.22:0/2162305428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04ec004d50 0x7f04ec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f05027fc640 1 -- 172.21.15.22:0/2162305428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0504073270 con 0x7f04ec004d50 2024-03-29T04:18:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f05027fc640 1 -- 172.21.15.22:0/2162305428 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f04ec00bbc0 con 0x7f04ec003fe0 2024-03-29T04:18:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f05027fc640 1 -- 172.21.15.22:0/2162305428 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f04ec00bdc0 con 0x7f050406ea60 2024-03-29T04:18:35.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f0503fff640 1 --2- 172.21.15.22:0/2162305428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04ec004d50 0x7f04ec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:35.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f05037fe640 1 --2- 172.21.15.22:0/2162305428 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f050406ea60 0x7f04ec0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:35.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f0502ffd640 1 --2- 172.21.15.22:0/2162305428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04ec003fe0 0x7f04ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:35.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f0503fff640 1 -- 172.21.15.22:0/2162305428 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f050406ea60 msgr2=0x7f04ec0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:35.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f0503fff640 1 --2- 172.21.15.22:0/2162305428 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f050406ea60 0x7f04ec0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f0503fff640 1 -- 172.21.15.22:0/2162305428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04ec003fe0 msgr2=0x7f04ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:35.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f0503fff640 1 --2- 172.21.15.22:0/2162305428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04ec003fe0 0x7f04ec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.676+0000 7f0503fff640 1 -- 172.21.15.22:0/2162305428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ec003530 con 0x7f04ec004d50 2024-03-29T04:18:35.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0503fff640 1 --2- 172.21.15.22:0/2162305428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04ec004d50 0x7f04ec0051d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0073e0 tx=0x7f04fc01cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d87d694ac12b66d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:35.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f05027fc640 1 -- 172.21.15.22:0/2162305428 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0b2070 con 0x7f04ec004d50 2024-03-29T04:18:35.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f05027fc640 1 -- 172.21.15.22:0/2162305428 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04fc0a5cb0 con 0x7f04ec004d50 2024-03-29T04:18:35.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f05027fc640 1 -- 172.21.15.22:0/2162305428 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc009c80 con 0x7f04ec004d50 2024-03-29T04:18:35.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0509b67640 1 -- 172.21.15.22:0/2162305428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04ec004d50 msgr2=0x7f04ec0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:35.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0509b67640 1 --2- 172.21.15.22:0/2162305428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04ec004d50 0x7f04ec0051d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0073e0 tx=0x7f04fc01cec0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0503fff640 1 -- 172.21.15.22:0/2162305428 reap_dead start 2024-03-29T04:18:35.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0509b67640 1 -- 172.21.15.22:0/2162305428 shutdown_connections 2024-03-29T04:18:35.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0509b67640 1 -- 172.21.15.22:0/2162305428 >> 172.21.15.22:0/2162305428 conn(0x7f05040cfee0 msgr2=0x7f05040d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:35.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0509b67640 1 -- 172.21.15.22:0/2162305428 shutdown_connections 2024-03-29T04:18:35.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0509b67640 1 -- 172.21.15.22:0/2162305428 wait complete. 2024-03-29T04:18:35.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0509b67640 1 Processor -- start 2024-03-29T04:18:35.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0509b67640 1 -- start start 2024-03-29T04:18:35.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0509b67640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f050414a3d0 0x7f05040ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:35.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0509b67640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05040ce5e0 0x7f05041480f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:35.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0509b67640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0504148630 0x7f050415ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:35.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0509b67640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f050415f270 con 0x7f05040ce5e0 2024-03-29T04:18:35.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0509b67640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f050415f0f0 con 0x7f050414a3d0 2024-03-29T04:18:35.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0509b67640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f05040728d0 con 0x7f0504148630 2024-03-29T04:18:35.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0502ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05040ce5e0 0x7f05041480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:35.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0502ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05040ce5e0 0x7f05041480f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51080/0 (socket says 172.21.15.22:51080) 2024-03-29T04:18:35.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0502ffd640 1 -- 172.21.15.22:0/2355357023 learned_addr learned my addr 172.21.15.22:0/2355357023 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:35.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f05037fe640 1 --2- 172.21.15.22:0/2355357023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f050414a3d0 0x7f05040ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:35.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0503fff640 1 --2- 172.21.15.22:0/2355357023 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0504148630 0x7f050415ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:35.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0502ffd640 1 -- 172.21.15.22:0/2355357023 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0504148630 msgr2=0x7f050415ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:35.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0502ffd640 1 --2- 172.21.15.22:0/2355357023 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0504148630 0x7f050415ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0502ffd640 1 -- 172.21.15.22:0/2355357023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f050414a3d0 msgr2=0x7f05040ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:35.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0502ffd640 1 --2- 172.21.15.22:0/2355357023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f050414a3d0 0x7f05040ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0502ffd640 1 -- 172.21.15.22:0/2355357023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0504147050 con 0x7f05040ce5e0 2024-03-29T04:18:35.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.680+0000 7f0502ffd640 1 --2- 172.21.15.22:0/2355357023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05040ce5e0 0x7f05041480f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f04f800e550 tx=0x7f04f800e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:35.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.684+0000 7f04ebfff640 1 -- 172.21.15.22:0/2355357023 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f80052e0 con 0x7f05040ce5e0 2024-03-29T04:18:35.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.684+0000 7f04ebfff640 1 -- 172.21.15.22:0/2355357023 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f8018070 con 0x7f05040ce5e0 2024-03-29T04:18:35.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.684+0000 7f0509b67640 1 -- 172.21.15.22:0/2355357023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0504147330 con 0x7f05040ce5e0 2024-03-29T04:18:35.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.684+0000 7f04ebfff640 1 -- 172.21.15.22:0/2355357023 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8003820 con 0x7f05040ce5e0 2024-03-29T04:18:35.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.684+0000 7f0509b67640 1 -- 172.21.15.22:0/2355357023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05041478f0 con 0x7f05040ce5e0 2024-03-29T04:18:35.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.684+0000 7f0509b67640 1 -- 172.21.15.22:0/2355357023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0504068890 con 0x7f05040ce5e0 2024-03-29T04:18:35.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.684+0000 7f04ebfff640 1 -- 172.21.15.22:0/2355357023 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f04f80039c0 con 0x7f05040ce5e0 2024-03-29T04:18:35.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.684+0000 7f04ebfff640 1 --2- 172.21.15.22:0/2355357023 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f04f0067ff0 0x7f04f006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:35.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.684+0000 7f04ebfff640 1 -- 172.21.15.22:0/2355357023 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f04f80865c0 con 0x7f05040ce5e0 2024-03-29T04:18:35.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.684+0000 7f05037fe640 1 --2- 172.21.15.22:0/2355357023 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f04f0067ff0 0x7f04f006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:35.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.684+0000 7f05037fe640 1 --2- 172.21.15.22:0/2355357023 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f04f0067ff0 0x7f04f006a4b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f04f4001ec0 tx=0x7f04f4001530 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:35.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.688+0000 7f04ebfff640 1 -- 172.21.15.22:0/2355357023 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f04f8056970 con 0x7f05040ce5e0 2024-03-29T04:18:35.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.848+0000 7f0509b67640 1 -- 172.21.15.22:0/2355357023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0504069020 con 0x7f05040ce5e0 2024-03-29T04:18:35.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.852+0000 7f04ebfff640 1 -- 172.21.15.22:0/2355357023 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18847 (secure 0 0 0) 0x7f04f8005480 con 0x7f05040ce5e0 2024-03-29T04:18:35.833 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:35.833 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:18:16.893439+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:35.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.852+0000 7f0509b67640 1 -- 172.21.15.22:0/2355357023 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f04f0067ff0 msgr2=0x7f04f006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:35.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.856+0000 7f0509b67640 1 --2- 172.21.15.22:0/2355357023 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f04f0067ff0 0x7f04f006a4b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f04f4001ec0 tx=0x7f04f4001530 comp rx=0 tx=0).stop 2024-03-29T04:18:35.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.856+0000 7f0509b67640 1 -- 172.21.15.22:0/2355357023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05040ce5e0 msgr2=0x7f05041480f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:35.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.856+0000 7f0509b67640 1 --2- 172.21.15.22:0/2355357023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05040ce5e0 0x7f05041480f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f04f800e550 tx=0x7f04f800e950 comp rx=0 tx=0).stop 2024-03-29T04:18:35.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.856+0000 7f0509b67640 1 -- 172.21.15.22:0/2355357023 shutdown_connections 2024-03-29T04:18:35.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.856+0000 7f0509b67640 1 --2- 172.21.15.22:0/2355357023 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0504148630 0x7f050415ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.856+0000 7f0509b67640 1 --2- 172.21.15.22:0/2355357023 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f04f0067ff0 0x7f04f006a4b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.856+0000 7f0509b67640 1 --2- 172.21.15.22:0/2355357023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05040ce5e0 0x7f05041480f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.856+0000 7f0509b67640 1 --2- 172.21.15.22:0/2355357023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f050414a3d0 0x7f05040ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.856+0000 7f0509b67640 1 -- 172.21.15.22:0/2355357023 >> 172.21.15.22:0/2355357023 conn(0x7f05040cfee0 msgr2=0x7f050414b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:35.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.856+0000 7f0509b67640 1 -- 172.21.15.22:0/2355357023 shutdown_connections 2024-03-29T04:18:35.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:35.856+0000 7f0509b67640 1 -- 172.21.15.22:0/2355357023 wait complete. 2024-03-29T04:18:35.853 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:17:34.034142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:17:34.886659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:17:38.628632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:18:35.853 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:18:35.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.004+0000 7f28bdb6f640 1 Processor -- start 2024-03-29T04:18:35.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.004+0000 7f28bdb6f640 1 -- start start 2024-03-29T04:18:35.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.004+0000 7f28bdb6f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f28b815e4c0 0x7f28b80b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:35.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.004+0000 7f28bdb6f640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28b806c850 con 0x7f28b806c070 2024-03-29T04:18:35.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.004+0000 7f28bdb6f640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28b806cb30 con 0x7f28b80cf6c0 2024-03-29T04:18:35.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.004+0000 7f28bdb6f640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f28b815e8a0 con 0x7f28b815e4c0 2024-03-29T04:18:35.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.004+0000 7f28b77fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f28b815e4c0 0x7f28b80b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:35.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.004+0000 7f28b77fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f28b815e4c0 0x7f28b80b9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51084/0 (socket says 172.21.15.22:51084) 2024-03-29T04:18:35.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.004+0000 7f28b77fe640 1 -- 172.21.15.22:0/3497327293 learned_addr learned my addr 172.21.15.22:0/3497327293 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:35.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.004+0000 7f28b77fe640 1 -- 172.21.15.22:0/3497327293 >> v1:172.21.15.151:6789/0 conn(0x7f28b80cf6c0 legacy=0x7f28b80d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:35.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.004+0000 7f28b77fe640 1 -- 172.21.15.22:0/3497327293 >> v1:172.21.15.22:6789/0 conn(0x7f28b806c070 legacy=0x7f28b80cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:35.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.004+0000 7f28b77fe640 1 -- 172.21.15.22:0/3497327293 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b806ce10 con 0x7f28b815e4c0 2024-03-29T04:18:35.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28b77fe640 1 --2- 172.21.15.22:0/3497327293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b815e4c0 0x7f28b80b9e30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009870 tx=0x7f28a801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a65113259063a03 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:35.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28b67fc640 1 -- 172.21.15.22:0/3497327293 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a80b2070 con 0x7f28b815e4c0 2024-03-29T04:18:35.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28b67fc640 1 -- 172.21.15.22:0/3497327293 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a80a9070 con 0x7f28b815e4c0 2024-03-29T04:18:35.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28b67fc640 1 -- 172.21.15.22:0/3497327293 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a80ad880 con 0x7f28b815e4c0 2024-03-29T04:18:35.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28bdb6f640 1 -- 172.21.15.22:0/3497327293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b815e4c0 msgr2=0x7f28b80b9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:35.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28bdb6f640 1 --2- 172.21.15.22:0/3497327293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b815e4c0 0x7f28b80b9e30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009870 tx=0x7f28a801c560 comp rx=0 tx=0).stop 2024-03-29T04:18:35.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28bdb6f640 1 -- 172.21.15.22:0/3497327293 shutdown_connections 2024-03-29T04:18:35.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28bdb6f640 1 --2- 172.21.15.22:0/3497327293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b815e4c0 0x7f28b80b9e30 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28bdb6f640 1 -- 172.21.15.22:0/3497327293 >> 172.21.15.22:0/3497327293 conn(0x7f28b80b6410 msgr2=0x7f28b80d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:35.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28bdb6f640 1 -- 172.21.15.22:0/3497327293 shutdown_connections 2024-03-29T04:18:35.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28bdb6f640 1 -- 172.21.15.22:0/3497327293 wait complete. 2024-03-29T04:18:35.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28bdb6f640 1 Processor -- start 2024-03-29T04:18:35.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28bdb6f640 1 -- start start 2024-03-29T04:18:35.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28bdb6f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b806c070 0x7f28b81595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:35.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28bdb6f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28b80cf6c0 0x7f28b815a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:35.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28bdb6f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b815e4c0 0x7f28b8157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:35.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28bdb6f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f28b816c050 con 0x7f28b806c070 2024-03-29T04:18:35.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28bdb6f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f28b80d3f60 con 0x7f28b815e4c0 2024-03-29T04:18:35.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28bdb6f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f28b816c1d0 con 0x7f28b80cf6c0 2024-03-29T04:18:35.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28b77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b806c070 0x7f28b81595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:35.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28b77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b806c070 0x7f28b81595c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51094/0 (socket says 172.21.15.22:51094) 2024-03-29T04:18:35.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28b77fe640 1 -- 172.21.15.22:0/45498061 learned_addr learned my addr 172.21.15.22:0/45498061 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:35.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28b6ffd640 1 --2- 172.21.15.22:0/45498061 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28b80cf6c0 0x7f28b815a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:35.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28b7fff640 1 --2- 172.21.15.22:0/45498061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b815e4c0 0x7f28b8157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:35.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28b77fe640 1 -- 172.21.15.22:0/45498061 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28b80cf6c0 msgr2=0x7f28b815a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:35.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28b77fe640 1 --2- 172.21.15.22:0/45498061 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28b80cf6c0 0x7f28b815a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28b77fe640 1 -- 172.21.15.22:0/45498061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b815e4c0 msgr2=0x7f28b8157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:35.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28b77fe640 1 --2- 172.21.15.22:0/45498061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b815e4c0 0x7f28b8157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:35.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28b77fe640 1 -- 172.21.15.22:0/45498061 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b80c9150 con 0x7f28b806c070 2024-03-29T04:18:35.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.008+0000 7f28b77fe640 1 --2- 172.21.15.22:0/45498061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b806c070 0x7f28b81595c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f28a80b0040 tx=0x7f28a8019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:35.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.012+0000 7f289ffff640 1 -- 172.21.15.22:0/45498061 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a80b2040 con 0x7f28b806c070 2024-03-29T04:18:35.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.012+0000 7f289ffff640 1 -- 172.21.15.22:0/45498061 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a80bb070 con 0x7f28b806c070 2024-03-29T04:18:35.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.012+0000 7f28bdb6f640 1 -- 172.21.15.22:0/45498061 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b80c9430 con 0x7f28b806c070 2024-03-29T04:18:35.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.012+0000 7f289ffff640 1 -- 172.21.15.22:0/45498061 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a80b6860 con 0x7f28b806c070 2024-03-29T04:18:35.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.012+0000 7f28bdb6f640 1 -- 172.21.15.22:0/45498061 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b80c9cd0 con 0x7f28b806c070 2024-03-29T04:18:35.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.012+0000 7f28bdb6f640 1 -- 172.21.15.22:0/45498061 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b80c9600 con 0x7f28b806c070 2024-03-29T04:18:35.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.012+0000 7f289ffff640 1 -- 172.21.15.22:0/45498061 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f28a80ab070 con 0x7f28b806c070 2024-03-29T04:18:35.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.012+0000 7f289ffff640 1 --2- 172.21.15.22:0/45498061 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f28a4067fc0 0x7f28a406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:35.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.012+0000 7f289ffff640 1 -- 172.21.15.22:0/45498061 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f28a8120590 con 0x7f28b806c070 2024-03-29T04:18:35.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.012+0000 7f28b6ffd640 1 --2- 172.21.15.22:0/45498061 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f28a4067fc0 0x7f28a406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:35.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.012+0000 7f28b6ffd640 1 --2- 172.21.15.22:0/45498061 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f28a4067fc0 0x7f28a406a480 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f28b80cc160 tx=0x7f28ac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:35.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.016+0000 7f289ffff640 1 -- 172.21.15.22:0/45498061 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f28a8016150 con 0x7f28b806c070 2024-03-29T04:18:36.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.180+0000 7f28bdb6f640 1 -- 172.21.15.22:0/45498061 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f28b8068900 con 0x7f28b806c070 2024-03-29T04:18:36.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.180+0000 7f289ffff640 1 -- 172.21.15.22:0/45498061 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f28a80f0940 con 0x7f28b806c070 2024-03-29T04:18:36.162 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:18:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.184+0000 7f28bdb6f640 1 -- 172.21.15.22:0/45498061 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f28a4067fc0 msgr2=0x7f28a406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.184+0000 7f28bdb6f640 1 --2- 172.21.15.22:0/45498061 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f28a4067fc0 0x7f28a406a480 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f28b80cc160 tx=0x7f28ac009040 comp rx=0 tx=0).stop 2024-03-29T04:18:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.184+0000 7f28bdb6f640 1 -- 172.21.15.22:0/45498061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b806c070 msgr2=0x7f28b81595c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.184+0000 7f28bdb6f640 1 --2- 172.21.15.22:0/45498061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b806c070 0x7f28b81595c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f28a80b0040 tx=0x7f28a8019300 comp rx=0 tx=0).stop 2024-03-29T04:18:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.184+0000 7f28bdb6f640 1 -- 172.21.15.22:0/45498061 shutdown_connections 2024-03-29T04:18:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.184+0000 7f28bdb6f640 1 --2- 172.21.15.22:0/45498061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b815e4c0 0x7f28b8157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.184+0000 7f28bdb6f640 1 --2- 172.21.15.22:0/45498061 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f28b80cf6c0 0x7f28b815a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.184+0000 7f28bdb6f640 1 --2- 172.21.15.22:0/45498061 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f28a4067fc0 0x7f28a406a480 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.184+0000 7f28bdb6f640 1 --2- 172.21.15.22:0/45498061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b806c070 0x7f28b81595c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.184+0000 7f28bdb6f640 1 -- 172.21.15.22:0/45498061 >> 172.21.15.22:0/45498061 conn(0x7f28b80b6410 msgr2=0x7f28b80b8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.184+0000 7f28bdb6f640 1 -- 172.21.15.22:0/45498061 shutdown_connections 2024-03-29T04:18:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.184+0000 7f28bdb6f640 1 -- 172.21.15.22:0/45498061 wait complete. 2024-03-29T04:18:36.182 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:18:36.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.328+0000 7f86a6271640 1 Processor -- start 2024-03-29T04:18:36.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.328+0000 7f86a6271640 1 -- start start 2024-03-29T04:18:36.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.332+0000 7f86a6271640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86a01479b0 0x7f86a006f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:36.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.332+0000 7f86a6271640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f86a0147d90 0x7f86a00ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:36.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.332+0000 7f86a6271640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86a006e960 con 0x7f86a00ca830 2024-03-29T04:18:36.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.332+0000 7f86a6271640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f86a006fc60 con 0x7f86a0147d90 2024-03-29T04:18:36.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.332+0000 7f86a6271640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f86a006ec40 con 0x7f86a01479b0 2024-03-29T04:18:36.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.332+0000 7f869ffff640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86a01479b0 0x7f86a006f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:36.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.332+0000 7f869ffff640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86a01479b0 0x7f86a006f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.22:43144/0 (socket says 172.21.15.22:43144) 2024-03-29T04:18:36.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.332+0000 7f869ffff640 1 -- 172.21.15.22:0/4248672190 learned_addr learned my addr 172.21.15.22:0/4248672190 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:36.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.332+0000 7f869f7fe640 1 --2- 172.21.15.22:0/4248672190 >> v2:172.21.15.151:3300/0 conn(0x7f86a0147d90 0x7f86a00ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.332+0000 7f869ffff640 1 -- 172.21.15.22:0/4248672190 >> v2:172.21.15.151:3300/0 conn(0x7f86a0147d90 msgr2=0x7f86a00ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.332+0000 7f869ffff640 1 --2- 172.21.15.22:0/4248672190 >> v2:172.21.15.151:3300/0 conn(0x7f86a0147d90 0x7f86a00ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.332+0000 7f869ffff640 1 -- 172.21.15.22:0/4248672190 >> v1:172.21.15.151:6789/0 conn(0x7f86a00ca830 legacy=0x7f86a00ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.332+0000 7f869ffff640 1 -- 172.21.15.22:0/4248672190 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a006ef20 con 0x7f86a01479b0 2024-03-29T04:18:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.332+0000 7f869effd640 1 -- 172.21.15.22:0/4248672190 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 162259764 0 0) 0x7f86a006e960 con 0x7f86a00ca830 2024-03-29T04:18:36.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.336+0000 7f869ffff640 1 --2- 172.21.15.22:0/4248672190 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86a01479b0 0x7f86a006f720 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8698009e80 tx=0x7f869801ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59f2a0212814d10c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:36.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.336+0000 7f869effd640 1 -- 172.21.15.22:0/4248672190 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86980a56e0 con 0x7f86a01479b0 2024-03-29T04:18:36.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.336+0000 7f869effd640 1 -- 172.21.15.22:0/4248672190 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86980a5880 con 0x7f86a01479b0 2024-03-29T04:18:36.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.336+0000 7f869effd640 1 -- 172.21.15.22:0/4248672190 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86980a5c50 con 0x7f86a01479b0 2024-03-29T04:18:36.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.336+0000 7f86a6271640 1 -- 172.21.15.22:0/4248672190 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86a01479b0 msgr2=0x7f86a006f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:36.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.336+0000 7f86a6271640 1 --2- 172.21.15.22:0/4248672190 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86a01479b0 0x7f86a006f720 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8698009e80 tx=0x7f869801ca10 comp rx=0 tx=0).stop 2024-03-29T04:18:36.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.336+0000 7f86a6271640 1 -- 172.21.15.22:0/4248672190 shutdown_connections 2024-03-29T04:18:36.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.336+0000 7f86a6271640 1 --2- 172.21.15.22:0/4248672190 >> v2:172.21.15.151:3300/0 conn(0x7f86a0147d90 0x7f86a00ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.336+0000 7f86a6271640 1 --2- 172.21.15.22:0/4248672190 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86a01479b0 0x7f86a006f720 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.336+0000 7f86a6271640 1 -- 172.21.15.22:0/4248672190 >> 172.21.15.22:0/4248672190 conn(0x7f86a00b2c70 msgr2=0x7f86a00b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:36.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.336+0000 7f86a6271640 1 -- 172.21.15.22:0/4248672190 shutdown_connections 2024-03-29T04:18:36.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.336+0000 7f86a6271640 1 -- 172.21.15.22:0/4248672190 wait complete. 2024-03-29T04:18:36.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.336+0000 7f86a6271640 1 Processor -- start 2024-03-29T04:18:36.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.336+0000 7f86a6271640 1 -- start start 2024-03-29T04:18:36.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.336+0000 7f86a6271640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86a00ca830 0x7f86a013fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:36.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.336+0000 7f86a6271640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86a01479b0 0x7f86a00b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:36.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.336+0000 7f86a6271640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86a0147d90 0x7f86a01466a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:36.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.336+0000 7f86a6271640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f86a01606f0 con 0x7f86a0147d90 2024-03-29T04:18:36.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.336+0000 7f86a6271640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f86a0160570 con 0x7f86a01479b0 2024-03-29T04:18:36.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.336+0000 7f86a6271640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f86a00cac80 con 0x7f86a00ca830 2024-03-29T04:18:36.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.340+0000 7f86a480e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86a0147d90 0x7f86a01466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:36.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.340+0000 7f86a480e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86a0147d90 0x7f86a01466a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51096/0 (socket says 172.21.15.22:51096) 2024-03-29T04:18:36.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.340+0000 7f86a480e640 1 -- 172.21.15.22:0/4206412722 learned_addr learned my addr 172.21.15.22:0/4206412722 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:36.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.340+0000 7f869ffff640 1 --2- 172.21.15.22:0/4206412722 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86a00ca830 0x7f86a013fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:36.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.340+0000 7f869f7fe640 1 --2- 172.21.15.22:0/4206412722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86a01479b0 0x7f86a00b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:36.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.340+0000 7f86a480e640 1 -- 172.21.15.22:0/4206412722 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86a00ca830 msgr2=0x7f86a013fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:36.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.340+0000 7f86a480e640 1 --2- 172.21.15.22:0/4206412722 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86a00ca830 0x7f86a013fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.340+0000 7f86a480e640 1 -- 172.21.15.22:0/4206412722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86a01479b0 msgr2=0x7f86a00b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:36.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.340+0000 7f86a480e640 1 --2- 172.21.15.22:0/4206412722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86a01479b0 0x7f86a00b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.340+0000 7f86a480e640 1 -- 172.21.15.22:0/4206412722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a0140070 con 0x7f86a0147d90 2024-03-29T04:18:36.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.340+0000 7f86a480e640 1 --2- 172.21.15.22:0/4206412722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86a0147d90 0x7f86a01466a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f869000c440 tx=0x7f869000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:36.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.340+0000 7f869cff9640 1 -- 172.21.15.22:0/4206412722 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8690017070 con 0x7f86a0147d90 2024-03-29T04:18:36.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.340+0000 7f869cff9640 1 -- 172.21.15.22:0/4206412722 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8690003600 con 0x7f86a0147d90 2024-03-29T04:18:36.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.340+0000 7f86a6271640 1 -- 172.21.15.22:0/4206412722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a0140260 con 0x7f86a0147d90 2024-03-29T04:18:36.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.340+0000 7f869cff9640 1 -- 172.21.15.22:0/4206412722 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86900039f0 con 0x7f86a0147d90 2024-03-29T04:18:36.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.340+0000 7f86a6271640 1 -- 172.21.15.22:0/4206412722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a0140820 con 0x7f86a0147d90 2024-03-29T04:18:36.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.340+0000 7f86a6271640 1 -- 172.21.15.22:0/4206412722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86a0068900 con 0x7f86a0147d90 2024-03-29T04:18:36.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.340+0000 7f869cff9640 1 -- 172.21.15.22:0/4206412722 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f869001a070 con 0x7f86a0147d90 2024-03-29T04:18:36.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.340+0000 7f869cff9640 1 --2- 172.21.15.22:0/4206412722 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8680067fc0 0x7f868006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:36.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.340+0000 7f869cff9640 1 -- 172.21.15.22:0/4206412722 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f8690085fa0 con 0x7f86a0147d90 2024-03-29T04:18:36.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.340+0000 7f869ffff640 1 --2- 172.21.15.22:0/4206412722 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8680067fc0 0x7f868006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:36.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.344+0000 7f869ffff640 1 --2- 172.21.15.22:0/4206412722 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8680067fc0 0x7f868006a480 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f86980ae000 tx=0x7f86980a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:36.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.344+0000 7f869cff9640 1 -- 172.21.15.22:0/4206412722 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f8690021020 con 0x7f86a0147d90 2024-03-29T04:18:36.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.508+0000 7f86a6271640 1 -- 172.21.15.22:0/4206412722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f86a006f7b0 con 0x7f86a0147d90 2024-03-29T04:18:36.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.508+0000 7f869cff9640 1 -- 172.21.15.22:0/4206412722 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f8690055de0 con 0x7f86a0147d90 2024-03-29T04:18:36.489 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:36.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.512+0000 7f86a6271640 1 -- 172.21.15.22:0/4206412722 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8680067fc0 msgr2=0x7f868006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:36.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.512+0000 7f86a6271640 1 --2- 172.21.15.22:0/4206412722 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8680067fc0 0x7f868006a480 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f86980ae000 tx=0x7f86980a7070 comp rx=0 tx=0).stop 2024-03-29T04:18:36.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.512+0000 7f86a6271640 1 -- 172.21.15.22:0/4206412722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86a0147d90 msgr2=0x7f86a01466a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:36.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.512+0000 7f86a6271640 1 --2- 172.21.15.22:0/4206412722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86a0147d90 0x7f86a01466a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f869000c440 tx=0x7f869000c840 comp rx=0 tx=0).stop 2024-03-29T04:18:36.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.512+0000 7f86a6271640 1 -- 172.21.15.22:0/4206412722 shutdown_connections 2024-03-29T04:18:36.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.512+0000 7f86a6271640 1 --2- 172.21.15.22:0/4206412722 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8680067fc0 0x7f868006a480 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.512+0000 7f86a6271640 1 --2- 172.21.15.22:0/4206412722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86a0147d90 0x7f86a01466a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.512+0000 7f86a6271640 1 --2- 172.21.15.22:0/4206412722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86a01479b0 0x7f86a00b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.512+0000 7f86a6271640 1 --2- 172.21.15.22:0/4206412722 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f86a00ca830 0x7f86a013fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.512+0000 7f86a6271640 1 -- 172.21.15.22:0/4206412722 >> 172.21.15.22:0/4206412722 conn(0x7f86a00b2c70 msgr2=0x7f86a00b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:36.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.512+0000 7f86a6271640 1 -- 172.21.15.22:0/4206412722 shutdown_connections 2024-03-29T04:18:36.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.512+0000 7f86a6271640 1 -- 172.21.15.22:0/4206412722 wait complete. 2024-03-29T04:18:36.510 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:18:36.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7fa000242640 1 Processor -- start 2024-03-29T04:18:36.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7fa000242640 1 -- start start 2024-03-29T04:18:36.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7fa000242640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ff80b46f0 0x7f9ff806f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:36.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7fa000242640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ff80b4ba0 con 0x7f9ff806ff20 2024-03-29T04:18:36.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7fa000242640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ff815ec60 con 0x7f9ff815afc0 2024-03-29T04:18:36.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7fa000242640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff8070c30 con 0x7f9ff80b46f0 2024-03-29T04:18:36.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7f9ffe7df640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f9ff806ff20 0x7f9ff80ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58478/0 (socket says 172.21.15.22:58478) 2024-03-29T04:18:36.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7f9ffe7df640 1 -- 172.21.15.22:0/2601901103 learned_addr learned my addr 172.21.15.22:0/2601901103 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:36.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7f9ffd7dd640 1 --2- 172.21.15.22:0/2601901103 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ff80b46f0 0x7f9ff806f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:36.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7f9ffcfdc640 1 -- 172.21.15.22:0/2601901103 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2395377077 0 0) 0x7f9ff80b4ba0 con 0x7f9ff806ff20 2024-03-29T04:18:36.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7f9ffcfdc640 1 -- 172.21.15.22:0/2601901103 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fe4003530 con 0x7f9ff806ff20 2024-03-29T04:18:36.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7f9ffcfdc640 1 -- 172.21.15.22:0/2601901103 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2319052606 0 0) 0x7f9fe4003530 con 0x7f9ff806ff20 2024-03-29T04:18:36.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7f9ffcfdc640 1 -- 172.21.15.22:0/2601901103 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ff80b46f0 msgr2=0x7f9ff806f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:36.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7f9ffcfdc640 1 --2- 172.21.15.22:0/2601901103 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ff80b46f0 0x7f9ff806f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7f9ffcfdc640 1 -- 172.21.15.22:0/2601901103 >> v1:172.21.15.151:6789/0 conn(0x7f9ff815afc0 legacy=0x7f9ff80b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:36.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7f9ffcfdc640 1 -- 172.21.15.22:0/2601901103 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff815ef40 con 0x7f9ff806ff20 2024-03-29T04:18:36.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7f9ffcfdc640 1 -- 172.21.15.22:0/2601901103 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f9ff4001680 con 0x7f9ff806ff20 2024-03-29T04:18:36.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7f9ffcfdc640 1 -- 172.21.15.22:0/2601901103 >> v1:172.21.15.22:6789/0 conn(0x7f9ff806ff20 legacy=0x7f9ff80ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:36.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7f9ffcfdc640 1 --2- 172.21.15.22:0/2601901103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fe40039f0 0x7f9fe4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:36.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7f9ffcfdc640 1 --2- 172.21.15.22:0/2601901103 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ff80b46f0 0x7f9fe4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:36.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7f9ffcfdc640 1 --2- 172.21.15.22:0/2601901103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fe4004d50 0x7f9fe40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:36.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7f9ffcfdc640 1 -- 172.21.15.22:0/2601901103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe400b9d0 con 0x7f9fe40039f0 2024-03-29T04:18:36.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7f9ffcfdc640 1 -- 172.21.15.22:0/2601901103 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe400bb50 con 0x7f9fe4004d50 2024-03-29T04:18:36.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7f9ffcfdc640 1 -- 172.21.15.22:0/2601901103 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe400bd50 con 0x7f9ff80b46f0 2024-03-29T04:18:36.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.652+0000 7f9ffdfde640 1 --2- 172.21.15.22:0/2601901103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fe40039f0 0x7f9fe4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:36.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.656+0000 7f9ffe7df640 1 --2- 172.21.15.22:0/2601901103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fe4004d50 0x7f9fe40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:36.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.656+0000 7f9ffd7dd640 1 --2- 172.21.15.22:0/2601901103 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ff80b46f0 0x7f9fe4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:36.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.656+0000 7f9ffdfde640 1 -- 172.21.15.22:0/2601901103 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ff80b46f0 msgr2=0x7f9fe4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:36.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.656+0000 7f9ffdfde640 1 --2- 172.21.15.22:0/2601901103 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ff80b46f0 0x7f9fe4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.656+0000 7f9ffdfde640 1 -- 172.21.15.22:0/2601901103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fe4004d50 msgr2=0x7f9fe40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:36.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.656+0000 7f9ffdfde640 1 --2- 172.21.15.22:0/2601901103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fe4004d50 0x7f9fe40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.656+0000 7f9ffdfde640 1 -- 172.21.15.22:0/2601901103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe4003530 con 0x7f9fe40039f0 2024-03-29T04:18:36.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.656+0000 7f9ffdfde640 1 --2- 172.21.15.22:0/2601901103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fe40039f0 0x7f9fe4003dd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0001f60 tx=0x7f9ff001e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e435c56efb969b4f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:36.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.656+0000 7f9ffcfdc640 1 -- 172.21.15.22:0/2601901103 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0001b60 con 0x7f9fe40039f0 2024-03-29T04:18:36.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.656+0000 7f9ffcfdc640 1 -- 172.21.15.22:0/2601901103 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ff00a86e0 con 0x7f9fe40039f0 2024-03-29T04:18:36.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.656+0000 7f9ffcfdc640 1 -- 172.21.15.22:0/2601901103 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff00a8ad0 con 0x7f9fe40039f0 2024-03-29T04:18:36.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.656+0000 7fa000242640 1 -- 172.21.15.22:0/2601901103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fe40039f0 msgr2=0x7f9fe4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:36.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.656+0000 7fa000242640 1 --2- 172.21.15.22:0/2601901103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fe40039f0 0x7f9fe4003dd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0001f60 tx=0x7f9ff001e9f0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.656+0000 7f9ffe7df640 1 -- 172.21.15.22:0/2601901103 reap_dead start 2024-03-29T04:18:36.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.656+0000 7fa000242640 1 -- 172.21.15.22:0/2601901103 shutdown_connections 2024-03-29T04:18:36.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.656+0000 7fa000242640 1 -- 172.21.15.22:0/2601901103 >> 172.21.15.22:0/2601901103 conn(0x7f9ff80d1070 msgr2=0x7f9ff80d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:36.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.656+0000 7fa000242640 1 -- 172.21.15.22:0/2601901103 shutdown_connections 2024-03-29T04:18:36.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.656+0000 7fa000242640 1 -- 172.21.15.22:0/2601901103 wait complete. 2024-03-29T04:18:36.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7fa000242640 1 Processor -- start 2024-03-29T04:18:36.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7fa000242640 1 -- start start 2024-03-29T04:18:36.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7fa000242640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fe40039f0 0x7f9ff81e9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:36.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7fa000242640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ff81592b0 0x7f9ff81eaa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:36.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7fa000242640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ff81eafa0 0x7f9ff8160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:36.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7fa000242640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff815b520 con 0x7f9fe40039f0 2024-03-29T04:18:36.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7fa000242640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff815b3a0 con 0x7f9ff81592b0 2024-03-29T04:18:36.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7fa000242640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff806b660 con 0x7f9ff81eafa0 2024-03-29T04:18:36.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7f9ffdfde640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fe40039f0 0x7f9ff81e9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:36.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7f9ffdfde640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fe40039f0 0x7f9ff81e9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51122/0 (socket says 172.21.15.22:51122) 2024-03-29T04:18:36.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7f9ffdfde640 1 -- 172.21.15.22:0/3738160634 learned_addr learned my addr 172.21.15.22:0/3738160634 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:36.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7f9ffd7dd640 1 --2- 172.21.15.22:0/3738160634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ff81592b0 0x7f9ff81eaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:36.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7f9ffdfde640 1 -- 172.21.15.22:0/3738160634 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ff81eafa0 msgr2=0x7f9ff8160ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:36.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7f9ffdfde640 1 --2- 172.21.15.22:0/3738160634 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ff81eafa0 0x7f9ff8160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7f9ffdfde640 1 -- 172.21.15.22:0/3738160634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ff81592b0 msgr2=0x7f9ff81eaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:36.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7f9ffdfde640 1 --2- 172.21.15.22:0/3738160634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ff81592b0 0x7f9ff81eaa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7f9ffdfde640 1 -- 172.21.15.22:0/3738160634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff81eb840 con 0x7f9fe40039f0 2024-03-29T04:18:36.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7f9ffd7dd640 1 --2- 172.21.15.22:0/3738160634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ff81592b0 0x7f9ff81eaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:36.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7f9ffdfde640 1 --2- 172.21.15.22:0/3738160634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fe40039f0 0x7f9ff81e9950 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00b2040 tx=0x7f9fe4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:36.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7f9fea7fc640 1 -- 172.21.15.22:0/3738160634 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff00a8ab0 con 0x7f9fe40039f0 2024-03-29T04:18:36.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7f9fea7fc640 1 -- 172.21.15.22:0/3738160634 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ff00bd070 con 0x7f9fe40039f0 2024-03-29T04:18:36.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7fa000242640 1 -- 172.21.15.22:0/3738160634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff81eb560 con 0x7f9fe40039f0 2024-03-29T04:18:36.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7f9fea7fc640 1 -- 172.21.15.22:0/3738160634 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff00a8e50 con 0x7f9fe40039f0 2024-03-29T04:18:36.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.660+0000 7fa000242640 1 -- 172.21.15.22:0/3738160634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff81e9150 con 0x7f9fe40039f0 2024-03-29T04:18:36.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.664+0000 7fa000242640 1 -- 172.21.15.22:0/3738160634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe00052a0 con 0x7f9fe40039f0 2024-03-29T04:18:36.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.664+0000 7f9fea7fc640 1 -- 172.21.15.22:0/3738160634 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9ff0003980 con 0x7f9fe40039f0 2024-03-29T04:18:36.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.664+0000 7f9fea7fc640 1 --2- 172.21.15.22:0/3738160634 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9fdc068160 0x7f9fdc06a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:36.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.664+0000 7f9fea7fc640 1 -- 172.21.15.22:0/3738160634 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f9ff0120ff0 con 0x7f9fe40039f0 2024-03-29T04:18:36.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.664+0000 7f9ffd7dd640 1 --2- 172.21.15.22:0/3738160634 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9fdc068160 0x7f9fdc06a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:36.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.664+0000 7f9ffd7dd640 1 --2- 172.21.15.22:0/3738160634 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9fdc068160 0x7f9fdc06a620 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9ff806c8f0 tx=0x7f9fec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:36.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.668+0000 7f9fea7fc640 1 -- 172.21.15.22:0/3738160634 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9ff0018700 con 0x7f9fe40039f0 2024-03-29T04:18:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.828+0000 7fa000242640 1 -- 172.21.15.22:0/3738160634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9fe0005860 con 0x7f9fe40039f0 2024-03-29T04:18:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.828+0000 7f9fea7fc640 1 -- 172.21.15.22:0/3738160634 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f9ff00f1430 con 0x7f9fe40039f0 2024-03-29T04:18:36.810 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-29T04:18:36.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.832+0000 7fa000242640 1 -- 172.21.15.22:0/3738160634 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9fdc068160 msgr2=0x7f9fdc06a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:36.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.832+0000 7fa000242640 1 --2- 172.21.15.22:0/3738160634 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9fdc068160 0x7f9fdc06a620 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9ff806c8f0 tx=0x7f9fec008040 comp rx=0 tx=0).stop 2024-03-29T04:18:36.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.832+0000 7fa000242640 1 -- 172.21.15.22:0/3738160634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fe40039f0 msgr2=0x7f9ff81e9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:36.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.832+0000 7fa000242640 1 --2- 172.21.15.22:0/3738160634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fe40039f0 0x7f9ff81e9950 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00b2040 tx=0x7f9fe4005840 comp rx=0 tx=0).stop 2024-03-29T04:18:36.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.832+0000 7fa000242640 1 -- 172.21.15.22:0/3738160634 shutdown_connections 2024-03-29T04:18:36.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.832+0000 7fa000242640 1 --2- 172.21.15.22:0/3738160634 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ff81eafa0 0x7f9ff8160ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.832+0000 7fa000242640 1 --2- 172.21.15.22:0/3738160634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ff81592b0 0x7f9ff81eaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.832+0000 7fa000242640 1 --2- 172.21.15.22:0/3738160634 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9fdc068160 0x7f9fdc06a620 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.832+0000 7fa000242640 1 --2- 172.21.15.22:0/3738160634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fe40039f0 0x7f9ff81e9950 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.832+0000 7fa000242640 1 -- 172.21.15.22:0/3738160634 >> 172.21.15.22:0/3738160634 conn(0x7f9ff80d1070 msgr2=0x7f9ff80ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:36.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.832+0000 7fa000242640 1 -- 172.21.15.22:0/3738160634 shutdown_connections 2024-03-29T04:18:36.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.832+0000 7fa000242640 1 -- 172.21.15.22:0/3738160634 wait complete. 2024-03-29T04:18:36.828 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:18:36.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.976+0000 7ff21bd62640 1 Processor -- start 2024-03-29T04:18:36.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.976+0000 7ff21bd62640 1 -- start start 2024-03-29T04:18:36.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.976+0000 7ff21bd62640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff21406cd80 0x7ff214166fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:36.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.976+0000 7ff21bd62640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff21406d710 con 0x7ff21413f140 2024-03-29T04:18:36.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.976+0000 7ff21bd62640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff21406d9f0 con 0x7ff21415ff40 2024-03-29T04:18:36.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.976+0000 7ff21bd62640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7ff21406e320 con 0x7ff21406cd80 2024-03-29T04:18:36.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.976+0000 7ff219afe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff21413f140 0x7ff2141606d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58484/0 (socket says 172.21.15.22:58484) 2024-03-29T04:18:36.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.976+0000 7ff21a2ff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff21406cd80 0x7ff214166fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:36.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.976+0000 7ff219afe640 1 -- 172.21.15.22:0/786976079 learned_addr learned my addr 172.21.15.22:0/786976079 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:36.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.976+0000 7ff21a2ff640 1 -- 172.21.15.22:0/786976079 >> v1:172.21.15.151:6789/0 conn(0x7ff21415ff40 legacy=0x7ff21406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:36.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.976+0000 7ff21a2ff640 1 -- 172.21.15.22:0/786976079 >> v1:172.21.15.22:6789/0 conn(0x7ff21413f140 legacy=0x7ff2141606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:36.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.976+0000 7ff21a2ff640 1 -- 172.21.15.22:0/786976079 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21406dcd0 con 0x7ff21406cd80 2024-03-29T04:18:36.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff21a2ff640 1 --2- 172.21.15.22:0/786976079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff21406cd80 0x7ff214166fe0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009d90 tx=0x7ff20c01c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545dcaab61d3b536 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:36.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff218afc640 1 -- 172.21.15.22:0/786976079 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c0b2070 con 0x7ff21406cd80 2024-03-29T04:18:36.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff218afc640 1 -- 172.21.15.22:0/786976079 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff20c0a5750 con 0x7ff21406cd80 2024-03-29T04:18:36.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff218afc640 1 -- 172.21.15.22:0/786976079 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c0a5b20 con 0x7ff21406cd80 2024-03-29T04:18:36.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff21bd62640 1 -- 172.21.15.22:0/786976079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff21406cd80 msgr2=0x7ff214166fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:36.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff21bd62640 1 --2- 172.21.15.22:0/786976079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff21406cd80 0x7ff214166fe0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009d90 tx=0x7ff20c01c800 comp rx=0 tx=0).stop 2024-03-29T04:18:36.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff21bd62640 1 -- 172.21.15.22:0/786976079 shutdown_connections 2024-03-29T04:18:36.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff21bd62640 1 --2- 172.21.15.22:0/786976079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff21406cd80 0x7ff214166fe0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff21bd62640 1 -- 172.21.15.22:0/786976079 >> 172.21.15.22:0/786976079 conn(0x7ff2140d00a0 msgr2=0x7ff2140d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:36.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff21bd62640 1 -- 172.21.15.22:0/786976079 shutdown_connections 2024-03-29T04:18:36.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff21bd62640 1 -- 172.21.15.22:0/786976079 wait complete. 2024-03-29T04:18:36.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff21bd62640 1 Processor -- start 2024-03-29T04:18:36.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff21bd62640 1 -- start start 2024-03-29T04:18:36.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff21bd62640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff21406cd80 0x7ff214137c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:36.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff21bd62640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff21413f140 0x7ff214155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:36.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff21bd62640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff21415ff40 0x7ff2141365b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:36.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff21bd62640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff21416a0d0 con 0x7ff21406cd80 2024-03-29T04:18:36.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff21bd62640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff214169f50 con 0x7ff21413f140 2024-03-29T04:18:36.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff21bd62640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff21416a250 con 0x7ff21415ff40 2024-03-29T04:18:36.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff219afe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff21406cd80 0x7ff214137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:36.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff219afe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff21406cd80 0x7ff214137c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51144/0 (socket says 172.21.15.22:51144) 2024-03-29T04:18:36.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff219afe640 1 -- 172.21.15.22:0/379086064 learned_addr learned my addr 172.21.15.22:0/379086064 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:36.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.980+0000 7ff2192fd640 1 --2- 172.21.15.22:0/379086064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff21413f140 0x7ff214155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:36.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.984+0000 7ff21a2ff640 1 --2- 172.21.15.22:0/379086064 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff21415ff40 0x7ff2141365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:36.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.984+0000 7ff219afe640 1 -- 172.21.15.22:0/379086064 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff21415ff40 msgr2=0x7ff2141365b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:36.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.984+0000 7ff219afe640 1 --2- 172.21.15.22:0/379086064 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff21415ff40 0x7ff2141365b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.984+0000 7ff219afe640 1 -- 172.21.15.22:0/379086064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff21413f140 msgr2=0x7ff214155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:36.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.984+0000 7ff219afe640 1 --2- 172.21.15.22:0/379086064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff21413f140 0x7ff214155630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:36.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.984+0000 7ff219afe640 1 -- 172.21.15.22:0/379086064 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff214136af0 con 0x7ff21406cd80 2024-03-29T04:18:36.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.984+0000 7ff21a2ff640 1 --2- 172.21.15.22:0/379086064 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff21415ff40 0x7ff2141365b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:36.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.984+0000 7ff219afe640 1 --2- 172.21.15.22:0/379086064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff21406cd80 0x7ff214137c30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff21000c7b0 tx=0x7ff21000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:36.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.984+0000 7ff20a7fc640 1 -- 172.21.15.22:0/379086064 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210018070 con 0x7ff21406cd80 2024-03-29T04:18:36.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.984+0000 7ff20a7fc640 1 -- 172.21.15.22:0/379086064 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2100074e0 con 0x7ff21406cd80 2024-03-29T04:18:36.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.984+0000 7ff20a7fc640 1 -- 172.21.15.22:0/379086064 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21000ce60 con 0x7ff21406cd80 2024-03-29T04:18:36.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.984+0000 7ff21bd62640 1 -- 172.21.15.22:0/379086064 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff214072d10 con 0x7ff21406cd80 2024-03-29T04:18:36.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.984+0000 7ff21bd62640 1 -- 172.21.15.22:0/379086064 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff214072a30 con 0x7ff21406cd80 2024-03-29T04:18:36.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.984+0000 7ff21bd62640 1 -- 172.21.15.22:0/379086064 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff214068900 con 0x7ff21406cd80 2024-03-29T04:18:36.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.984+0000 7ff20a7fc640 1 -- 172.21.15.22:0/379086064 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff21001b070 con 0x7ff21406cd80 2024-03-29T04:18:36.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.984+0000 7ff20a7fc640 1 --2- 172.21.15.22:0/379086064 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff1f8067ff0 0x7ff1f806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:36.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.984+0000 7ff20a7fc640 1 -- 172.21.15.22:0/379086064 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7ff210085e30 con 0x7ff21406cd80 2024-03-29T04:18:36.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.984+0000 7ff2192fd640 1 --2- 172.21.15.22:0/379086064 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff1f8067ff0 0x7ff1f806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:36.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.984+0000 7ff2192fd640 1 --2- 172.21.15.22:0/379086064 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff1f8067ff0 0x7ff1f806a4b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff2140cd5c0 tx=0x7ff204009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:36.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:36.988+0000 7ff20a7fc640 1 -- 172.21.15.22:0/379086064 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff214136af0 con 0x7ff21406cd80 2024-03-29T04:18:37.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.152+0000 7ff21bd62640 1 -- 172.21.15.22:0/379086064 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff21413d2d0 con 0x7ff21406cd80 2024-03-29T04:18:37.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.152+0000 7ff20a7fc640 1 -- 172.21.15.22:0/379086064 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7ff2100561e0 con 0x7ff21406cd80 2024-03-29T04:18:37.133 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:37.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.156+0000 7ff21bd62640 1 -- 172.21.15.22:0/379086064 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff1f8067ff0 msgr2=0x7ff1f806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:37.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.156+0000 7ff21bd62640 1 --2- 172.21.15.22:0/379086064 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff1f8067ff0 0x7ff1f806a4b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff2140cd5c0 tx=0x7ff204009040 comp rx=0 tx=0).stop 2024-03-29T04:18:37.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.156+0000 7ff21bd62640 1 -- 172.21.15.22:0/379086064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff21406cd80 msgr2=0x7ff214137c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:37.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.156+0000 7ff21bd62640 1 --2- 172.21.15.22:0/379086064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff21406cd80 0x7ff214137c30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff21000c7b0 tx=0x7ff21000cc70 comp rx=0 tx=0).stop 2024-03-29T04:18:37.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.156+0000 7ff21bd62640 1 -- 172.21.15.22:0/379086064 shutdown_connections 2024-03-29T04:18:37.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.156+0000 7ff21bd62640 1 --2- 172.21.15.22:0/379086064 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff21415ff40 0x7ff2141365b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.156+0000 7ff21bd62640 1 --2- 172.21.15.22:0/379086064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff21413f140 0x7ff214155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.156+0000 7ff21bd62640 1 --2- 172.21.15.22:0/379086064 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff1f8067ff0 0x7ff1f806a4b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.156+0000 7ff21bd62640 1 --2- 172.21.15.22:0/379086064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff21406cd80 0x7ff214137c30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.156+0000 7ff21bd62640 1 -- 172.21.15.22:0/379086064 >> 172.21.15.22:0/379086064 conn(0x7ff2140d00a0 msgr2=0x7ff2140af820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:37.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.156+0000 7ff21bd62640 1 -- 172.21.15.22:0/379086064 shutdown_connections 2024-03-29T04:18:37.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.156+0000 7ff21bd62640 1 -- 172.21.15.22:0/379086064 wait complete. 2024-03-29T04:18:37.154 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:18:37.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.300+0000 7f0943f6a640 1 Processor -- start 2024-03-29T04:18:37.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.300+0000 7f0943f6a640 1 -- start start 2024-03-29T04:18:37.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.300+0000 7f0943f6a640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f093c0cef80 0x7f093c0b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:37.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.300+0000 7f0943f6a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f093c0cf360 0x7f093c0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:37.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.300+0000 7f0943f6a640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f093c0b4500 con 0x7f093c0b3840 2024-03-29T04:18:37.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.300+0000 7f0943f6a640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f093c0b47e0 con 0x7f093c0cf360 2024-03-29T04:18:37.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.300+0000 7f0943f6a640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f093c0b4ac0 con 0x7f093c0cef80 2024-03-29T04:18:37.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.300+0000 7f0941505640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f093c0cf360 0x7f093c0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:37.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.300+0000 7f0941505640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f093c0cf360 0x7f093c0cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51150/0 (socket says 172.21.15.22:51150) 2024-03-29T04:18:37.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.300+0000 7f0941505640 1 -- 172.21.15.22:0/3268014705 learned_addr learned my addr 172.21.15.22:0/3268014705 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:37.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.300+0000 7f0941d06640 1 --2- 172.21.15.22:0/3268014705 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f093c0cef80 0x7f093c0b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:37.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.300+0000 7f0941505640 1 -- 172.21.15.22:0/3268014705 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f093c0cef80 msgr2=0x7f093c0b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:37.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.300+0000 7f0941505640 1 --2- 172.21.15.22:0/3268014705 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f093c0cef80 0x7f093c0b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.300+0000 7f0941505640 1 -- 172.21.15.22:0/3268014705 >> v1:172.21.15.151:6789/0 conn(0x7f093c0b3840 legacy=0x7f093c0b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:37.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.300+0000 7f0941505640 1 -- 172.21.15.22:0/3268014705 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093c0b4da0 con 0x7f093c0cf360 2024-03-29T04:18:37.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0941505640 1 --2- 172.21.15.22:0/3268014705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f093c0cf360 0x7f093c0cf7c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0938009870 tx=0x7f093801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d663d7348fff7f13 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:37.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0940d04640 1 -- 172.21.15.22:0/3268014705 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09380b2070 con 0x7f093c0cf360 2024-03-29T04:18:37.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0940d04640 1 -- 172.21.15.22:0/3268014705 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09380a9070 con 0x7f093c0cf360 2024-03-29T04:18:37.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0940d04640 1 -- 172.21.15.22:0/3268014705 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09380ad880 con 0x7f093c0cf360 2024-03-29T04:18:37.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0943f6a640 1 -- 172.21.15.22:0/3268014705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f093c0cf360 msgr2=0x7f093c0cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:37.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0943f6a640 1 --2- 172.21.15.22:0/3268014705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f093c0cf360 0x7f093c0cf7c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0938009870 tx=0x7f093801c560 comp rx=0 tx=0).stop 2024-03-29T04:18:37.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0943f6a640 1 -- 172.21.15.22:0/3268014705 shutdown_connections 2024-03-29T04:18:37.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0943f6a640 1 --2- 172.21.15.22:0/3268014705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f093c0cf360 0x7f093c0cf7c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0943f6a640 1 --2- 172.21.15.22:0/3268014705 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f093c0cef80 0x7f093c0b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0943f6a640 1 -- 172.21.15.22:0/3268014705 >> 172.21.15.22:0/3268014705 conn(0x7f093c0d4310 msgr2=0x7f093c0d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:37.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0943f6a640 1 -- 172.21.15.22:0/3268014705 shutdown_connections 2024-03-29T04:18:37.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0943f6a640 1 -- 172.21.15.22:0/3268014705 wait complete. 2024-03-29T04:18:37.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0943f6a640 1 Processor -- start 2024-03-29T04:18:37.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0943f6a640 1 -- start start 2024-03-29T04:18:37.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0943f6a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f093c0b3840 0x7f093c15dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:37.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0943f6a640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f093c0cef80 0x7f093c0cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:37.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0943f6a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f093c0cf360 0x7f093c14f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:37.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0943f6a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f093c16aeb0 con 0x7f093c0b3840 2024-03-29T04:18:37.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0943f6a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f093c16ad30 con 0x7f093c0cf360 2024-03-29T04:18:37.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0943f6a640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f093c16b030 con 0x7f093c0cef80 2024-03-29T04:18:37.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0941d06640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f093c0b3840 0x7f093c15dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:37.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0941d06640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f093c0b3840 0x7f093c15dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51158/0 (socket says 172.21.15.22:51158) 2024-03-29T04:18:37.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0941d06640 1 -- 172.21.15.22:0/1809932505 learned_addr learned my addr 172.21.15.22:0/1809932505 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:37.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0942507640 1 --2- 172.21.15.22:0/1809932505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f093c0cf360 0x7f093c14f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:37.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0941505640 1 --2- 172.21.15.22:0/1809932505 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f093c0cef80 0x7f093c0cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:37.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.304+0000 7f0941d06640 1 -- 172.21.15.22:0/1809932505 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f093c0cef80 msgr2=0x7f093c0cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:37.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.308+0000 7f0941d06640 1 --2- 172.21.15.22:0/1809932505 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f093c0cef80 0x7f093c0cdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.308+0000 7f0941d06640 1 -- 172.21.15.22:0/1809932505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f093c0cf360 msgr2=0x7f093c14f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:37.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.308+0000 7f0941d06640 1 --2- 172.21.15.22:0/1809932505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f093c0cf360 0x7f093c14f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.308+0000 7f0941d06640 1 -- 172.21.15.22:0/1809932505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093c0cc6d0 con 0x7f093c0b3840 2024-03-29T04:18:37.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.308+0000 7f0941505640 1 --2- 172.21.15.22:0/1809932505 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f093c0cef80 0x7f093c0cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:37.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.308+0000 7f0941d06640 1 --2- 172.21.15.22:0/1809932505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f093c0b3840 0x7f093c15dab0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f092c00d800 tx=0x7f092c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:37.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.308+0000 7f09327fc640 1 -- 172.21.15.22:0/1809932505 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c00b6c0 con 0x7f093c0b3840 2024-03-29T04:18:37.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.308+0000 7f09327fc640 1 -- 172.21.15.22:0/1809932505 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f092c019070 con 0x7f093c0b3840 2024-03-29T04:18:37.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.308+0000 7f0943f6a640 1 -- 172.21.15.22:0/1809932505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f093c0cc3f0 con 0x7f093c0b3840 2024-03-29T04:18:37.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.308+0000 7f09327fc640 1 -- 172.21.15.22:0/1809932505 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c00bae0 con 0x7f093c0b3840 2024-03-29T04:18:37.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.308+0000 7f0943f6a640 1 -- 172.21.15.22:0/1809932505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f093c15d2b0 con 0x7f093c0b3840 2024-03-29T04:18:37.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.308+0000 7f0943f6a640 1 -- 172.21.15.22:0/1809932505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f093c068890 con 0x7f093c0b3840 2024-03-29T04:18:37.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.308+0000 7f09327fc640 1 -- 172.21.15.22:0/1809932505 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f092c00bc80 con 0x7f093c0b3840 2024-03-29T04:18:37.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.308+0000 7f09327fc640 1 --2- 172.21.15.22:0/1809932505 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0920067fc0 0x7f092006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:37.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.308+0000 7f09327fc640 1 -- 172.21.15.22:0/1809932505 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f092c086db0 con 0x7f093c0b3840 2024-03-29T04:18:37.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.308+0000 7f0941505640 1 --2- 172.21.15.22:0/1809932505 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0920067fc0 0x7f092006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:37.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.308+0000 7f0941505640 1 --2- 172.21.15.22:0/1809932505 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0920067fc0 0x7f092006a480 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f09380af000 tx=0x7f0938008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:37.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.312+0000 7f09327fc640 1 -- 172.21.15.22:0/1809932505 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f092c057160 con 0x7f093c0b3840 2024-03-29T04:18:37.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.484+0000 7f0943f6a640 1 -- 172.21.15.22:0/1809932505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f093c0b3f00 con 0x7f093c0b3840 2024-03-29T04:18:37.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.484+0000 7f09327fc640 1 -- 172.21.15.22:0/1809932505 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f092c056a60 con 0x7f093c0b3840 2024-03-29T04:18:37.467 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-03-29T04:18:37.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.488+0000 7f0943f6a640 1 -- 172.21.15.22:0/1809932505 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0920067fc0 msgr2=0x7f092006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:37.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.488+0000 7f0943f6a640 1 --2- 172.21.15.22:0/1809932505 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0920067fc0 0x7f092006a480 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f09380af000 tx=0x7f0938008040 comp rx=0 tx=0).stop 2024-03-29T04:18:37.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.488+0000 7f0943f6a640 1 -- 172.21.15.22:0/1809932505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f093c0b3840 msgr2=0x7f093c15dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:37.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.488+0000 7f0943f6a640 1 --2- 172.21.15.22:0/1809932505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f093c0b3840 0x7f093c15dab0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f092c00d800 tx=0x7f092c00dcc0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.488+0000 7f0943f6a640 1 -- 172.21.15.22:0/1809932505 shutdown_connections 2024-03-29T04:18:37.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.488+0000 7f0943f6a640 1 --2- 172.21.15.22:0/1809932505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f093c0cf360 0x7f093c14f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.488+0000 7f0943f6a640 1 --2- 172.21.15.22:0/1809932505 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f093c0cef80 0x7f093c0cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.488+0000 7f0943f6a640 1 --2- 172.21.15.22:0/1809932505 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0920067fc0 0x7f092006a480 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.488+0000 7f0943f6a640 1 --2- 172.21.15.22:0/1809932505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f093c0b3840 0x7f093c15dab0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.488+0000 7f0943f6a640 1 -- 172.21.15.22:0/1809932505 >> 172.21.15.22:0/1809932505 conn(0x7f093c0d4310 msgr2=0x7f093c0b97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:37.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.488+0000 7f0943f6a640 1 -- 172.21.15.22:0/1809932505 shutdown_connections 2024-03-29T04:18:37.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.488+0000 7f0943f6a640 1 -- 172.21.15.22:0/1809932505 wait complete. 2024-03-29T04:18:37.485 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:18:37.485 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:18:37.491 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:37.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92ccecb640 1 Processor -- start 2024-03-29T04:18:37.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92ccecb640 1 -- start start 2024-03-29T04:18:37.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92ccecb640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f92c815eb10 0x7f92c81667a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:37.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92ccecb640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92c8072cb0 con 0x7f92c80b3f00 2024-03-29T04:18:37.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92ccecb640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92c8072f90 con 0x7f92c815a840 2024-03-29T04:18:37.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92ccecb640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f92c806cd80 con 0x7f92c815eb10 2024-03-29T04:18:37.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92bffff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f92c80b3f00 0x7f92c815e400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58494/0 (socket says 172.21.15.22:58494) 2024-03-29T04:18:37.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92bffff640 1 -- 172.21.15.22:0/19934460 learned_addr learned my addr 172.21.15.22:0/19934460 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:37.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c6d9d640 1 --2- 172.21.15.22:0/19934460 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f92c815eb10 0x7f92c81667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:37.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c5d9b640 1 -- 172.21.15.22:0/19934460 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2697337643 0 0) 0x7f92c8072cb0 con 0x7f92c80b3f00 2024-03-29T04:18:37.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c5d9b640 1 -- 172.21.15.22:0/19934460 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92b4003530 con 0x7f92c80b3f00 2024-03-29T04:18:37.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c5d9b640 1 -- 172.21.15.22:0/19934460 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3716202578 0 0) 0x7f92b4003530 con 0x7f92c80b3f00 2024-03-29T04:18:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c5d9b640 1 -- 172.21.15.22:0/19934460 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f92c815eb10 msgr2=0x7f92c81667a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c5d9b640 1 --2- 172.21.15.22:0/19934460 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f92c815eb10 0x7f92c81667a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c5d9b640 1 -- 172.21.15.22:0/19934460 >> v1:172.21.15.151:6789/0 conn(0x7f92c815a840 legacy=0x7f92c806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c5d9b640 1 -- 172.21.15.22:0/19934460 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c8073270 con 0x7f92c80b3f00 2024-03-29T04:18:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c5d9b640 1 -- 172.21.15.22:0/19934460 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f92b00016a0 con 0x7f92c80b3f00 2024-03-29T04:18:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c5d9b640 1 -- 172.21.15.22:0/19934460 >> v1:172.21.15.22:6789/0 conn(0x7f92c80b3f00 legacy=0x7f92c815e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c5d9b640 1 --2- 172.21.15.22:0/19934460 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f92c815eb10 0x7f92b40039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c5d9b640 1 --2- 172.21.15.22:0/19934460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92b4003f30 0x7f92b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c5d9b640 1 --2- 172.21.15.22:0/19934460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92b4004d50 0x7f92b40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c5d9b640 1 -- 172.21.15.22:0/19934460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f92c806cd80 con 0x7f92b4004d50 2024-03-29T04:18:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c5d9b640 1 -- 172.21.15.22:0/19934460 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f92b400bae0 con 0x7f92b4003f30 2024-03-29T04:18:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c5d9b640 1 -- 172.21.15.22:0/19934460 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f92b400bce0 con 0x7f92c815eb10 2024-03-29T04:18:37.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c6d9d640 1 --2- 172.21.15.22:0/19934460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92b4004d50 0x7f92b40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:37.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92bffff640 1 --2- 172.21.15.22:0/19934460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92b4003f30 0x7f92b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:37.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c659c640 1 --2- 172.21.15.22:0/19934460 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f92c815eb10 0x7f92b40039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:37.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c6d9d640 1 -- 172.21.15.22:0/19934460 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f92c815eb10 msgr2=0x7f92b40039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:37.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c6d9d640 1 --2- 172.21.15.22:0/19934460 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f92c815eb10 0x7f92b40039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c6d9d640 1 -- 172.21.15.22:0/19934460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92b4003f30 msgr2=0x7f92b4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:37.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c6d9d640 1 --2- 172.21.15.22:0/19934460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92b4003f30 0x7f92b4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.668+0000 7f92c6d9d640 1 -- 172.21.15.22:0/19934460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b4003530 con 0x7f92b4004d50 2024-03-29T04:18:37.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.672+0000 7f92c6d9d640 1 --2- 172.21.15.22:0/19934460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92b4004d50 0x7f92b40051d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f92c8071310 tx=0x7f92c001cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6503af0895a3a1d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:37.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.672+0000 7f92c5d9b640 1 -- 172.21.15.22:0/19934460 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c00b2070 con 0x7f92b4004d50 2024-03-29T04:18:37.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.672+0000 7f92c5d9b640 1 -- 172.21.15.22:0/19934460 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92c00a5cd0 con 0x7f92b4004d50 2024-03-29T04:18:37.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.672+0000 7f92c5d9b640 1 -- 172.21.15.22:0/19934460 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0009bf0 con 0x7f92b4004d50 2024-03-29T04:18:37.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.672+0000 7f92ccecb640 1 -- 172.21.15.22:0/19934460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92b4004d50 msgr2=0x7f92b40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:37.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.672+0000 7f92ccecb640 1 --2- 172.21.15.22:0/19934460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92b4004d50 0x7f92b40051d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f92c8071310 tx=0x7f92c001cc00 comp rx=0 tx=0).stop 2024-03-29T04:18:37.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.672+0000 7f92c6d9d640 1 -- 172.21.15.22:0/19934460 reap_dead start 2024-03-29T04:18:37.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.672+0000 7f92ccecb640 1 -- 172.21.15.22:0/19934460 shutdown_connections 2024-03-29T04:18:37.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.672+0000 7f92ccecb640 1 -- 172.21.15.22:0/19934460 >> 172.21.15.22:0/19934460 conn(0x7f92c80cfee0 msgr2=0x7f92c80d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:37.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.672+0000 7f92ccecb640 1 -- 172.21.15.22:0/19934460 shutdown_connections 2024-03-29T04:18:37.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.672+0000 7f92ccecb640 1 -- 172.21.15.22:0/19934460 wait complete. 2024-03-29T04:18:37.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.672+0000 7f92ccecb640 1 Processor -- start 2024-03-29T04:18:37.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.672+0000 7f92ccecb640 1 -- start start 2024-03-29T04:18:37.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92ccecb640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92c814bee0 0x7f92c815ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:37.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92ccecb640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f92c8153b90 0x7f92c81f8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:37.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92ccecb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92c81f85b0 0x7f92c806da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:37.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92ccecb640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f92c80b4360 con 0x7f92c81f85b0 2024-03-29T04:18:37.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92ccecb640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f92c815ac20 con 0x7f92c814bee0 2024-03-29T04:18:37.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92ccecb640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f92c81f7900 con 0x7f92c8153b90 2024-03-29T04:18:37.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92c6d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92c81f85b0 0x7f92c806da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:37.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92c6d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92c81f85b0 0x7f92c806da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51178/0 (socket says 172.21.15.22:51178) 2024-03-29T04:18:37.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92c6d9d640 1 -- 172.21.15.22:0/3205354481 learned_addr learned my addr 172.21.15.22:0/3205354481 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:37.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92c659c640 1 --2- 172.21.15.22:0/3205354481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92c814bee0 0x7f92c815ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:37.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92bffff640 1 --2- 172.21.15.22:0/3205354481 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f92c8153b90 0x7f92c81f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:37.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92c6d9d640 1 -- 172.21.15.22:0/3205354481 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f92c8153b90 msgr2=0x7f92c81f8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:37.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92c6d9d640 1 --2- 172.21.15.22:0/3205354481 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f92c8153b90 0x7f92c81f8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92c6d9d640 1 -- 172.21.15.22:0/3205354481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92c814bee0 msgr2=0x7f92c815ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:37.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92c6d9d640 1 --2- 172.21.15.22:0/3205354481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92c814bee0 0x7f92c815ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92c6d9d640 1 -- 172.21.15.22:0/3205354481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c815b410 con 0x7f92c81f85b0 2024-03-29T04:18:37.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92c6d9d640 1 --2- 172.21.15.22:0/3205354481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92c81f85b0 0x7f92c806da50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f92c0002240 tx=0x7f92c0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:37.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92beffd640 1 -- 172.21.15.22:0/3205354481 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c00b2040 con 0x7f92c81f85b0 2024-03-29T04:18:37.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92beffd640 1 -- 172.21.15.22:0/3205354481 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92c00a5ad0 con 0x7f92c81f85b0 2024-03-29T04:18:37.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92ccecb640 1 -- 172.21.15.22:0/3205354481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92c8154010 con 0x7f92c81f85b0 2024-03-29T04:18:37.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92beffd640 1 -- 172.21.15.22:0/3205354481 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0009bf0 con 0x7f92c81f85b0 2024-03-29T04:18:37.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92ccecb640 1 -- 172.21.15.22:0/3205354481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92c80aefa0 con 0x7f92c81f85b0 2024-03-29T04:18:37.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92ccecb640 1 -- 172.21.15.22:0/3205354481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92c8068890 con 0x7f92c81f85b0 2024-03-29T04:18:37.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92beffd640 1 -- 172.21.15.22:0/3205354481 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f92c00c0030 con 0x7f92c81f85b0 2024-03-29T04:18:37.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92beffd640 1 --2- 172.21.15.22:0/3205354481 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f92a8067fc0 0x7f92a806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:37.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92beffd640 1 -- 172.21.15.22:0/3205354481 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f92c0120e30 con 0x7f92c81f85b0 2024-03-29T04:18:37.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.676+0000 7f92c659c640 1 --2- 172.21.15.22:0/3205354481 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f92a8067fc0 0x7f92a806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:37.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.680+0000 7f92c659c640 1 --2- 172.21.15.22:0/3205354481 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f92a8067fc0 0x7f92a806a480 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f92b8013520 tx=0x7f92b800aa70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:37.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.680+0000 7f92beffd640 1 -- 172.21.15.22:0/3205354481 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f92c00c7060 con 0x7f92c81f85b0 2024-03-29T04:18:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.840+0000 7f92ccecb640 1 -- 172.21.15.22:0/3205354481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92c80719d0 con 0x7f92c81f85b0 2024-03-29T04:18:37.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.844+0000 7f92beffd640 1 -- 172.21.15.22:0/3205354481 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18847 (secure 0 0 0) 0x7f92c815b410 con 0x7f92c81f85b0 2024-03-29T04:18:37.824 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:37.824 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:18:16.893439+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:37.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.844+0000 7f92ccecb640 1 -- 172.21.15.22:0/3205354481 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f92a8067fc0 msgr2=0x7f92a806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:37.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.844+0000 7f92ccecb640 1 --2- 172.21.15.22:0/3205354481 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f92a8067fc0 0x7f92a806a480 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f92b8013520 tx=0x7f92b800aa70 comp rx=0 tx=0).stop 2024-03-29T04:18:37.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.844+0000 7f92ccecb640 1 -- 172.21.15.22:0/3205354481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92c81f85b0 msgr2=0x7f92c806da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:37.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.844+0000 7f92ccecb640 1 --2- 172.21.15.22:0/3205354481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92c81f85b0 0x7f92c806da50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f92c0002240 tx=0x7f92c0009750 comp rx=0 tx=0).stop 2024-03-29T04:18:37.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.844+0000 7f92ccecb640 1 -- 172.21.15.22:0/3205354481 shutdown_connections 2024-03-29T04:18:37.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.844+0000 7f92ccecb640 1 --2- 172.21.15.22:0/3205354481 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f92a8067fc0 0x7f92a806a480 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.844+0000 7f92ccecb640 1 --2- 172.21.15.22:0/3205354481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92c81f85b0 0x7f92c806da50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.844+0000 7f92ccecb640 1 --2- 172.21.15.22:0/3205354481 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f92c8153b90 0x7f92c81f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.844+0000 7f92ccecb640 1 --2- 172.21.15.22:0/3205354481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92c814bee0 0x7f92c815ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.844+0000 7f92ccecb640 1 -- 172.21.15.22:0/3205354481 >> 172.21.15.22:0/3205354481 conn(0x7f92c80cfee0 msgr2=0x7f92c806c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:37.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.844+0000 7f92ccecb640 1 -- 172.21.15.22:0/3205354481 shutdown_connections 2024-03-29T04:18:37.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.844+0000 7f92ccecb640 1 -- 172.21.15.22:0/3205354481 wait complete. 2024-03-29T04:18:37.842 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:17:34.034142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:17:34.886659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:17:38.628632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:18:37.842 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:18:37.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.988+0000 7fefdf06e640 1 Processor -- start 2024-03-29T04:18:37.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.988+0000 7fefdf06e640 1 -- start start 2024-03-29T04:18:37.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.988+0000 7fefdf06e640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fefd8159c90 0x7fefd8160970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:37.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.988+0000 7fefdf06e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fefd815d6d0 0x7fefd80700f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:37.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.988+0000 7fefdf06e640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefd815c970 con 0x7fefd80cc6b0 2024-03-29T04:18:37.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.988+0000 7fefdf06e640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fefd815d240 con 0x7fefd815d6d0 2024-03-29T04:18:37.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.988+0000 7fefdf06e640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fefd80ce6b0 con 0x7fefd8159c90 2024-03-29T04:18:37.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.988+0000 7fefd7fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fefd815d6d0 0x7fefd80700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:37.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.992+0000 7fefd7fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fefd815d6d0 0x7fefd80700f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51180/0 (socket says 172.21.15.22:51180) 2024-03-29T04:18:37.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.992+0000 7fefd7fff640 1 -- 172.21.15.22:0/2904390701 learned_addr learned my addr 172.21.15.22:0/2904390701 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:37.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.992+0000 7fefdce0a640 1 --2- 172.21.15.22:0/2904390701 >> v2:172.21.15.151:3300/0 conn(0x7fefd8159c90 0x7fefd8160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:37.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.992+0000 7fefd7fff640 1 -- 172.21.15.22:0/2904390701 >> v2:172.21.15.151:3300/0 conn(0x7fefd8159c90 msgr2=0x7fefd8160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:37.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.992+0000 7fefd7fff640 1 --2- 172.21.15.22:0/2904390701 >> v2:172.21.15.151:3300/0 conn(0x7fefd8159c90 0x7fefd8160970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.992+0000 7fefd7fff640 1 -- 172.21.15.22:0/2904390701 >> v1:172.21.15.151:6789/0 conn(0x7fefd80cc6b0 legacy=0x7fefd80b6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:37.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.992+0000 7fefd7fff640 1 -- 172.21.15.22:0/2904390701 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd80ccbe0 con 0x7fefd815d6d0 2024-03-29T04:18:37.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.992+0000 7fefd7fff640 1 --2- 172.21.15.22:0/2904390701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefd815d6d0 0x7fefd80700f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fefd8072820 tx=0x7fefcc01c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e031ccc04dca98 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:37.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.992+0000 7fefd77fe640 1 -- 172.21.15.22:0/2904390701 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc0b0070 con 0x7fefd815d6d0 2024-03-29T04:18:37.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.992+0000 7fefd77fe640 1 -- 172.21.15.22:0/2904390701 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefcc0ab600 con 0x7fefd815d6d0 2024-03-29T04:18:37.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.992+0000 7fefd77fe640 1 -- 172.21.15.22:0/2904390701 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc0ab9d0 con 0x7fefd815d6d0 2024-03-29T04:18:37.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdf06e640 1 -- 172.21.15.22:0/2904390701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefd815d6d0 msgr2=0x7fefd80700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:37.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdf06e640 1 --2- 172.21.15.22:0/2904390701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefd815d6d0 0x7fefd80700f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fefd8072820 tx=0x7fefcc01c300 comp rx=0 tx=0).stop 2024-03-29T04:18:37.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdf06e640 1 -- 172.21.15.22:0/2904390701 shutdown_connections 2024-03-29T04:18:37.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdf06e640 1 --2- 172.21.15.22:0/2904390701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefd815d6d0 0x7fefd80700f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdf06e640 1 --2- 172.21.15.22:0/2904390701 >> v2:172.21.15.151:3300/0 conn(0x7fefd8159c90 0x7fefd8160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdf06e640 1 -- 172.21.15.22:0/2904390701 >> 172.21.15.22:0/2904390701 conn(0x7fefd80ceae0 msgr2=0x7fefd80ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:37.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdf06e640 1 -- 172.21.15.22:0/2904390701 shutdown_connections 2024-03-29T04:18:37.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdf06e640 1 -- 172.21.15.22:0/2904390701 wait complete. 2024-03-29T04:18:37.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdf06e640 1 Processor -- start 2024-03-29T04:18:37.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdf06e640 1 -- start start 2024-03-29T04:18:37.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdf06e640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fefd80cc6b0 0x7fefd81595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:37.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdf06e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefd8159c90 0x7fefd80c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:37.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdf06e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefd815d6d0 0x7fefd80c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:37.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdf06e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fefd80d0b00 con 0x7fefd815d6d0 2024-03-29T04:18:37.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdf06e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fefd80d0980 con 0x7fefd8159c90 2024-03-29T04:18:37.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdf06e640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fefd80d0c80 con 0x7fefd80cc6b0 2024-03-29T04:18:37.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdd60b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefd815d6d0 0x7fefd80c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:37.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdd60b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefd815d6d0 0x7fefd80c62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51192/0 (socket says 172.21.15.22:51192) 2024-03-29T04:18:37.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdd60b640 1 -- 172.21.15.22:0/747273947 learned_addr learned my addr 172.21.15.22:0/747273947 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:37.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdce0a640 1 --2- 172.21.15.22:0/747273947 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fefd80cc6b0 0x7fefd81595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:37.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefd7fff640 1 --2- 172.21.15.22:0/747273947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefd8159c90 0x7fefd80c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:37.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdd60b640 1 -- 172.21.15.22:0/747273947 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fefd80cc6b0 msgr2=0x7fefd81595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:37.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdd60b640 1 --2- 172.21.15.22:0/747273947 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fefd80cc6b0 0x7fefd81595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdd60b640 1 -- 172.21.15.22:0/747273947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefd8159c90 msgr2=0x7fefd80c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:37.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdd60b640 1 --2- 172.21.15.22:0/747273947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefd8159c90 0x7fefd80c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:37.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdd60b640 1 -- 172.21.15.22:0/747273947 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd80c78f0 con 0x7fefd815d6d0 2024-03-29T04:18:37.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:37.996+0000 7fefdd60b640 1 --2- 172.21.15.22:0/747273947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefd815d6d0 0x7fefd80c62d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fefd000b970 tx=0x7fefd000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:37.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.000+0000 7fefd57fa640 1 -- 172.21.15.22:0/747273947 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0019070 con 0x7fefd815d6d0 2024-03-29T04:18:37.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.000+0000 7fefd57fa640 1 -- 172.21.15.22:0/747273947 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd000cb40 con 0x7fefd815d6d0 2024-03-29T04:18:37.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.000+0000 7fefdf06e640 1 -- 172.21.15.22:0/747273947 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd80c7bd0 con 0x7fefd815d6d0 2024-03-29T04:18:37.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.000+0000 7fefd57fa640 1 -- 172.21.15.22:0/747273947 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0014440 con 0x7fefd815d6d0 2024-03-29T04:18:37.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.000+0000 7fefdf06e640 1 -- 172.21.15.22:0/747273947 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd8157140 con 0x7fefd815d6d0 2024-03-29T04:18:37.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.000+0000 7fefdf06e640 1 -- 172.21.15.22:0/747273947 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd8068900 con 0x7fefd815d6d0 2024-03-29T04:18:37.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.000+0000 7fefd57fa640 1 -- 172.21.15.22:0/747273947 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fefd001c070 con 0x7fefd815d6d0 2024-03-29T04:18:37.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.000+0000 7fefd57fa640 1 --2- 172.21.15.22:0/747273947 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fefc4067fc0 0x7fefc406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:37.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.000+0000 7fefd57fa640 1 -- 172.21.15.22:0/747273947 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fefd0086470 con 0x7fefd815d6d0 2024-03-29T04:18:37.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.000+0000 7fefdce0a640 1 --2- 172.21.15.22:0/747273947 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fefc4067fc0 0x7fefc406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:37.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.004+0000 7fefdce0a640 1 --2- 172.21.15.22:0/747273947 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fefc4067fc0 0x7fefc406a480 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fefd814a5e0 tx=0x7fefc8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:37.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.004+0000 7fefd57fa640 1 -- 172.21.15.22:0/747273947 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fefd0056260 con 0x7fefd815d6d0 2024-03-29T04:18:38.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.160+0000 7fefdf06e640 1 -- 172.21.15.22:0/747273947 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fefd80d1c50 con 0x7fefd815d6d0 2024-03-29T04:18:38.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.164+0000 7fefd57fa640 1 -- 172.21.15.22:0/747273947 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7fefd005a560 con 0x7fefd815d6d0 2024-03-29T04:18:38.144 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:18:38.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.164+0000 7fefdf06e640 1 -- 172.21.15.22:0/747273947 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fefc4067fc0 msgr2=0x7fefc406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:38.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.164+0000 7fefdf06e640 1 --2- 172.21.15.22:0/747273947 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fefc4067fc0 0x7fefc406a480 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fefd814a5e0 tx=0x7fefc8008040 comp rx=0 tx=0).stop 2024-03-29T04:18:38.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.164+0000 7fefdf06e640 1 -- 172.21.15.22:0/747273947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefd815d6d0 msgr2=0x7fefd80c62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:38.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.164+0000 7fefdf06e640 1 --2- 172.21.15.22:0/747273947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefd815d6d0 0x7fefd80c62d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fefd000b970 tx=0x7fefd000be30 comp rx=0 tx=0).stop 2024-03-29T04:18:38.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.164+0000 7fefdf06e640 1 -- 172.21.15.22:0/747273947 shutdown_connections 2024-03-29T04:18:38.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.164+0000 7fefdf06e640 1 --2- 172.21.15.22:0/747273947 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fefc4067fc0 0x7fefc406a480 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.164+0000 7fefdf06e640 1 --2- 172.21.15.22:0/747273947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefd815d6d0 0x7fefd80c62d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.164+0000 7fefdf06e640 1 --2- 172.21.15.22:0/747273947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefd8159c90 0x7fefd80c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.164+0000 7fefdf06e640 1 --2- 172.21.15.22:0/747273947 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fefd80cc6b0 0x7fefd81595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.164+0000 7fefdf06e640 1 -- 172.21.15.22:0/747273947 >> 172.21.15.22:0/747273947 conn(0x7fefd80ceae0 msgr2=0x7fefd806eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:38.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.164+0000 7fefdf06e640 1 -- 172.21.15.22:0/747273947 shutdown_connections 2024-03-29T04:18:38.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.164+0000 7fefdf06e640 1 -- 172.21.15.22:0/747273947 wait complete. 2024-03-29T04:18:38.162 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:18:38.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.308+0000 7f18cb679640 1 Processor -- start 2024-03-29T04:18:38.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.312+0000 7f18cb679640 1 -- start start 2024-03-29T04:18:38.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.312+0000 7f18cb679640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f18c4144930 0x7f18c40b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:38.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.312+0000 7f18cb679640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f18c40b4870 0x7f18c406c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:38.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.312+0000 7f18cb679640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18c40b33c0 con 0x7f18c406cbb0 2024-03-29T04:18:38.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.312+0000 7f18cb679640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f18c406d000 con 0x7f18c4144930 2024-03-29T04:18:38.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.312+0000 7f18cb679640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f18c40cdb90 con 0x7f18c40b4870 2024-03-29T04:18:38.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.312+0000 7f18c9415640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f18c4144930 0x7f18c40b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:38.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.312+0000 7f18c9415640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f18c4144930 0x7f18c40b2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51208/0 (socket says 172.21.15.22:51208) 2024-03-29T04:18:38.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.312+0000 7f18c9415640 1 -- 172.21.15.22:0/1795605433 learned_addr learned my addr 172.21.15.22:0/1795605433 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:38.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.312+0000 7f18c8c14640 1 --2- 172.21.15.22:0/1795605433 >> v2:172.21.15.151:3300/0 conn(0x7f18c40b4870 0x7f18c406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:38.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.312+0000 7f18c9415640 1 -- 172.21.15.22:0/1795605433 >> v2:172.21.15.151:3300/0 conn(0x7f18c40b4870 msgr2=0x7f18c406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:38.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.312+0000 7f18c9415640 1 --2- 172.21.15.22:0/1795605433 >> v2:172.21.15.151:3300/0 conn(0x7f18c40b4870 0x7f18c406c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.312+0000 7f18c9415640 1 -- 172.21.15.22:0/1795605433 >> v1:172.21.15.151:6789/0 conn(0x7f18c406cbb0 legacy=0x7f18c40713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:38.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.312+0000 7f18c9415640 1 -- 172.21.15.22:0/1795605433 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c40b3660 con 0x7f18c4144930 2024-03-29T04:18:38.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.316+0000 7f18c9415640 1 --2- 172.21.15.22:0/1795605433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c4144930 0x7f18c40b2e80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f18c40b4d70 tx=0x7f18c001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f38d7ef9d24fa9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:38.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.316+0000 7f18bbfff640 1 -- 172.21.15.22:0/1795605433 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c00ab660 con 0x7f18c4144930 2024-03-29T04:18:38.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.316+0000 7f18bbfff640 1 -- 172.21.15.22:0/1795605433 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18c00b0070 con 0x7f18c4144930 2024-03-29T04:18:38.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.316+0000 7f18bbfff640 1 -- 172.21.15.22:0/1795605433 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c00aba20 con 0x7f18c4144930 2024-03-29T04:18:38.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.316+0000 7f18cb679640 1 -- 172.21.15.22:0/1795605433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c4144930 msgr2=0x7f18c40b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:38.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.316+0000 7f18cb679640 1 --2- 172.21.15.22:0/1795605433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c4144930 0x7f18c40b2e80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f18c40b4d70 tx=0x7f18c001c460 comp rx=0 tx=0).stop 2024-03-29T04:18:38.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.316+0000 7f18cb679640 1 -- 172.21.15.22:0/1795605433 shutdown_connections 2024-03-29T04:18:38.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.316+0000 7f18cb679640 1 --2- 172.21.15.22:0/1795605433 >> v2:172.21.15.151:3300/0 conn(0x7f18c40b4870 0x7f18c406c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.316+0000 7f18cb679640 1 --2- 172.21.15.22:0/1795605433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c4144930 0x7f18c40b2e80 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.316+0000 7f18cb679640 1 -- 172.21.15.22:0/1795605433 >> 172.21.15.22:0/1795605433 conn(0x7f18c40d00a0 msgr2=0x7f18c40d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:38.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.316+0000 7f18cb679640 1 -- 172.21.15.22:0/1795605433 shutdown_connections 2024-03-29T04:18:38.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.316+0000 7f18cb679640 1 -- 172.21.15.22:0/1795605433 wait complete. 2024-03-29T04:18:38.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18cb679640 1 Processor -- start 2024-03-29T04:18:38.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18cb679640 1 -- start start 2024-03-29T04:18:38.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18cb679640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18c406cbb0 0x7f18c40aad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:38.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18cb679640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18c40b4870 0x7f18c40ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:38.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18cb679640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c4144930 0x7f18c40acd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:38.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18cb679640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f18c4167df0 con 0x7f18c4144930 2024-03-29T04:18:38.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18cb679640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f18c40cd7b0 con 0x7f18c40b4870 2024-03-29T04:18:38.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18cb679640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f18c4167f70 con 0x7f18c406cbb0 2024-03-29T04:18:38.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18c9c16640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c4144930 0x7f18c40acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:38.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18c9c16640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c4144930 0x7f18c40acd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51214/0 (socket says 172.21.15.22:51214) 2024-03-29T04:18:38.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18c9c16640 1 -- 172.21.15.22:0/388831826 learned_addr learned my addr 172.21.15.22:0/388831826 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:38.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18c9415640 1 --2- 172.21.15.22:0/388831826 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18c406cbb0 0x7f18c40aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:38.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18c8c14640 1 --2- 172.21.15.22:0/388831826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18c40b4870 0x7f18c40ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:38.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18c9c16640 1 -- 172.21.15.22:0/388831826 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18c406cbb0 msgr2=0x7f18c40aad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:38.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18c9c16640 1 --2- 172.21.15.22:0/388831826 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18c406cbb0 0x7f18c40aad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18c9c16640 1 -- 172.21.15.22:0/388831826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18c40b4870 msgr2=0x7f18c40ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:38.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18c9c16640 1 --2- 172.21.15.22:0/388831826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18c40b4870 0x7f18c40ac7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18c9c16640 1 -- 172.21.15.22:0/388831826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c40ab280 con 0x7f18c4144930 2024-03-29T04:18:38.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18c9c16640 1 --2- 172.21.15.22:0/388831826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c4144930 0x7f18c40acd20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00c730 tx=0x7f18bc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:38.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18b9ffb640 1 -- 172.21.15.22:0/388831826 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc003ba0 con 0x7f18c4144930 2024-03-29T04:18:38.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18b9ffb640 1 -- 172.21.15.22:0/388831826 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18bc003d60 con 0x7f18c4144930 2024-03-29T04:18:38.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18b9ffb640 1 -- 172.21.15.22:0/388831826 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc00aa90 con 0x7f18c4144930 2024-03-29T04:18:38.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18cb679640 1 -- 172.21.15.22:0/388831826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c406e530 con 0x7f18c4144930 2024-03-29T04:18:38.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18cb679640 1 -- 172.21.15.22:0/388831826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c40aa260 con 0x7f18c4144930 2024-03-29T04:18:38.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18cb679640 1 -- 172.21.15.22:0/388831826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c4068900 con 0x7f18c4144930 2024-03-29T04:18:38.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18b9ffb640 1 -- 172.21.15.22:0/388831826 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f18bc00ac30 con 0x7f18c4144930 2024-03-29T04:18:38.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.320+0000 7f18b9ffb640 1 --2- 172.21.15.22:0/388831826 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f18a8067fc0 0x7f18a806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:38.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.324+0000 7f18b9ffb640 1 -- 172.21.15.22:0/388831826 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f18bc086740 con 0x7f18c4144930 2024-03-29T04:18:38.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.324+0000 7f18c9415640 1 --2- 172.21.15.22:0/388831826 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f18a8067fc0 0x7f18a806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:38.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.324+0000 7f18c9415640 1 --2- 172.21.15.22:0/388831826 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f18a8067fc0 0x7f18a806a480 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f18c00af000 tx=0x7f18c00a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:38.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.328+0000 7f18b9ffb640 1 -- 172.21.15.22:0/388831826 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f18bc056b80 con 0x7f18c4144930 2024-03-29T04:18:38.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.480+0000 7f18cb679640 1 -- 172.21.15.22:0/388831826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f18c40d0620 con 0x7f18c4144930 2024-03-29T04:18:38.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.484+0000 7f18b9ffb640 1 -- 172.21.15.22:0/388831826 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f18bc056480 con 0x7f18c4144930 2024-03-29T04:18:38.463 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:38.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.484+0000 7f18cb679640 1 -- 172.21.15.22:0/388831826 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f18a8067fc0 msgr2=0x7f18a806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:38.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.484+0000 7f18cb679640 1 --2- 172.21.15.22:0/388831826 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f18a8067fc0 0x7f18a806a480 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f18c00af000 tx=0x7f18c00a7000 comp rx=0 tx=0).stop 2024-03-29T04:18:38.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.484+0000 7f18cb679640 1 -- 172.21.15.22:0/388831826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c4144930 msgr2=0x7f18c40acd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:38.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.484+0000 7f18cb679640 1 --2- 172.21.15.22:0/388831826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c4144930 0x7f18c40acd20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00c730 tx=0x7f18bc00cbf0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.484+0000 7f18cb679640 1 -- 172.21.15.22:0/388831826 shutdown_connections 2024-03-29T04:18:38.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.484+0000 7f18cb679640 1 --2- 172.21.15.22:0/388831826 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f18a8067fc0 0x7f18a806a480 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.484+0000 7f18cb679640 1 --2- 172.21.15.22:0/388831826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18c4144930 0x7f18c40acd20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.484+0000 7f18cb679640 1 --2- 172.21.15.22:0/388831826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18c40b4870 0x7f18c40ac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.484+0000 7f18cb679640 1 --2- 172.21.15.22:0/388831826 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18c406cbb0 0x7f18c40aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.484+0000 7f18cb679640 1 -- 172.21.15.22:0/388831826 >> 172.21.15.22:0/388831826 conn(0x7f18c40d00a0 msgr2=0x7f18c41421c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:38.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.484+0000 7f18cb679640 1 -- 172.21.15.22:0/388831826 shutdown_connections 2024-03-29T04:18:38.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.484+0000 7f18cb679640 1 -- 172.21.15.22:0/388831826 wait complete. 2024-03-29T04:18:38.481 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:18:38.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.628+0000 7fcb8ff26640 1 Processor -- start 2024-03-29T04:18:38.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.628+0000 7fcb8ff26640 1 -- start start 2024-03-29T04:18:38.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.628+0000 7fcb8ff26640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcb8813f140 0x7fcb881606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:38.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.628+0000 7fcb8ff26640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcb8815ff40 0x7fcb8806c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:38.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.628+0000 7fcb8ff26640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb8806d710 con 0x7fcb8806cbb0 2024-03-29T04:18:38.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.628+0000 7fcb8ff26640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fcb8806d000 con 0x7fcb8815ff40 2024-03-29T04:18:38.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.628+0000 7fcb8ff26640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fcb8806e320 con 0x7fcb8813f140 2024-03-29T04:18:38.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.628+0000 7fcb8d4c1640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcb8815ff40 0x7fcb8806c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:38.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.628+0000 7fcb8e4c3640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fcb8806cbb0 0x7fcb88141a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58510/0 (socket says 172.21.15.22:58510) 2024-03-29T04:18:38.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.628+0000 7fcb8e4c3640 1 -- 172.21.15.22:0/25746067 learned_addr learned my addr 172.21.15.22:0/25746067 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:38.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.628+0000 7fcb8dcc2640 1 --2- 172.21.15.22:0/25746067 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcb8813f140 0x7fcb881606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:38.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.628+0000 7fcb8d4c1640 1 -- 172.21.15.22:0/25746067 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcb8813f140 msgr2=0x7fcb881606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:38.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.628+0000 7fcb8d4c1640 1 --2- 172.21.15.22:0/25746067 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcb8813f140 0x7fcb881606d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.628+0000 7fcb8d4c1640 1 -- 172.21.15.22:0/25746067 >> v1:172.21.15.22:6789/0 conn(0x7fcb8806cbb0 legacy=0x7fcb88141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:38.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.628+0000 7fcb8d4c1640 1 -- 172.21.15.22:0/25746067 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8806d9f0 con 0x7fcb8815ff40 2024-03-29T04:18:38.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.628+0000 7fcb8ccc0640 1 -- 172.21.15.22:0/25746067 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 249256561 0 0) 0x7fcb8806d710 con 0x7fcb8806cbb0 2024-03-29T04:18:38.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8d4c1640 1 --2- 172.21.15.22:0/25746067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb8815ff40 0x7fcb8806c670 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcb780097c0 tx=0x7fcb7801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1e6e65694d4b17 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:38.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8ccc0640 1 -- 172.21.15.22:0/25746067 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb780a7070 con 0x7fcb8815ff40 2024-03-29T04:18:38.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8ccc0640 1 -- 172.21.15.22:0/25746067 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb780b0070 con 0x7fcb8815ff40 2024-03-29T04:18:38.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8ccc0640 1 -- 172.21.15.22:0/25746067 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb780ab890 con 0x7fcb8815ff40 2024-03-29T04:18:38.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8ff26640 1 -- 172.21.15.22:0/25746067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb8815ff40 msgr2=0x7fcb8806c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:38.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8ff26640 1 --2- 172.21.15.22:0/25746067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb8815ff40 0x7fcb8806c670 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcb780097c0 tx=0x7fcb7801c470 comp rx=0 tx=0).stop 2024-03-29T04:18:38.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8ff26640 1 -- 172.21.15.22:0/25746067 shutdown_connections 2024-03-29T04:18:38.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8ff26640 1 --2- 172.21.15.22:0/25746067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb8815ff40 0x7fcb8806c670 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8ff26640 1 --2- 172.21.15.22:0/25746067 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcb8813f140 0x7fcb881606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8ff26640 1 -- 172.21.15.22:0/25746067 >> 172.21.15.22:0/25746067 conn(0x7fcb880d00a0 msgr2=0x7fcb880d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:38.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8ff26640 1 -- 172.21.15.22:0/25746067 shutdown_connections 2024-03-29T04:18:38.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8ff26640 1 -- 172.21.15.22:0/25746067 wait complete. 2024-03-29T04:18:38.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8ff26640 1 Processor -- start 2024-03-29T04:18:38.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8ff26640 1 -- start start 2024-03-29T04:18:38.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8ff26640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcb8806cbb0 0x7fcb88138c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:38.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8ff26640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb8813f140 0x7fcb8813df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:38.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8ff26640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb8815ff40 0x7fcb8813a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:38.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8ff26640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcb88160d90 con 0x7fcb8815ff40 2024-03-29T04:18:38.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8ff26640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcb88160c10 con 0x7fcb8813f140 2024-03-29T04:18:38.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8ff26640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fcb88168ff0 con 0x7fcb8806cbb0 2024-03-29T04:18:38.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8e4c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb8815ff40 0x7fcb8813a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:38.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8e4c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb8815ff40 0x7fcb8813a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51226/0 (socket says 172.21.15.22:51226) 2024-03-29T04:18:38.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8e4c3640 1 -- 172.21.15.22:0/918952758 learned_addr learned my addr 172.21.15.22:0/918952758 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:38.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8d4c1640 1 --2- 172.21.15.22:0/918952758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb8813f140 0x7fcb8813df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:38.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8dcc2640 1 --2- 172.21.15.22:0/918952758 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcb8806cbb0 0x7fcb88138c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:38.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8e4c3640 1 -- 172.21.15.22:0/918952758 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcb8806cbb0 msgr2=0x7fcb88138c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:38.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8e4c3640 1 --2- 172.21.15.22:0/918952758 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcb8806cbb0 0x7fcb88138c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8e4c3640 1 -- 172.21.15.22:0/918952758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb8813f140 msgr2=0x7fcb8813df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:38.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8e4c3640 1 --2- 172.21.15.22:0/918952758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb8813f140 0x7fcb8813df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8e4c3640 1 -- 172.21.15.22:0/918952758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb881395e0 con 0x7fcb8815ff40 2024-03-29T04:18:38.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb8e4c3640 1 --2- 172.21.15.22:0/918952758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb8815ff40 0x7fcb8813a680 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000c7a0 tx=0x7fcb8000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:38.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.632+0000 7fcb7e7fc640 1 -- 172.21.15.22:0/918952758 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb80018070 con 0x7fcb8815ff40 2024-03-29T04:18:38.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.636+0000 7fcb7e7fc640 1 -- 172.21.15.22:0/918952758 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb8000a640 con 0x7fcb8815ff40 2024-03-29T04:18:38.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.636+0000 7fcb8ff26640 1 -- 172.21.15.22:0/918952758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb881398c0 con 0x7fcb8815ff40 2024-03-29T04:18:38.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.636+0000 7fcb7e7fc640 1 -- 172.21.15.22:0/918952758 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8000aab0 con 0x7fcb8815ff40 2024-03-29T04:18:38.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.636+0000 7fcb8ff26640 1 -- 172.21.15.22:0/918952758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb88139e80 con 0x7fcb8815ff40 2024-03-29T04:18:38.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.636+0000 7fcb8ff26640 1 -- 172.21.15.22:0/918952758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb88068900 con 0x7fcb8815ff40 2024-03-29T04:18:38.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.636+0000 7fcb7e7fc640 1 -- 172.21.15.22:0/918952758 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fcb8001b070 con 0x7fcb8815ff40 2024-03-29T04:18:38.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.636+0000 7fcb7e7fc640 1 --2- 172.21.15.22:0/918952758 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcb6c067fc0 0x7fcb6c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:38.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.636+0000 7fcb7e7fc640 1 -- 172.21.15.22:0/918952758 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fcb80085da0 con 0x7fcb8815ff40 2024-03-29T04:18:38.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.636+0000 7fcb8dcc2640 1 --2- 172.21.15.22:0/918952758 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcb6c067fc0 0x7fcb6c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:38.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.636+0000 7fcb8dcc2640 1 --2- 172.21.15.22:0/918952758 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcb6c067fc0 0x7fcb6c06a480 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcb881f86d0 tx=0x7fcb84009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:38.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.640+0000 7fcb7e7fc640 1 -- 172.21.15.22:0/918952758 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fcb80055b90 con 0x7fcb8815ff40 2024-03-29T04:18:38.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.800+0000 7fcb8ff26640 1 -- 172.21.15.22:0/918952758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcb881416e0 con 0x7fcb8815ff40 2024-03-29T04:18:38.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.800+0000 7fcb7e7fc640 1 -- 172.21.15.22:0/918952758 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fcb80059e90 con 0x7fcb8815ff40 2024-03-29T04:18:38.781 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-29T04:18:38.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.804+0000 7fcb8ff26640 1 -- 172.21.15.22:0/918952758 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcb6c067fc0 msgr2=0x7fcb6c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:38.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.804+0000 7fcb8ff26640 1 --2- 172.21.15.22:0/918952758 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcb6c067fc0 0x7fcb6c06a480 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcb881f86d0 tx=0x7fcb84009190 comp rx=0 tx=0).stop 2024-03-29T04:18:38.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.804+0000 7fcb8ff26640 1 -- 172.21.15.22:0/918952758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb8815ff40 msgr2=0x7fcb8813a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:38.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.804+0000 7fcb8ff26640 1 --2- 172.21.15.22:0/918952758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb8815ff40 0x7fcb8813a680 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000c7a0 tx=0x7fcb8000cc60 comp rx=0 tx=0).stop 2024-03-29T04:18:38.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.804+0000 7fcb8ff26640 1 -- 172.21.15.22:0/918952758 shutdown_connections 2024-03-29T04:18:38.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.804+0000 7fcb8ff26640 1 --2- 172.21.15.22:0/918952758 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcb6c067fc0 0x7fcb6c06a480 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.804+0000 7fcb8ff26640 1 --2- 172.21.15.22:0/918952758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb8815ff40 0x7fcb8813a680 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.804+0000 7fcb8ff26640 1 --2- 172.21.15.22:0/918952758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb8813f140 0x7fcb8813df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.804+0000 7fcb8ff26640 1 --2- 172.21.15.22:0/918952758 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcb8806cbb0 0x7fcb88138c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.804+0000 7fcb8ff26640 1 -- 172.21.15.22:0/918952758 >> 172.21.15.22:0/918952758 conn(0x7fcb880d00a0 msgr2=0x7fcb880b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:38.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.804+0000 7fcb8ff26640 1 -- 172.21.15.22:0/918952758 shutdown_connections 2024-03-29T04:18:38.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.804+0000 7fcb8ff26640 1 -- 172.21.15.22:0/918952758 wait complete. 2024-03-29T04:18:38.800 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:18:38.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.936+0000 7fca96967640 1 Processor -- start 2024-03-29T04:18:38.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.936+0000 7fca96967640 1 -- start start 2024-03-29T04:18:38.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.936+0000 7fca96967640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fca901479b0 0x7fca9006f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:38.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.936+0000 7fca96967640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fca90147d90 0x7fca900ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:38.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.936+0000 7fca96967640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fca900ca830 0x7fca900ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:38.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.936+0000 7fca96967640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fca9006fc60 con 0x7fca900ca830 2024-03-29T04:18:38.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.936+0000 7fca96967640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fca9006e960 con 0x7fca90147d90 2024-03-29T04:18:38.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.936+0000 7fca96967640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fca9006ec40 con 0x7fca901479b0 2024-03-29T04:18:38.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.936+0000 7fca94f04640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fca900ca830 0x7fca900ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:38.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.936+0000 7fca94f04640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fca900ca830 0x7fca900ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51238/0 (socket says 172.21.15.22:51238) 2024-03-29T04:18:38.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.936+0000 7fca94f04640 1 -- 172.21.15.22:0/1316612777 learned_addr learned my addr 172.21.15.22:0/1316612777 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:38.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.936+0000 7fca8f7fe640 1 --2- 172.21.15.22:0/1316612777 >> v2:172.21.15.151:3300/0 conn(0x7fca90147d90 0x7fca900ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:38.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.936+0000 7fca8ffff640 1 --2- 172.21.15.22:0/1316612777 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fca901479b0 0x7fca9006f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:38.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.936+0000 7fca94f04640 1 -- 172.21.15.22:0/1316612777 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fca901479b0 msgr2=0x7fca9006f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:38.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.936+0000 7fca94f04640 1 --2- 172.21.15.22:0/1316612777 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fca901479b0 0x7fca9006f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.936+0000 7fca94f04640 1 -- 172.21.15.22:0/1316612777 >> v2:172.21.15.151:3300/0 conn(0x7fca90147d90 msgr2=0x7fca900ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:38.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.936+0000 7fca94f04640 1 --2- 172.21.15.22:0/1316612777 >> v2:172.21.15.151:3300/0 conn(0x7fca90147d90 0x7fca900ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.936+0000 7fca94f04640 1 -- 172.21.15.22:0/1316612777 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9006ef20 con 0x7fca900ca830 2024-03-29T04:18:38.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.936+0000 7fca8ffff640 1 --2- 172.21.15.22:0/1316612777 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fca901479b0 0x7fca9006f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:38.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca94f04640 1 --2- 172.21.15.22:0/1316612777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca900ca830 0x7fca900ccbf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fca80009e50 tx=0x7fca8001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4d26ead5384008 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:38.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca8effd640 1 -- 172.21.15.22:0/1316612777 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca800a5670 con 0x7fca900ca830 2024-03-29T04:18:38.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca8effd640 1 -- 172.21.15.22:0/1316612777 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca800a5810 con 0x7fca900ca830 2024-03-29T04:18:38.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca8effd640 1 -- 172.21.15.22:0/1316612777 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca800a5be0 con 0x7fca900ca830 2024-03-29T04:18:38.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca96967640 1 -- 172.21.15.22:0/1316612777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca900ca830 msgr2=0x7fca900ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:38.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca96967640 1 --2- 172.21.15.22:0/1316612777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca900ca830 0x7fca900ccbf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fca80009e50 tx=0x7fca8001c9f0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca96967640 1 -- 172.21.15.22:0/1316612777 shutdown_connections 2024-03-29T04:18:38.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca96967640 1 --2- 172.21.15.22:0/1316612777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca900ca830 0x7fca900ccbf0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca96967640 1 --2- 172.21.15.22:0/1316612777 >> v2:172.21.15.151:3300/0 conn(0x7fca90147d90 0x7fca900ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca96967640 1 --2- 172.21.15.22:0/1316612777 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fca901479b0 0x7fca9006f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca96967640 1 -- 172.21.15.22:0/1316612777 >> 172.21.15.22:0/1316612777 conn(0x7fca900b2c70 msgr2=0x7fca900b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:38.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca96967640 1 -- 172.21.15.22:0/1316612777 shutdown_connections 2024-03-29T04:18:38.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca96967640 1 -- 172.21.15.22:0/1316612777 wait complete. 2024-03-29T04:18:38.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca96967640 1 Processor -- start 2024-03-29T04:18:38.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca96967640 1 -- start start 2024-03-29T04:18:38.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca96967640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fca900ca830 0x7fca9015c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:38.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca96967640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca901479b0 0x7fca900cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:38.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca96967640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca90147d90 0x7fca9015dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:38.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca96967640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fca901606b0 con 0x7fca90147d90 2024-03-29T04:18:38.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca96967640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fca90160530 con 0x7fca901479b0 2024-03-29T04:18:38.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca96967640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fca90162010 con 0x7fca900ca830 2024-03-29T04:18:38.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca94f04640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca90147d90 0x7fca9015dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:38.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca94f04640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca90147d90 0x7fca9015dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51242/0 (socket says 172.21.15.22:51242) 2024-03-29T04:18:38.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca94f04640 1 -- 172.21.15.22:0/2131824031 learned_addr learned my addr 172.21.15.22:0/2131824031 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:38.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca8f7fe640 1 --2- 172.21.15.22:0/2131824031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca901479b0 0x7fca900cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:38.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca8ffff640 1 --2- 172.21.15.22:0/2131824031 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fca900ca830 0x7fca9015c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:38.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca94f04640 1 -- 172.21.15.22:0/2131824031 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fca900ca830 msgr2=0x7fca9015c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:38.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca94f04640 1 --2- 172.21.15.22:0/2131824031 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fca900ca830 0x7fca9015c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca94f04640 1 -- 172.21.15.22:0/2131824031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca901479b0 msgr2=0x7fca900cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:38.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.940+0000 7fca94f04640 1 --2- 172.21.15.22:0/2131824031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca901479b0 0x7fca900cfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:38.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.944+0000 7fca94f04640 1 -- 172.21.15.22:0/2131824031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9015cc00 con 0x7fca90147d90 2024-03-29T04:18:38.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.944+0000 7fca94f04640 1 --2- 172.21.15.22:0/2131824031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca90147d90 0x7fca9015dc80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fca800af040 tx=0x7fca80019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:38.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.944+0000 7fca8cff9640 1 -- 172.21.15.22:0/2131824031 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca80009510 con 0x7fca90147d90 2024-03-29T04:18:38.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.944+0000 7fca8cff9640 1 -- 172.21.15.22:0/2131824031 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca800be070 con 0x7fca90147d90 2024-03-29T04:18:38.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.944+0000 7fca8cff9640 1 -- 172.21.15.22:0/2131824031 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca800b88b0 con 0x7fca90147d90 2024-03-29T04:18:38.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.944+0000 7fca96967640 1 -- 172.21.15.22:0/2131824031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9015cee0 con 0x7fca90147d90 2024-03-29T04:18:38.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.944+0000 7fca96967640 1 -- 172.21.15.22:0/2131824031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca9015d780 con 0x7fca90147d90 2024-03-29T04:18:38.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.944+0000 7fca96967640 1 -- 172.21.15.22:0/2131824031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca9015d2d0 con 0x7fca90147d90 2024-03-29T04:18:38.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.944+0000 7fca8cff9640 1 -- 172.21.15.22:0/2131824031 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fca800b8a50 con 0x7fca90147d90 2024-03-29T04:18:38.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.944+0000 7fca8cff9640 1 --2- 172.21.15.22:0/2131824031 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fca70067ff0 0x7fca7006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:38.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.944+0000 7fca8cff9640 1 -- 172.21.15.22:0/2131824031 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fca801214c0 con 0x7fca90147d90 2024-03-29T04:18:38.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.944+0000 7fca8ffff640 1 --2- 172.21.15.22:0/2131824031 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fca70067ff0 0x7fca7006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:38.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.944+0000 7fca8ffff640 1 --2- 172.21.15.22:0/2131824031 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fca70067ff0 0x7fca7006a4b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fca8800ab60 tx=0x7fca88008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:38.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:38.948+0000 7fca8cff9640 1 -- 172.21.15.22:0/2131824031 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fca800165e0 con 0x7fca90147d90 2024-03-29T04:18:39.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.104+0000 7fca96967640 1 -- 172.21.15.22:0/2131824031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fca9015d0b0 con 0x7fca90147d90 2024-03-29T04:18:39.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.104+0000 7fca8cff9640 1 -- 172.21.15.22:0/2131824031 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7fca800f1340 con 0x7fca90147d90 2024-03-29T04:18:39.087 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:39.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.108+0000 7fca96967640 1 -- 172.21.15.22:0/2131824031 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fca70067ff0 msgr2=0x7fca7006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:39.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.108+0000 7fca96967640 1 --2- 172.21.15.22:0/2131824031 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fca70067ff0 0x7fca7006a4b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fca8800ab60 tx=0x7fca88008040 comp rx=0 tx=0).stop 2024-03-29T04:18:39.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.108+0000 7fca96967640 1 -- 172.21.15.22:0/2131824031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca90147d90 msgr2=0x7fca9015dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:39.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.108+0000 7fca96967640 1 --2- 172.21.15.22:0/2131824031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca90147d90 0x7fca9015dc80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fca800af040 tx=0x7fca80019790 comp rx=0 tx=0).stop 2024-03-29T04:18:39.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.108+0000 7fca96967640 1 -- 172.21.15.22:0/2131824031 shutdown_connections 2024-03-29T04:18:39.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.108+0000 7fca96967640 1 --2- 172.21.15.22:0/2131824031 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fca70067ff0 0x7fca7006a4b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.108+0000 7fca96967640 1 --2- 172.21.15.22:0/2131824031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca90147d90 0x7fca9015dc80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.108+0000 7fca96967640 1 --2- 172.21.15.22:0/2131824031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca901479b0 0x7fca900cfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.108+0000 7fca96967640 1 --2- 172.21.15.22:0/2131824031 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fca900ca830 0x7fca9015c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.108+0000 7fca96967640 1 -- 172.21.15.22:0/2131824031 >> 172.21.15.22:0/2131824031 conn(0x7fca900b2c70 msgr2=0x7fca901466a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:39.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.108+0000 7fca96967640 1 -- 172.21.15.22:0/2131824031 shutdown_connections 2024-03-29T04:18:39.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.108+0000 7fca96967640 1 -- 172.21.15.22:0/2131824031 wait complete. 2024-03-29T04:18:39.104 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:18:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff0456d4640 1 Processor -- start 2024-03-29T04:18:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff0456d4640 1 -- start start 2024-03-29T04:18:39.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff0456d4640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff0400cc910 0x7ff040166ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:39.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff0456d4640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0400ccef0 con 0x7ff04006c670 2024-03-29T04:18:39.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff0456d4640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0400b3710 con 0x7ff0400b41f0 2024-03-29T04:18:39.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff0456d4640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7ff0400b34c0 con 0x7ff0400cc910 2024-03-29T04:18:39.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff03effd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff04006c670 0x7ff04006e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58514/0 (socket says 172.21.15.22:58514) 2024-03-29T04:18:39.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff03effd640 1 -- 172.21.15.22:0/2735944939 learned_addr learned my addr 172.21.15.22:0/2735944939 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:39.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff03f7fe640 1 --2- 172.21.15.22:0/2735944939 >> v2:172.21.15.151:3300/0 conn(0x7ff0400cc910 0x7ff040166ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:39.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff03dffb640 1 -- 172.21.15.22:0/2735944939 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3192898043 0 0) 0x7ff0400ccef0 con 0x7ff04006c670 2024-03-29T04:18:39.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff03dffb640 1 -- 172.21.15.22:0/2735944939 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff028003530 con 0x7ff04006c670 2024-03-29T04:18:39.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff03dffb640 1 -- 172.21.15.22:0/2735944939 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1541727066 0 0) 0x7ff028003530 con 0x7ff04006c670 2024-03-29T04:18:39.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff03dffb640 1 -- 172.21.15.22:0/2735944939 >> v2:172.21.15.151:3300/0 conn(0x7ff0400cc910 msgr2=0x7ff040166ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:39.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff03dffb640 1 --2- 172.21.15.22:0/2735944939 >> v2:172.21.15.151:3300/0 conn(0x7ff0400cc910 0x7ff040166ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff03dffb640 1 -- 172.21.15.22:0/2735944939 >> v1:172.21.15.151:6789/0 conn(0x7ff0400b41f0 legacy=0x7ff04006ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:39.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff03dffb640 1 -- 172.21.15.22:0/2735944939 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0400b39f0 con 0x7ff04006c670 2024-03-29T04:18:39.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff03dffb640 1 -- 172.21.15.22:0/2735944939 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7ff030001680 con 0x7ff04006c670 2024-03-29T04:18:39.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff03dffb640 1 -- 172.21.15.22:0/2735944939 >> v1:172.21.15.22:6789/0 conn(0x7ff04006c670 legacy=0x7ff04006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:39.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff03dffb640 1 --2- 172.21.15.22:0/2735944939 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0280039c0 0x7ff028003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:39.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff03dffb640 1 --2- 172.21.15.22:0/2735944939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff028004810 0x7ff028004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff03dffb640 1 --2- 172.21.15.22:0/2735944939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0280051d0 0x7ff02800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff03dffb640 1 -- 172.21.15.22:0/2735944939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff0400b34c0 con 0x7ff0280051d0 2024-03-29T04:18:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff03dffb640 1 -- 172.21.15.22:0/2735944939 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff02800c070 con 0x7ff028004810 2024-03-29T04:18:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff03dffb640 1 -- 172.21.15.22:0/2735944939 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff02800c270 con 0x7ff0280039c0 2024-03-29T04:18:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.236+0000 7ff03f7fe640 1 --2- 172.21.15.22:0/2735944939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0280051d0 0x7ff02800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.240+0000 7ff03e7fc640 1 --2- 172.21.15.22:0/2735944939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff028004810 0x7ff028004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.240+0000 7ff03effd640 1 --2- 172.21.15.22:0/2735944939 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0280039c0 0x7ff028003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.240+0000 7ff03f7fe640 1 -- 172.21.15.22:0/2735944939 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0280039c0 msgr2=0x7ff028003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.240+0000 7ff03f7fe640 1 --2- 172.21.15.22:0/2735944939 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff0280039c0 0x7ff028003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.240+0000 7ff03f7fe640 1 -- 172.21.15.22:0/2735944939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff028004810 msgr2=0x7ff028004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:39.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.240+0000 7ff03f7fe640 1 --2- 172.21.15.22:0/2735944939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff028004810 0x7ff028004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.240+0000 7ff03f7fe640 1 -- 172.21.15.22:0/2735944939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff028003530 con 0x7ff0280051d0 2024-03-29T04:18:39.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.240+0000 7ff03f7fe640 1 -- 172.21.15.22:0/2735944939 reap_dead start 2024-03-29T04:18:39.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.240+0000 7ff03f7fe640 1 --2- 172.21.15.22:0/2735944939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0280051d0 0x7ff02800b9d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff038009fc0 tx=0x7ff03801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47160ccad4eca18c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:39.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.240+0000 7ff03dffb640 1 -- 172.21.15.22:0/2735944939 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038009c10 con 0x7ff0280051d0 2024-03-29T04:18:39.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.240+0000 7ff03dffb640 1 -- 172.21.15.22:0/2735944939 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0380a5cd0 con 0x7ff0280051d0 2024-03-29T04:18:39.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.240+0000 7ff03dffb640 1 -- 172.21.15.22:0/2735944939 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038003be0 con 0x7ff0280051d0 2024-03-29T04:18:39.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.240+0000 7ff0456d4640 1 -- 172.21.15.22:0/2735944939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0280051d0 msgr2=0x7ff02800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:39.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.240+0000 7ff0456d4640 1 --2- 172.21.15.22:0/2735944939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0280051d0 0x7ff02800b9d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff038009fc0 tx=0x7ff03801ccc0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.240+0000 7ff0456d4640 1 -- 172.21.15.22:0/2735944939 shutdown_connections 2024-03-29T04:18:39.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.240+0000 7ff0456d4640 1 --2- 172.21.15.22:0/2735944939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0280051d0 0x7ff02800b9d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.240+0000 7ff0456d4640 1 -- 172.21.15.22:0/2735944939 >> 172.21.15.22:0/2735944939 conn(0x7ff0400cfee0 msgr2=0x7ff0400d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:39.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.240+0000 7ff0456d4640 1 -- 172.21.15.22:0/2735944939 shutdown_connections 2024-03-29T04:18:39.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.240+0000 7ff0456d4640 1 -- 172.21.15.22:0/2735944939 wait complete. 2024-03-29T04:18:39.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff0456d4640 1 Processor -- start 2024-03-29T04:18:39.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff0456d4640 1 -- start start 2024-03-29T04:18:39.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff0456d4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0280051d0 0x7ff04013a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:39.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff0456d4640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff040073290 0x7ff0400cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:39.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff0456d4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0400cbde0 0x7ff04015e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:39.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff0456d4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff040145e80 con 0x7ff0400cbde0 2024-03-29T04:18:39.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff0456d4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0400ccd20 con 0x7ff0280051d0 2024-03-29T04:18:39.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff0456d4640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff040146000 con 0x7ff040073290 2024-03-29T04:18:39.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff03f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0400cbde0 0x7ff04015e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:39.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff03f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0400cbde0 0x7ff04015e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51260/0 (socket says 172.21.15.22:51260) 2024-03-29T04:18:39.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff03f7fe640 1 -- 172.21.15.22:0/1829072546 learned_addr learned my addr 172.21.15.22:0/1829072546 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:39.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff03effd640 1 --2- 172.21.15.22:0/1829072546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0280051d0 0x7ff04013a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:39.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff03e7fc640 1 --2- 172.21.15.22:0/1829072546 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff040073290 0x7ff0400cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:39.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff03f7fe640 1 -- 172.21.15.22:0/1829072546 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff040073290 msgr2=0x7ff0400cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:39.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff03f7fe640 1 --2- 172.21.15.22:0/1829072546 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff040073290 0x7ff0400cb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff03f7fe640 1 -- 172.21.15.22:0/1829072546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0280051d0 msgr2=0x7ff04013a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:39.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff03f7fe640 1 --2- 172.21.15.22:0/1829072546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0280051d0 0x7ff04013a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff03f7fe640 1 -- 172.21.15.22:0/1829072546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04013af00 con 0x7ff0400cbde0 2024-03-29T04:18:39.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff03f7fe640 1 --2- 172.21.15.22:0/1829072546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0400cbde0 0x7ff04015e9f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff03801c730 tx=0x7ff038009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:39.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff0277fe640 1 -- 172.21.15.22:0/1829072546 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038009c10 con 0x7ff0400cbde0 2024-03-29T04:18:39.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff0277fe640 1 -- 172.21.15.22:0/1829072546 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0380bd070 con 0x7ff0400cbde0 2024-03-29T04:18:39.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff0277fe640 1 -- 172.21.15.22:0/1829072546 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0380039a0 con 0x7ff0400cbde0 2024-03-29T04:18:39.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff0456d4640 1 -- 172.21.15.22:0/1829072546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff04013b1e0 con 0x7ff0400cbde0 2024-03-29T04:18:39.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff0456d4640 1 -- 172.21.15.22:0/1829072546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0400c97a0 con 0x7ff0400cbde0 2024-03-29T04:18:39.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.244+0000 7ff0456d4640 1 -- 172.21.15.22:0/1829072546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff040068890 con 0x7ff0400cbde0 2024-03-29T04:18:39.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.248+0000 7ff0277fe640 1 -- 172.21.15.22:0/1829072546 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff038003b40 con 0x7ff0400cbde0 2024-03-29T04:18:39.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.248+0000 7ff0277fe640 1 --2- 172.21.15.22:0/1829072546 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff02c067fc0 0x7ff02c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:39.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.248+0000 7ff0277fe640 1 -- 172.21.15.22:0/1829072546 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7ff038122380 con 0x7ff0400cbde0 2024-03-29T04:18:39.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.248+0000 7ff03effd640 1 --2- 172.21.15.22:0/1829072546 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff02c067fc0 0x7ff02c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:39.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.248+0000 7ff03effd640 1 --2- 172.21.15.22:0/1829072546 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff02c067fc0 0x7ff02c06a480 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff030000c10 tx=0x7ff030007900 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:39.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.252+0000 7ff0277fe640 1 -- 172.21.15.22:0/1829072546 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff0380169d0 con 0x7ff0400cbde0 2024-03-29T04:18:39.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.416+0000 7ff0456d4640 1 -- 172.21.15.22:0/1829072546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff0400ccea0 con 0x7ff0400cbde0 2024-03-29T04:18:39.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.416+0000 7ff0277fe640 1 -- 172.21.15.22:0/1829072546 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7ff0380f20e0 con 0x7ff0400cbde0 2024-03-29T04:18:39.398 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-03-29T04:18:39.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.420+0000 7ff0456d4640 1 -- 172.21.15.22:0/1829072546 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff02c067fc0 msgr2=0x7ff02c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:39.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.420+0000 7ff0456d4640 1 --2- 172.21.15.22:0/1829072546 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff02c067fc0 0x7ff02c06a480 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff030000c10 tx=0x7ff030007900 comp rx=0 tx=0).stop 2024-03-29T04:18:39.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.420+0000 7ff0456d4640 1 -- 172.21.15.22:0/1829072546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0400cbde0 msgr2=0x7ff04015e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:39.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.420+0000 7ff0456d4640 1 --2- 172.21.15.22:0/1829072546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0400cbde0 0x7ff04015e9f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff03801c730 tx=0x7ff038009750 comp rx=0 tx=0).stop 2024-03-29T04:18:39.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.420+0000 7ff0456d4640 1 -- 172.21.15.22:0/1829072546 shutdown_connections 2024-03-29T04:18:39.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.420+0000 7ff0456d4640 1 --2- 172.21.15.22:0/1829072546 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff02c067fc0 0x7ff02c06a480 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.420+0000 7ff0456d4640 1 --2- 172.21.15.22:0/1829072546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0400cbde0 0x7ff04015e9f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.420+0000 7ff0456d4640 1 --2- 172.21.15.22:0/1829072546 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff040073290 0x7ff0400cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.420+0000 7ff0456d4640 1 --2- 172.21.15.22:0/1829072546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0280051d0 0x7ff04013a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.420+0000 7ff0456d4640 1 -- 172.21.15.22:0/1829072546 >> 172.21.15.22:0/1829072546 conn(0x7ff0400cfee0 msgr2=0x7ff0401442e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:39.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.420+0000 7ff0456d4640 1 -- 172.21.15.22:0/1829072546 shutdown_connections 2024-03-29T04:18:39.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.420+0000 7ff0456d4640 1 -- 172.21.15.22:0/1829072546 wait complete. 2024-03-29T04:18:39.416 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi022.front.sepia.ceph.com... 2024-03-29T04:18:39.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:39.416 DEBUG:teuthology.orchestra.run.smithi022:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-03-29T04:18:39.426 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:18:39.426 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-03-29T04:18:39.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:39.427 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-29T04:18:39.475 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:18:39.475 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:18:39.482 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:18:39.482 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:18:39.530 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:39.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.676+0000 7f582e109640 1 Processor -- start 2024-03-29T04:18:39.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.676+0000 7f582e109640 1 -- start start 2024-03-29T04:18:39.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.676+0000 7f582e109640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58280b41f0 0x7f58280cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:39.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.676+0000 7f582e109640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f582806ca50 0x7f58280ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:39.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.676+0000 7f582e109640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58280ccef0 con 0x7f582806c670 2024-03-29T04:18:39.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.676+0000 7f582e109640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f58280b34c0 con 0x7f582806ca50 2024-03-29T04:18:39.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.676+0000 7f582e109640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f58280b3710 con 0x7f58280b41f0 2024-03-29T04:18:39.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.676+0000 7f5827fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f582806ca50 0x7f58280ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:39.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.676+0000 7f58277fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f582806c670 0x7f582806e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58530/0 (socket says 172.21.15.22:58530) 2024-03-29T04:18:39.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.676+0000 7f58277fe640 1 -- 172.21.15.22:0/1971096292 learned_addr learned my addr 172.21.15.22:0/1971096292 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:39.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.676+0000 7f581ffff640 1 --2- 172.21.15.22:0/1971096292 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58280b41f0 0x7f58280cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:39.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.676+0000 7f5827fff640 1 -- 172.21.15.22:0/1971096292 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58280b41f0 msgr2=0x7f58280cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:39.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.676+0000 7f5827fff640 1 --2- 172.21.15.22:0/1971096292 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58280b41f0 0x7f58280cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.676+0000 7f5827fff640 1 -- 172.21.15.22:0/1971096292 >> v1:172.21.15.22:6789/0 conn(0x7f582806c670 legacy=0x7f582806e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:39.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.676+0000 7f5827fff640 1 -- 172.21.15.22:0/1971096292 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58280b39f0 con 0x7f582806ca50 2024-03-29T04:18:39.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.676+0000 7f5826ffd640 1 -- 172.21.15.22:0/1971096292 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2941230963 0 0) 0x7f58280ccef0 con 0x7f582806c670 2024-03-29T04:18:39.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.680+0000 7f5827fff640 1 --2- 172.21.15.22:0/1971096292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f582806ca50 0x7f58280ce5b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5820009d90 tx=0x7f582001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702e85933bce90ce server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:39.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.680+0000 7f5826ffd640 1 -- 172.21.15.22:0/1971096292 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58200b2070 con 0x7f582806ca50 2024-03-29T04:18:39.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.680+0000 7f5826ffd640 1 -- 172.21.15.22:0/1971096292 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58200a5850 con 0x7f582806ca50 2024-03-29T04:18:39.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.680+0000 7f5826ffd640 1 -- 172.21.15.22:0/1971096292 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58200a5c20 con 0x7f582806ca50 2024-03-29T04:18:39.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.680+0000 7f582e109640 1 -- 172.21.15.22:0/1971096292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f582806ca50 msgr2=0x7f58280ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:39.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.680+0000 7f582e109640 1 --2- 172.21.15.22:0/1971096292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f582806ca50 0x7f58280ce5b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5820009d90 tx=0x7f582001c970 comp rx=0 tx=0).stop 2024-03-29T04:18:39.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.680+0000 7f582e109640 1 -- 172.21.15.22:0/1971096292 shutdown_connections 2024-03-29T04:18:39.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.680+0000 7f582e109640 1 --2- 172.21.15.22:0/1971096292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f582806ca50 0x7f58280ce5b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.680+0000 7f582e109640 1 --2- 172.21.15.22:0/1971096292 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58280b41f0 0x7f58280cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.680+0000 7f582e109640 1 -- 172.21.15.22:0/1971096292 >> 172.21.15.22:0/1971096292 conn(0x7f58280cfee0 msgr2=0x7f58280d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:39.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.680+0000 7f582e109640 1 -- 172.21.15.22:0/1971096292 shutdown_connections 2024-03-29T04:18:39.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.680+0000 7f582e109640 1 -- 172.21.15.22:0/1971096292 wait complete. 2024-03-29T04:18:39.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f582e109640 1 Processor -- start 2024-03-29T04:18:39.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f582e109640 1 -- start start 2024-03-29T04:18:39.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f582e109640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f582806c670 0x7f5828166920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:39.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f582e109640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f582806ca50 0x7f582813fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:39.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f582e109640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58280b41f0 0x7f582815b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:39.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f582e109640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5828169070 con 0x7f58280b41f0 2024-03-29T04:18:39.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f582e109640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5828168ef0 con 0x7f582806c670 2024-03-29T04:18:39.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f582e109640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f58281691f0 con 0x7f582806ca50 2024-03-29T04:18:39.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f5827fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58280b41f0 0x7f582815b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:39.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f5827fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58280b41f0 0x7f582815b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51288/0 (socket says 172.21.15.22:51288) 2024-03-29T04:18:39.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f5827fff640 1 -- 172.21.15.22:0/1510688400 learned_addr learned my addr 172.21.15.22:0/1510688400 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:39.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f58277fe640 1 --2- 172.21.15.22:0/1510688400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f582806c670 0x7f5828166920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:39.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f581ffff640 1 --2- 172.21.15.22:0/1510688400 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f582806ca50 0x7f582813fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:39.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f5827fff640 1 -- 172.21.15.22:0/1510688400 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f582806ca50 msgr2=0x7f582813fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:39.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f5827fff640 1 --2- 172.21.15.22:0/1510688400 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f582806ca50 0x7f582813fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f5827fff640 1 -- 172.21.15.22:0/1510688400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f582806c670 msgr2=0x7f5828166920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:39.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f5827fff640 1 --2- 172.21.15.22:0/1510688400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f582806c670 0x7f5828166920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f5827fff640 1 -- 172.21.15.22:0/1510688400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58280732a0 con 0x7f58280b41f0 2024-03-29T04:18:39.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f5827fff640 1 --2- 172.21.15.22:0/1510688400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58280b41f0 0x7f582815b0b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f582001b020 tx=0x7f5820019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:39.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f5824ff9640 1 -- 172.21.15.22:0/1510688400 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58200b2050 con 0x7f58280b41f0 2024-03-29T04:18:39.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f5824ff9640 1 -- 172.21.15.22:0/1510688400 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58200be070 con 0x7f58280b41f0 2024-03-29T04:18:39.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f582e109640 1 -- 172.21.15.22:0/1510688400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5828165e40 con 0x7f58280b41f0 2024-03-29T04:18:39.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f5824ff9640 1 -- 172.21.15.22:0/1510688400 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58200ada50 con 0x7f58280b41f0 2024-03-29T04:18:39.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f582e109640 1 -- 172.21.15.22:0/1510688400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f582815bee0 con 0x7f58280b41f0 2024-03-29T04:18:39.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f582e109640 1 -- 172.21.15.22:0/1510688400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5828165c60 con 0x7f58280b41f0 2024-03-29T04:18:39.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.684+0000 7f5824ff9640 1 -- 172.21.15.22:0/1510688400 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f58200a56d0 con 0x7f58280b41f0 2024-03-29T04:18:39.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.688+0000 7f5824ff9640 1 --2- 172.21.15.22:0/1510688400 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5808067fc0 0x7f580806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:39.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.688+0000 7f5824ff9640 1 -- 172.21.15.22:0/1510688400 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f5820121b30 con 0x7f58280b41f0 2024-03-29T04:18:39.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.688+0000 7f58277fe640 1 --2- 172.21.15.22:0/1510688400 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5808067fc0 0x7f580806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:39.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.688+0000 7f58277fe640 1 --2- 172.21.15.22:0/1510688400 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5808067fc0 0x7f580806a480 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5818000f70 tx=0x7f5818007df0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:39.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.692+0000 7f5824ff9640 1 -- 172.21.15.22:0/1510688400 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f5828165c60 con 0x7f58280b41f0 2024-03-29T04:18:39.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.848+0000 7f582e109640 1 -- 172.21.15.22:0/1510688400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5828068890 con 0x7f58280b41f0 2024-03-29T04:18:39.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.852+0000 7f5824ff9640 1 -- 172.21.15.22:0/1510688400 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18847 (secure 0 0 0) 0x7f5820016560 con 0x7f58280b41f0 2024-03-29T04:18:39.832 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:39.832 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:18:16.893439+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:39.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.852+0000 7f582e109640 1 -- 172.21.15.22:0/1510688400 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5808067fc0 msgr2=0x7f580806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:39.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.852+0000 7f582e109640 1 --2- 172.21.15.22:0/1510688400 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5808067fc0 0x7f580806a480 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5818000f70 tx=0x7f5818007df0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.852+0000 7f582e109640 1 -- 172.21.15.22:0/1510688400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58280b41f0 msgr2=0x7f582815b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:39.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.852+0000 7f582e109640 1 --2- 172.21.15.22:0/1510688400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58280b41f0 0x7f582815b0b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f582001b020 tx=0x7f5820019710 comp rx=0 tx=0).stop 2024-03-29T04:18:39.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.852+0000 7f582e109640 1 -- 172.21.15.22:0/1510688400 shutdown_connections 2024-03-29T04:18:39.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.852+0000 7f582e109640 1 --2- 172.21.15.22:0/1510688400 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5808067fc0 0x7f580806a480 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.852+0000 7f582e109640 1 --2- 172.21.15.22:0/1510688400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58280b41f0 0x7f582815b0b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.852+0000 7f582e109640 1 --2- 172.21.15.22:0/1510688400 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f582806ca50 0x7f582813fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.852+0000 7f582e109640 1 --2- 172.21.15.22:0/1510688400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f582806c670 0x7f5828166920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.852+0000 7f582e109640 1 -- 172.21.15.22:0/1510688400 >> 172.21.15.22:0/1510688400 conn(0x7f58280cfee0 msgr2=0x7f58280cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:39.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.852+0000 7f582e109640 1 -- 172.21.15.22:0/1510688400 shutdown_connections 2024-03-29T04:18:39.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.852+0000 7f582e109640 1 -- 172.21.15.22:0/1510688400 wait complete. 2024-03-29T04:18:39.851 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:17:34.034142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:17:34.886659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:17:38.628632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:18:39.851 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:18:39.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.988+0000 7f3b2820f640 1 Processor -- start 2024-03-29T04:18:39.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.988+0000 7f3b2820f640 1 -- start start 2024-03-29T04:18:39.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.992+0000 7f3b2820f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b2013f140 0x7f3b201606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:39.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.992+0000 7f3b2820f640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f3b2015ff40 0x7f3b2006c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:39.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.992+0000 7f3b2820f640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b2006d710 con 0x7f3b2006cbb0 2024-03-29T04:18:39.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.992+0000 7f3b2820f640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f3b2006d000 con 0x7f3b2015ff40 2024-03-29T04:18:39.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.992+0000 7f3b2820f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2006e320 con 0x7f3b2013f140 2024-03-29T04:18:39.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.992+0000 7f3b25fab640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b2013f140 0x7f3b201606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:39.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.992+0000 7f3b25fab640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b2013f140 0x7f3b201606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.22:43342/0 (socket says 172.21.15.22:43342) 2024-03-29T04:18:39.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.992+0000 7f3b25fab640 1 -- 172.21.15.22:0/4134386182 learned_addr learned my addr 172.21.15.22:0/4134386182 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:39.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.992+0000 7f3b257aa640 1 --2- 172.21.15.22:0/4134386182 >> v2:172.21.15.151:3300/0 conn(0x7f3b2015ff40 0x7f3b2006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:39.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.992+0000 7f3b25fab640 1 -- 172.21.15.22:0/4134386182 >> v2:172.21.15.151:3300/0 conn(0x7f3b2015ff40 msgr2=0x7f3b2006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:39.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.992+0000 7f3b25fab640 1 --2- 172.21.15.22:0/4134386182 >> v2:172.21.15.151:3300/0 conn(0x7f3b2015ff40 0x7f3b2006c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.992+0000 7f3b25fab640 1 -- 172.21.15.22:0/4134386182 >> v1:172.21.15.151:6789/0 conn(0x7f3b2006cbb0 legacy=0x7f3b20141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:39.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.992+0000 7f3b25fab640 1 -- 172.21.15.22:0/4134386182 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2006d9f0 con 0x7f3b2013f140 2024-03-29T04:18:39.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b25fab640 1 --2- 172.21.15.22:0/4134386182 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b2013f140 0x7f3b201606d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3b10009870 tx=0x7f3b1001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47d355b6e7f4bc2c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:39.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b24fa9640 1 -- 172.21.15.22:0/4134386182 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b100a95f0 con 0x7f3b2013f140 2024-03-29T04:18:39.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b24fa9640 1 -- 172.21.15.22:0/4134386182 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b100a9790 con 0x7f3b2013f140 2024-03-29T04:18:39.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b24fa9640 1 -- 172.21.15.22:0/4134386182 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b100a9b60 con 0x7f3b2013f140 2024-03-29T04:18:39.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b2820f640 1 -- 172.21.15.22:0/4134386182 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b2013f140 msgr2=0x7f3b201606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:39.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b2820f640 1 --2- 172.21.15.22:0/4134386182 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b2013f140 0x7f3b201606d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3b10009870 tx=0x7f3b1001c530 comp rx=0 tx=0).stop 2024-03-29T04:18:39.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b2820f640 1 -- 172.21.15.22:0/4134386182 shutdown_connections 2024-03-29T04:18:39.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b2820f640 1 --2- 172.21.15.22:0/4134386182 >> v2:172.21.15.151:3300/0 conn(0x7f3b2015ff40 0x7f3b2006c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b2820f640 1 --2- 172.21.15.22:0/4134386182 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b2013f140 0x7f3b201606d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b2820f640 1 -- 172.21.15.22:0/4134386182 >> 172.21.15.22:0/4134386182 conn(0x7f3b200d00a0 msgr2=0x7f3b200d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:39.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b2820f640 1 -- 172.21.15.22:0/4134386182 shutdown_connections 2024-03-29T04:18:39.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b2820f640 1 -- 172.21.15.22:0/4134386182 wait complete. 2024-03-29T04:18:39.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b2820f640 1 Processor -- start 2024-03-29T04:18:39.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b2820f640 1 -- start start 2024-03-29T04:18:39.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b2820f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b2006cbb0 0x7f3b2013ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:39.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b2820f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b2013f140 0x7f3b200b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:39.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b2820f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b2015ff40 0x7f3b20151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:39.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b2820f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3b20160d90 con 0x7f3b2013f140 2024-03-29T04:18:39.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b2820f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3b20160c10 con 0x7f3b2015ff40 2024-03-29T04:18:39.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b2820f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f3b20168ff0 con 0x7f3b2006cbb0 2024-03-29T04:18:39.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b257aa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b2013f140 0x7f3b200b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:39.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b257aa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b2013f140 0x7f3b200b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51298/0 (socket says 172.21.15.22:51298) 2024-03-29T04:18:39.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b257aa640 1 -- 172.21.15.22:0/3950437975 learned_addr learned my addr 172.21.15.22:0/3950437975 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:39.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b25fab640 1 --2- 172.21.15.22:0/3950437975 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b2006cbb0 0x7f3b2013ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b267ac640 1 --2- 172.21.15.22:0/3950437975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b2015ff40 0x7f3b20151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b257aa640 1 -- 172.21.15.22:0/3950437975 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b2006cbb0 msgr2=0x7f3b2013ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b257aa640 1 --2- 172.21.15.22:0/3950437975 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b2006cbb0 0x7f3b2013ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b257aa640 1 -- 172.21.15.22:0/3950437975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b2015ff40 msgr2=0x7f3b20151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b257aa640 1 --2- 172.21.15.22:0/3950437975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b2015ff40 0x7f3b20151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b257aa640 1 -- 172.21.15.22:0/3950437975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b201517d0 con 0x7f3b2013f140 2024-03-29T04:18:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b25fab640 1 --2- 172.21.15.22:0/3950437975 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b2006cbb0 0x7f3b2013ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:39.996+0000 7f3b257aa640 1 --2- 172.21.15.22:0/3950437975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b2013f140 0x7f3b200b2b90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400e930 tx=0x7f3b1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.000+0000 7f3b1e7fc640 1 -- 172.21.15.22:0/3950437975 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b14017070 con 0x7f3b2013f140 2024-03-29T04:18:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.000+0000 7f3b1e7fc640 1 -- 172.21.15.22:0/3950437975 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b1400ca00 con 0x7f3b2013f140 2024-03-29T04:18:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.000+0000 7f3b1e7fc640 1 -- 172.21.15.22:0/3950437975 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1400cdb0 con 0x7f3b2013f140 2024-03-29T04:18:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.000+0000 7f3b2820f640 1 -- 172.21.15.22:0/3950437975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b20141ac0 con 0x7f3b2013f140 2024-03-29T04:18:39.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.000+0000 7f3b2820f640 1 -- 172.21.15.22:0/3950437975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b20150a90 con 0x7f3b2013f140 2024-03-29T04:18:39.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.000+0000 7f3b2820f640 1 -- 172.21.15.22:0/3950437975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b20068900 con 0x7f3b2013f140 2024-03-29T04:18:39.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.000+0000 7f3b1e7fc640 1 -- 172.21.15.22:0/3950437975 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f3b1401a070 con 0x7f3b2013f140 2024-03-29T04:18:39.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.000+0000 7f3b1e7fc640 1 --2- 172.21.15.22:0/3950437975 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b0c067fc0 0x7f3b0c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:39.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.000+0000 7f3b1e7fc640 1 -- 172.21.15.22:0/3950437975 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f3b140868d0 con 0x7f3b2013f140 2024-03-29T04:18:39.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.000+0000 7f3b25fab640 1 --2- 172.21.15.22:0/3950437975 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b0c067fc0 0x7f3b0c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:39.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.004+0000 7f3b25fab640 1 --2- 172.21.15.22:0/3950437975 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b0c067fc0 0x7f3b0c06a480 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3b100ae000 tx=0x7f3b100b3040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.008+0000 7f3b1e7fc640 1 -- 172.21.15.22:0/3950437975 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f3b14056dd0 con 0x7f3b2013f140 2024-03-29T04:18:40.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.160+0000 7f3b2820f640 1 -- 172.21.15.22:0/3950437975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3b2006c000 con 0x7f3b2013f140 2024-03-29T04:18:40.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.164+0000 7f3b1e7fc640 1 -- 172.21.15.22:0/3950437975 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f3b14084e20 con 0x7f3b2013f140 2024-03-29T04:18:40.143 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:18:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.164+0000 7f3b2820f640 1 -- 172.21.15.22:0/3950437975 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b0c067fc0 msgr2=0x7f3b0c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.164+0000 7f3b2820f640 1 --2- 172.21.15.22:0/3950437975 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b0c067fc0 0x7f3b0c06a480 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3b100ae000 tx=0x7f3b100b3040 comp rx=0 tx=0).stop 2024-03-29T04:18:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.164+0000 7f3b2820f640 1 -- 172.21.15.22:0/3950437975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b2013f140 msgr2=0x7f3b200b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.164+0000 7f3b2820f640 1 --2- 172.21.15.22:0/3950437975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b2013f140 0x7f3b200b2b90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400e930 tx=0x7f3b1400edf0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.164+0000 7f3b2820f640 1 -- 172.21.15.22:0/3950437975 shutdown_connections 2024-03-29T04:18:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.164+0000 7f3b2820f640 1 --2- 172.21.15.22:0/3950437975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b2015ff40 0x7f3b20151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.164+0000 7f3b2820f640 1 --2- 172.21.15.22:0/3950437975 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b0c067fc0 0x7f3b0c06a480 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.164+0000 7f3b2820f640 1 --2- 172.21.15.22:0/3950437975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b2013f140 0x7f3b200b2b90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.164+0000 7f3b2820f640 1 --2- 172.21.15.22:0/3950437975 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b2006cbb0 0x7f3b2013ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.164+0000 7f3b2820f640 1 -- 172.21.15.22:0/3950437975 >> 172.21.15.22:0/3950437975 conn(0x7f3b200d00a0 msgr2=0x7f3b2015f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.164+0000 7f3b2820f640 1 -- 172.21.15.22:0/3950437975 shutdown_connections 2024-03-29T04:18:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.164+0000 7f3b2820f640 1 -- 172.21.15.22:0/3950437975 wait complete. 2024-03-29T04:18:40.162 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:18:40.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.300+0000 7f259cd4f640 1 Processor -- start 2024-03-29T04:18:40.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.300+0000 7f259cd4f640 1 -- start start 2024-03-29T04:18:40.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.300+0000 7f259cd4f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f259806cd80 0x7f2598166fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:40.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.300+0000 7f259cd4f640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f259806d710 con 0x7f259815ff40 2024-03-29T04:18:40.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.300+0000 7f259cd4f640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f259806d9f0 con 0x7f259813f140 2024-03-29T04:18:40.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.300+0000 7f259cd4f640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f259806e320 con 0x7f259806cd80 2024-03-29T04:18:40.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.300+0000 7f2595d9b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f259815ff40 0x7f259806c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58540/0 (socket says 172.21.15.22:58540) 2024-03-29T04:18:40.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.300+0000 7f2595d9b640 1 -- 172.21.15.22:0/3966177092 learned_addr learned my addr 172.21.15.22:0/3966177092 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:40.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.300+0000 7f2596d9d640 1 --2- 172.21.15.22:0/3966177092 >> v2:172.21.15.22:3300/0 conn(0x7f259806cd80 0x7f2598166fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:40.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.300+0000 7f259559a640 1 -- 172.21.15.22:0/3966177092 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1361548103 0 0) 0x7f259806d710 con 0x7f259815ff40 2024-03-29T04:18:40.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.300+0000 7f259559a640 1 -- 172.21.15.22:0/3966177092 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2580003530 con 0x7f259815ff40 2024-03-29T04:18:40.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.300+0000 7f2596d9d640 1 -- 172.21.15.22:0/3966177092 >> v1:172.21.15.151:6789/0 conn(0x7f259813f140 legacy=0x7f25981606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:40.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.300+0000 7f2596d9d640 1 -- 172.21.15.22:0/3966177092 >> v1:172.21.15.22:6789/0 conn(0x7f259815ff40 legacy=0x7f259806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:40.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.300+0000 7f2596d9d640 1 -- 172.21.15.22:0/3966177092 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259806dcd0 con 0x7f259806cd80 2024-03-29T04:18:40.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f2596d9d640 1 --2- 172.21.15.22:0/3966177092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f259806cd80 0x7f2598166fe0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f25900097c0 tx=0x7f259001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4885c140688f2f22 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:40.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259559a640 1 -- 172.21.15.22:0/3966177092 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25900b1070 con 0x7f259806cd80 2024-03-29T04:18:40.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259559a640 1 -- 172.21.15.22:0/3966177092 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25900ac410 con 0x7f259806cd80 2024-03-29T04:18:40.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259559a640 1 -- 172.21.15.22:0/3966177092 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25900ac7c0 con 0x7f259806cd80 2024-03-29T04:18:40.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259cd4f640 1 -- 172.21.15.22:0/3966177092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f259806cd80 msgr2=0x7f2598166fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:40.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259cd4f640 1 --2- 172.21.15.22:0/3966177092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f259806cd80 0x7f2598166fe0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f25900097c0 tx=0x7f259001c4a0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259cd4f640 1 -- 172.21.15.22:0/3966177092 shutdown_connections 2024-03-29T04:18:40.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259cd4f640 1 --2- 172.21.15.22:0/3966177092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f259806cd80 0x7f2598166fe0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259cd4f640 1 -- 172.21.15.22:0/3966177092 >> 172.21.15.22:0/3966177092 conn(0x7f25980d00a0 msgr2=0x7f25980d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:40.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259cd4f640 1 -- 172.21.15.22:0/3966177092 shutdown_connections 2024-03-29T04:18:40.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259cd4f640 1 -- 172.21.15.22:0/3966177092 wait complete. 2024-03-29T04:18:40.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259cd4f640 1 Processor -- start 2024-03-29T04:18:40.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259cd4f640 1 -- start start 2024-03-29T04:18:40.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259cd4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f259806cd80 0x7f259813a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:40.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259cd4f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f259813f140 0x7f2598072440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:40.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259cd4f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f259815ff40 0x7f2598072980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:40.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259cd4f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f259816a0d0 con 0x7f259806cd80 2024-03-29T04:18:40.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259cd4f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2598169f50 con 0x7f259815ff40 2024-03-29T04:18:40.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259cd4f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f259816a250 con 0x7f259813f140 2024-03-29T04:18:40.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f259806cd80 0x7f259813a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:40.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f259806cd80 0x7f259813a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51322/0 (socket says 172.21.15.22:51322) 2024-03-29T04:18:40.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259659c640 1 -- 172.21.15.22:0/3040742532 learned_addr learned my addr 172.21.15.22:0/3040742532 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:40.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f2596d9d640 1 --2- 172.21.15.22:0/3040742532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f259815ff40 0x7f2598072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:40.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259659c640 1 -- 172.21.15.22:0/3040742532 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f259813f140 msgr2=0x7f2598072440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T04:18:40.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259659c640 1 --2- 172.21.15.22:0/3040742532 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f259813f140 0x7f2598072440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259659c640 1 -- 172.21.15.22:0/3040742532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f259815ff40 msgr2=0x7f2598072980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:40.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259659c640 1 --2- 172.21.15.22:0/3040742532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f259815ff40 0x7f2598072980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259659c640 1 -- 172.21.15.22:0/3040742532 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25981f5fc0 con 0x7f259806cd80 2024-03-29T04:18:40.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f2596d9d640 1 --2- 172.21.15.22:0/3040742532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f259815ff40 0x7f2598072980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:40.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.304+0000 7f259659c640 1 --2- 172.21.15.22:0/3040742532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f259806cd80 0x7f259813a390 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f258800c500 tx=0x7f258800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:40.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.308+0000 7f2586ffd640 1 -- 172.21.15.22:0/3040742532 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588016020 con 0x7f259806cd80 2024-03-29T04:18:40.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.308+0000 7f2586ffd640 1 -- 172.21.15.22:0/3040742532 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2588005220 con 0x7f259806cd80 2024-03-29T04:18:40.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.308+0000 7f259cd4f640 1 -- 172.21.15.22:0/3040742532 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2598154f50 con 0x7f259806cd80 2024-03-29T04:18:40.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.308+0000 7f2586ffd640 1 -- 172.21.15.22:0/3040742532 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588007030 con 0x7f259806cd80 2024-03-29T04:18:40.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.308+0000 7f259cd4f640 1 -- 172.21.15.22:0/3040742532 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2598155510 con 0x7f259806cd80 2024-03-29T04:18:40.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.308+0000 7f259cd4f640 1 -- 172.21.15.22:0/3040742532 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2598155710 con 0x7f259806cd80 2024-03-29T04:18:40.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.308+0000 7f2586ffd640 1 -- 172.21.15.22:0/3040742532 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f25880071f0 con 0x7f259806cd80 2024-03-29T04:18:40.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.308+0000 7f2586ffd640 1 --2- 172.21.15.22:0/3040742532 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2578067ff0 0x7f257806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:40.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.308+0000 7f2586ffd640 1 -- 172.21.15.22:0/3040742532 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f25880862d0 con 0x7f259806cd80 2024-03-29T04:18:40.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.308+0000 7f2595d9b640 1 --2- 172.21.15.22:0/3040742532 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2578067ff0 0x7f257806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:40.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.308+0000 7f2595d9b640 1 --2- 172.21.15.22:0/3040742532 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2578067ff0 0x7f257806a4b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f25980af5e0 tx=0x7f258c0015c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:40.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.312+0000 7f2586ffd640 1 -- 172.21.15.22:0/3040742532 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f2588020020 con 0x7f259806cd80 2024-03-29T04:18:40.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.468+0000 7f259cd4f640 1 -- 172.21.15.22:0/3040742532 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2598068900 con 0x7f259806cd80 2024-03-29T04:18:40.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.468+0000 7f2586ffd640 1 -- 172.21.15.22:0/3040742532 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f2588056690 con 0x7f259806cd80 2024-03-29T04:18:40.450 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:40.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.468+0000 7f259cd4f640 1 -- 172.21.15.22:0/3040742532 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2578067ff0 msgr2=0x7f257806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:40.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.468+0000 7f259cd4f640 1 --2- 172.21.15.22:0/3040742532 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2578067ff0 0x7f257806a4b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f25980af5e0 tx=0x7f258c0015c0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.468+0000 7f259cd4f640 1 -- 172.21.15.22:0/3040742532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f259806cd80 msgr2=0x7f259813a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:40.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.468+0000 7f259cd4f640 1 --2- 172.21.15.22:0/3040742532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f259806cd80 0x7f259813a390 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f258800c500 tx=0x7f258800c9c0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.472+0000 7f259cd4f640 1 -- 172.21.15.22:0/3040742532 shutdown_connections 2024-03-29T04:18:40.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.472+0000 7f259cd4f640 1 --2- 172.21.15.22:0/3040742532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f259815ff40 0x7f2598072980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.472+0000 7f259cd4f640 1 --2- 172.21.15.22:0/3040742532 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f259813f140 0x7f2598072440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.472+0000 7f259cd4f640 1 --2- 172.21.15.22:0/3040742532 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2578067ff0 0x7f257806a4b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.472+0000 7f259cd4f640 1 --2- 172.21.15.22:0/3040742532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f259806cd80 0x7f259813a390 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.472+0000 7f259cd4f640 1 -- 172.21.15.22:0/3040742532 >> 172.21.15.22:0/3040742532 conn(0x7f25980d00a0 msgr2=0x7f259815c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:40.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.472+0000 7f259cd4f640 1 -- 172.21.15.22:0/3040742532 shutdown_connections 2024-03-29T04:18:40.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.472+0000 7f259cd4f640 1 -- 172.21.15.22:0/3040742532 wait complete. 2024-03-29T04:18:40.468 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:18:40.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.612+0000 7fc2acc22640 1 Processor -- start 2024-03-29T04:18:40.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.612+0000 7fc2acc22640 1 -- start start 2024-03-29T04:18:40.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.612+0000 7fc2acc22640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc2a8147e00 0x7fc2a80b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:40.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.612+0000 7fc2acc22640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2a80b5470 con 0x7fc2a80b4f30 2024-03-29T04:18:40.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.612+0000 7fc2acc22640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2a80b5750 con 0x7fc2a806c780 2024-03-29T04:18:40.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.612+0000 7fc2acc22640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fc2a80b5a30 con 0x7fc2a8147e00 2024-03-29T04:18:40.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.612+0000 7fc29ffff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fc2a80b4f30 0x7fc2a806c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58542/0 (socket says 172.21.15.22:58542) 2024-03-29T04:18:40.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.612+0000 7fc2a659c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc2a8147e00 0x7fc2a80b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:40.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.612+0000 7fc29ffff640 1 -- 172.21.15.22:0/3531199710 learned_addr learned my addr 172.21.15.22:0/3531199710 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:40.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.612+0000 7fc2a659c640 1 -- 172.21.15.22:0/3531199710 >> v1:172.21.15.151:6789/0 conn(0x7fc2a806c780 legacy=0x7fc2a80adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:40.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.612+0000 7fc2a659c640 1 -- 172.21.15.22:0/3531199710 >> v1:172.21.15.22:6789/0 conn(0x7fc2a80b4f30 legacy=0x7fc2a806c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:40.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.612+0000 7fc2a5d9b640 1 -- 172.21.15.22:0/3531199710 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 882227703 0 0) 0x7fc2a80b5470 con 0x7fc2a80b4f30 2024-03-29T04:18:40.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.612+0000 7fc2a659c640 1 -- 172.21.15.22:0/3531199710 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a80b5d10 con 0x7fc2a8147e00 2024-03-29T04:18:40.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.616+0000 7fc2a659c640 1 --2- 172.21.15.22:0/3531199710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a8147e00 0x7fc2a80b49f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc2980097c0 tx=0x7fc29801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f566805c63543e18 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:40.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.616+0000 7fc2a5d9b640 1 -- 172.21.15.22:0/3531199710 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2980b0070 con 0x7fc2a8147e00 2024-03-29T04:18:40.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.616+0000 7fc2a5d9b640 1 -- 172.21.15.22:0/3531199710 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2980ab600 con 0x7fc2a8147e00 2024-03-29T04:18:40.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.616+0000 7fc2a5d9b640 1 -- 172.21.15.22:0/3531199710 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2980ab9d0 con 0x7fc2a8147e00 2024-03-29T04:18:40.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.616+0000 7fc2acc22640 1 -- 172.21.15.22:0/3531199710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a8147e00 msgr2=0x7fc2a80b49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:40.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.616+0000 7fc2acc22640 1 --2- 172.21.15.22:0/3531199710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a8147e00 0x7fc2a80b49f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc2980097c0 tx=0x7fc29801c4a0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.616+0000 7fc2acc22640 1 -- 172.21.15.22:0/3531199710 shutdown_connections 2024-03-29T04:18:40.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.616+0000 7fc2acc22640 1 --2- 172.21.15.22:0/3531199710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a8147e00 0x7fc2a80b49f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.616+0000 7fc2acc22640 1 -- 172.21.15.22:0/3531199710 >> 172.21.15.22:0/3531199710 conn(0x7fc2a80d1070 msgr2=0x7fc2a80d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:40.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.616+0000 7fc2acc22640 1 -- 172.21.15.22:0/3531199710 shutdown_connections 2024-03-29T04:18:40.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.616+0000 7fc2acc22640 1 -- 172.21.15.22:0/3531199710 wait complete. 2024-03-29T04:18:40.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.616+0000 7fc2acc22640 1 Processor -- start 2024-03-29T04:18:40.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.616+0000 7fc2acc22640 1 -- start start 2024-03-29T04:18:40.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc2acc22640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2a806c780 0x7fc2a815b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:40.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc2acc22640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc2a80b4f30 0x7fc2a81607d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:40.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc2acc22640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a8147e00 0x7fc2a8160d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:40.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc2acc22640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a80aeeb0 con 0x7fc2a8147e00 2024-03-29T04:18:40.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc2acc22640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a80aed30 con 0x7fc2a806c780 2024-03-29T04:18:40.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc2acc22640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc2a816c130 con 0x7fc2a80b4f30 2024-03-29T04:18:40.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc2a6d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a8147e00 0x7fc2a8160d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:40.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc2a6d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a8147e00 0x7fc2a8160d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51334/0 (socket says 172.21.15.22:51334) 2024-03-29T04:18:40.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc2a6d9d640 1 -- 172.21.15.22:0/1116418218 learned_addr learned my addr 172.21.15.22:0/1116418218 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:40.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc2a659c640 1 --2- 172.21.15.22:0/1116418218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2a806c780 0x7fc2a815b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:40.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc29ffff640 1 --2- 172.21.15.22:0/1116418218 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc2a80b4f30 0x7fc2a81607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:40.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc2a6d9d640 1 -- 172.21.15.22:0/1116418218 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc2a80b4f30 msgr2=0x7fc2a81607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:40.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc2a6d9d640 1 --2- 172.21.15.22:0/1116418218 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc2a80b4f30 0x7fc2a81607d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc2a6d9d640 1 -- 172.21.15.22:0/1116418218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2a806c780 msgr2=0x7fc2a815b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:40.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc2a6d9d640 1 --2- 172.21.15.22:0/1116418218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2a806c780 0x7fc2a815b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc2a6d9d640 1 -- 172.21.15.22:0/1116418218 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a815bdf0 con 0x7fc2a8147e00 2024-03-29T04:18:40.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc2a6d9d640 1 --2- 172.21.15.22:0/1116418218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a8147e00 0x7fc2a8160d10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000c7d0 tx=0x7fc2a000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:40.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc29effd640 1 -- 172.21.15.22:0/1116418218 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0003b80 con 0x7fc2a8147e00 2024-03-29T04:18:40.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc29effd640 1 -- 172.21.15.22:0/1116418218 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a0003d20 con 0x7fc2a8147e00 2024-03-29T04:18:40.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc2acc22640 1 -- 172.21.15.22:0/1116418218 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a81691a0 con 0x7fc2a8147e00 2024-03-29T04:18:40.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc29effd640 1 -- 172.21.15.22:0/1116418218 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a000aa30 con 0x7fc2a8147e00 2024-03-29T04:18:40.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc2acc22640 1 -- 172.21.15.22:0/1116418218 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a815ceb0 con 0x7fc2a8147e00 2024-03-29T04:18:40.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc2acc22640 1 -- 172.21.15.22:0/1116418218 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a8068900 con 0x7fc2a8147e00 2024-03-29T04:18:40.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc29effd640 1 -- 172.21.15.22:0/1116418218 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fc2a000abd0 con 0x7fc2a8147e00 2024-03-29T04:18:40.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc29effd640 1 --2- 172.21.15.22:0/1116418218 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc288067fc0 0x7fc28806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:40.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc29effd640 1 -- 172.21.15.22:0/1116418218 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fc2a0086830 con 0x7fc2a8147e00 2024-03-29T04:18:40.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.620+0000 7fc2a659c640 1 --2- 172.21.15.22:0/1116418218 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc288067fc0 0x7fc28806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:40.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.624+0000 7fc2a659c640 1 --2- 172.21.15.22:0/1116418218 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc288067fc0 0x7fc28806a480 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc2980ad000 tx=0x7fc2980a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:40.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.628+0000 7fc29effd640 1 -- 172.21.15.22:0/1116418218 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fc2a0056b40 con 0x7fc2a8147e00 2024-03-29T04:18:40.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.780+0000 7fc2acc22640 1 -- 172.21.15.22:0/1116418218 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc2a806bd80 con 0x7fc2a8147e00 2024-03-29T04:18:40.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.784+0000 7fc29effd640 1 -- 172.21.15.22:0/1116418218 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fc2a0056440 con 0x7fc2a8147e00 2024-03-29T04:18:40.763 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-29T04:18:40.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.784+0000 7fc2acc22640 1 -- 172.21.15.22:0/1116418218 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc288067fc0 msgr2=0x7fc28806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:40.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.784+0000 7fc2acc22640 1 --2- 172.21.15.22:0/1116418218 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc288067fc0 0x7fc28806a480 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc2980ad000 tx=0x7fc2980a6000 comp rx=0 tx=0).stop 2024-03-29T04:18:40.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.784+0000 7fc2acc22640 1 -- 172.21.15.22:0/1116418218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a8147e00 msgr2=0x7fc2a8160d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:40.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.784+0000 7fc2acc22640 1 --2- 172.21.15.22:0/1116418218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a8147e00 0x7fc2a8160d10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000c7d0 tx=0x7fc2a000cc90 comp rx=0 tx=0).stop 2024-03-29T04:18:40.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.784+0000 7fc2acc22640 1 -- 172.21.15.22:0/1116418218 shutdown_connections 2024-03-29T04:18:40.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.784+0000 7fc2acc22640 1 --2- 172.21.15.22:0/1116418218 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc288067fc0 0x7fc28806a480 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.784+0000 7fc2acc22640 1 --2- 172.21.15.22:0/1116418218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a8147e00 0x7fc2a8160d10 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.784+0000 7fc2acc22640 1 --2- 172.21.15.22:0/1116418218 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc2a80b4f30 0x7fc2a81607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.784+0000 7fc2acc22640 1 --2- 172.21.15.22:0/1116418218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2a806c780 0x7fc2a815b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.784+0000 7fc2acc22640 1 -- 172.21.15.22:0/1116418218 >> 172.21.15.22:0/1116418218 conn(0x7fc2a80d1070 msgr2=0x7fc2a8169920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:40.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.784+0000 7fc2acc22640 1 -- 172.21.15.22:0/1116418218 shutdown_connections 2024-03-29T04:18:40.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.784+0000 7fc2acc22640 1 -- 172.21.15.22:0/1116418218 wait complete. 2024-03-29T04:18:40.781 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:18:40.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.916+0000 7f1e44b3b640 1 Processor -- start 2024-03-29T04:18:40.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.916+0000 7f1e44b3b640 1 -- start start 2024-03-29T04:18:40.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.916+0000 7f1e44b3b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1e4013f140 0x7f1e401606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:40.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.916+0000 7f1e44b3b640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4015ff40 0x7f1e4006c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:40.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.916+0000 7f1e44b3b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e4006d710 con 0x7f1e4006cbb0 2024-03-29T04:18:40.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.916+0000 7f1e44b3b640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f1e4006d000 con 0x7f1e4013f140 2024-03-29T04:18:40.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.916+0000 7f1e44b3b640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1e4006e320 con 0x7f1e4015ff40 2024-03-29T04:18:40.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.916+0000 7f1e3e59c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1e4013f140 0x7f1e401606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:40.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.916+0000 7f1e3ed9d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1e4006cbb0 0x7f1e40141a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58558/0 (socket says 172.21.15.22:58558) 2024-03-29T04:18:40.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.916+0000 7f1e3ed9d640 1 -- 172.21.15.22:0/334377113 learned_addr learned my addr 172.21.15.22:0/334377113 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:40.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.916+0000 7f1e3e59c640 1 -- 172.21.15.22:0/334377113 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4015ff40 msgr2=0x7f1e4006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:40.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.916+0000 7f1e3e59c640 1 --2- 172.21.15.22:0/334377113 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4015ff40 0x7f1e4006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.916+0000 7f1e3e59c640 1 -- 172.21.15.22:0/334377113 >> v1:172.21.15.22:6789/0 conn(0x7f1e4006cbb0 legacy=0x7f1e40141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:40.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.916+0000 7f1e3e59c640 1 -- 172.21.15.22:0/334377113 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4006d9f0 con 0x7f1e4013f140 2024-03-29T04:18:40.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.916+0000 7f1e3d59a640 1 -- 172.21.15.22:0/334377113 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1825045085 0 0) 0x7f1e4006d710 con 0x7f1e4006cbb0 2024-03-29T04:18:40.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.920+0000 7f1e3e59c640 1 --2- 172.21.15.22:0/334377113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4013f140 0x7f1e401606d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1e300097c0 tx=0x7f1e3001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26923d91fbc82d11 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:40.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.920+0000 7f1e3d59a640 1 -- 172.21.15.22:0/334377113 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e300a7070 con 0x7f1e4013f140 2024-03-29T04:18:40.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.920+0000 7f1e3d59a640 1 -- 172.21.15.22:0/334377113 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e300b0070 con 0x7f1e4013f140 2024-03-29T04:18:40.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.920+0000 7f1e3d59a640 1 -- 172.21.15.22:0/334377113 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e300ab890 con 0x7f1e4013f140 2024-03-29T04:18:40.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.920+0000 7f1e44b3b640 1 -- 172.21.15.22:0/334377113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4013f140 msgr2=0x7f1e401606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:40.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.920+0000 7f1e44b3b640 1 --2- 172.21.15.22:0/334377113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4013f140 0x7f1e401606d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1e300097c0 tx=0x7f1e3001c470 comp rx=0 tx=0).stop 2024-03-29T04:18:40.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.920+0000 7f1e44b3b640 1 -- 172.21.15.22:0/334377113 shutdown_connections 2024-03-29T04:18:40.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.920+0000 7f1e44b3b640 1 --2- 172.21.15.22:0/334377113 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4015ff40 0x7f1e4006c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.920+0000 7f1e44b3b640 1 --2- 172.21.15.22:0/334377113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4013f140 0x7f1e401606d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.920+0000 7f1e44b3b640 1 -- 172.21.15.22:0/334377113 >> 172.21.15.22:0/334377113 conn(0x7f1e400d00a0 msgr2=0x7f1e400d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:40.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.920+0000 7f1e44b3b640 1 -- 172.21.15.22:0/334377113 shutdown_connections 2024-03-29T04:18:40.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.920+0000 7f1e44b3b640 1 -- 172.21.15.22:0/334377113 wait complete. 2024-03-29T04:18:40.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.920+0000 7f1e44b3b640 1 Processor -- start 2024-03-29T04:18:40.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.920+0000 7f1e44b3b640 1 -- start start 2024-03-29T04:18:40.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e44b3b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4006cbb0 0x7f1e40153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:40.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e44b3b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e4013f140 0x7f1e4015b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:40.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e44b3b640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4015ff40 0x7f1e400b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:40.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e44b3b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1e40160d90 con 0x7f1e4006cbb0 2024-03-29T04:18:40.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e44b3b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e40160c10 con 0x7f1e4013f140 2024-03-29T04:18:40.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e44b3b640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f1e40168ff0 con 0x7f1e4015ff40 2024-03-29T04:18:40.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e3e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4006cbb0 0x7f1e40153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:40.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e3e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4006cbb0 0x7f1e40153950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51346/0 (socket says 172.21.15.22:51346) 2024-03-29T04:18:40.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e3e59c640 1 -- 172.21.15.22:0/1389329183 learned_addr learned my addr 172.21.15.22:0/1389329183 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:40.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e3dd9b640 1 --2- 172.21.15.22:0/1389329183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e4013f140 0x7f1e4015b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:40.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e3ed9d640 1 --2- 172.21.15.22:0/1389329183 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4015ff40 0x7f1e400b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:40.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e3e59c640 1 -- 172.21.15.22:0/1389329183 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4015ff40 msgr2=0x7f1e400b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:40.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e3e59c640 1 --2- 172.21.15.22:0/1389329183 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4015ff40 0x7f1e400b3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e3e59c640 1 -- 172.21.15.22:0/1389329183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e4013f140 msgr2=0x7f1e4015b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:40.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e3e59c640 1 --2- 172.21.15.22:0/1389329183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e4013f140 0x7f1e4015b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:40.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e3e59c640 1 -- 172.21.15.22:0/1389329183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e40153e90 con 0x7f1e4006cbb0 2024-03-29T04:18:40.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e3e59c640 1 --2- 172.21.15.22:0/1389329183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4006cbb0 0x7f1e40153950 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1e300ad040 tx=0x7f1e3001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:40.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e2effd640 1 -- 172.21.15.22:0/1389329183 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e300bf610 con 0x7f1e4006cbb0 2024-03-29T04:18:40.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e2effd640 1 -- 172.21.15.22:0/1389329183 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e300bf7b0 con 0x7f1e4006cbb0 2024-03-29T04:18:40.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e44b3b640 1 -- 172.21.15.22:0/1389329183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e40140c30 con 0x7f1e4006cbb0 2024-03-29T04:18:40.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e2effd640 1 -- 172.21.15.22:0/1389329183 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e300bfba0 con 0x7f1e4006cbb0 2024-03-29T04:18:40.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e44b3b640 1 -- 172.21.15.22:0/1389329183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e401411f0 con 0x7f1e4006cbb0 2024-03-29T04:18:40.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e44b3b640 1 -- 172.21.15.22:0/1389329183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e40068900 con 0x7f1e4006cbb0 2024-03-29T04:18:40.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e2effd640 1 -- 172.21.15.22:0/1389329183 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1e300bd020 con 0x7f1e4006cbb0 2024-03-29T04:18:40.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e2effd640 1 --2- 172.21.15.22:0/1389329183 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1e20067fc0 0x7f1e2006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:40.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e2effd640 1 -- 172.21.15.22:0/1389329183 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f1e30121320 con 0x7f1e4006cbb0 2024-03-29T04:18:40.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.924+0000 7f1e3dd9b640 1 --2- 172.21.15.22:0/1389329183 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1e20067fc0 0x7f1e2006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:40.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.928+0000 7f1e3dd9b640 1 --2- 172.21.15.22:0/1389329183 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1e20067fc0 0x7f1e2006a480 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1e40072b50 tx=0x7f1e34006f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:40.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:40.928+0000 7f1e2effd640 1 -- 172.21.15.22:0/1389329183 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1e30016090 con 0x7f1e4006cbb0 2024-03-29T04:18:41.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.092+0000 7f1e44b3b640 1 -- 172.21.15.22:0/1389329183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1e40160800 con 0x7f1e4006cbb0 2024-03-29T04:18:41.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.092+0000 7f1e2effd640 1 -- 172.21.15.22:0/1389329183 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f1e300f1080 con 0x7f1e4006cbb0 2024-03-29T04:18:41.073 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:41.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.096+0000 7f1e44b3b640 1 -- 172.21.15.22:0/1389329183 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1e20067fc0 msgr2=0x7f1e2006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:41.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.096+0000 7f1e44b3b640 1 --2- 172.21.15.22:0/1389329183 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1e20067fc0 0x7f1e2006a480 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1e40072b50 tx=0x7f1e34006f90 comp rx=0 tx=0).stop 2024-03-29T04:18:41.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.096+0000 7f1e44b3b640 1 -- 172.21.15.22:0/1389329183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4006cbb0 msgr2=0x7f1e40153950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:41.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.096+0000 7f1e44b3b640 1 --2- 172.21.15.22:0/1389329183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4006cbb0 0x7f1e40153950 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1e300ad040 tx=0x7f1e3001acb0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.096+0000 7f1e44b3b640 1 -- 172.21.15.22:0/1389329183 shutdown_connections 2024-03-29T04:18:41.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.096+0000 7f1e44b3b640 1 --2- 172.21.15.22:0/1389329183 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f1e4015ff40 0x7f1e400b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.096+0000 7f1e44b3b640 1 --2- 172.21.15.22:0/1389329183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e4013f140 0x7f1e4015b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.096+0000 7f1e44b3b640 1 --2- 172.21.15.22:0/1389329183 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1e20067fc0 0x7f1e2006a480 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.096+0000 7f1e44b3b640 1 --2- 172.21.15.22:0/1389329183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e4006cbb0 0x7f1e40153950 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.096+0000 7f1e44b3b640 1 -- 172.21.15.22:0/1389329183 >> 172.21.15.22:0/1389329183 conn(0x7f1e400d00a0 msgr2=0x7f1e400b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:41.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.096+0000 7f1e44b3b640 1 -- 172.21.15.22:0/1389329183 shutdown_connections 2024-03-29T04:18:41.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.096+0000 7f1e44b3b640 1 -- 172.21.15.22:0/1389329183 wait complete. 2024-03-29T04:18:41.093 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:18:41.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c042b1640 1 Processor -- start 2024-03-29T04:18:41.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c042b1640 1 -- start start 2024-03-29T04:18:41.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c042b1640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7bfc06f9e0 0x7f7bfc06fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:41.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c042b1640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f7bfc06c770 0x7f7bfc06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:41.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c042b1640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bfc070300 con 0x7f7bfc0b4530 2024-03-29T04:18:41.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c042b1640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f7bfc152550 con 0x7f7bfc06c770 2024-03-29T04:18:41.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c042b1640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7bfc06d430 con 0x7f7bfc06f9e0 2024-03-29T04:18:41.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0284e640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f7bfc0b4530 0x7f7bfc0b4980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58574/0 (socket says 172.21.15.22:58574) 2024-03-29T04:18:41.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0284e640 1 -- 172.21.15.22:0/86396554 learned_addr learned my addr 172.21.15.22:0/86396554 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:41.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0204d640 1 --2- 172.21.15.22:0/86396554 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7bfc06f9e0 0x7f7bfc06fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:41.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0184c640 1 --2- 172.21.15.22:0/86396554 >> v2:172.21.15.151:3300/0 conn(0x7f7bfc06c770 0x7f7bfc06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:41.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0104b640 1 -- 172.21.15.22:0/86396554 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2990924522 0 0) 0x7f7bfc070300 con 0x7f7bfc0b4530 2024-03-29T04:18:41.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0104b640 1 -- 172.21.15.22:0/86396554 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7be4003530 con 0x7f7bfc0b4530 2024-03-29T04:18:41.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0104b640 1 -- 172.21.15.22:0/86396554 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2145253591 0 0) 0x7f7be4003530 con 0x7f7bfc0b4530 2024-03-29T04:18:41.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0104b640 1 -- 172.21.15.22:0/86396554 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7bfc06f9e0 msgr2=0x7f7bfc06fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:41.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0104b640 1 --2- 172.21.15.22:0/86396554 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7bfc06f9e0 0x7f7bfc06fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0104b640 1 -- 172.21.15.22:0/86396554 >> v2:172.21.15.151:3300/0 conn(0x7f7bfc06c770 msgr2=0x7f7bfc06cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:41.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0104b640 1 --2- 172.21.15.22:0/86396554 >> v2:172.21.15.151:3300/0 conn(0x7f7bfc06c770 0x7f7bfc06cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0104b640 1 -- 172.21.15.22:0/86396554 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bfc06d710 con 0x7f7bfc0b4530 2024-03-29T04:18:41.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0104b640 1 -- 172.21.15.22:0/86396554 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f7bec001680 con 0x7f7bfc0b4530 2024-03-29T04:18:41.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0104b640 1 -- 172.21.15.22:0/86396554 >> v1:172.21.15.22:6789/0 conn(0x7f7bfc0b4530 legacy=0x7f7bfc0b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:41.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0104b640 1 --2- 172.21.15.22:0/86396554 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7bfc06f9e0 0x7f7be40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:41.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0104b640 1 --2- 172.21.15.22:0/86396554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7be4003f40 0x7f7be4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:41.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0104b640 1 --2- 172.21.15.22:0/86396554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be4004d50 0x7f7be40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:41.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0104b640 1 -- 172.21.15.22:0/86396554 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc152550 con 0x7f7be4004d50 2024-03-29T04:18:41.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0104b640 1 -- 172.21.15.22:0/86396554 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc06d430 con 0x7f7be4003f40 2024-03-29T04:18:41.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0104b640 1 -- 172.21.15.22:0/86396554 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7be400b9d0 con 0x7f7bfc06f9e0 2024-03-29T04:18:41.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0284e640 1 --2- 172.21.15.22:0/86396554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be4004d50 0x7f7be40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:41.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0204d640 1 --2- 172.21.15.22:0/86396554 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7bfc06f9e0 0x7f7be40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:41.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0184c640 1 --2- 172.21.15.22:0/86396554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7be4003f40 0x7f7be4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:41.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0284e640 1 -- 172.21.15.22:0/86396554 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7bfc06f9e0 msgr2=0x7f7be40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:41.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0284e640 1 --2- 172.21.15.22:0/86396554 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7bfc06f9e0 0x7f7be40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0284e640 1 -- 172.21.15.22:0/86396554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7be4003f40 msgr2=0x7f7be4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:41.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0284e640 1 --2- 172.21.15.22:0/86396554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7be4003f40 0x7f7be4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0284e640 1 -- 172.21.15.22:0/86396554 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be4003530 con 0x7f7be4004d50 2024-03-29T04:18:41.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.240+0000 7f7c0184c640 1 --2- 172.21.15.22:0/86396554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7be4003f40 0x7f7be4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:41.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.244+0000 7f7c0284e640 1 --2- 172.21.15.22:0/86396554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be4004d50 0x7f7be40051d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00c240 tx=0x7f7bec01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6b5e7bbc6531e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:41.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.244+0000 7f7c0104b640 1 -- 172.21.15.22:0/86396554 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec007d40 con 0x7f7be4004d50 2024-03-29T04:18:41.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.244+0000 7f7c0104b640 1 -- 172.21.15.22:0/86396554 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bec0a9020 con 0x7f7be4004d50 2024-03-29T04:18:41.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.244+0000 7f7c0104b640 1 -- 172.21.15.22:0/86396554 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec0a5bc0 con 0x7f7be4004d50 2024-03-29T04:18:41.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.244+0000 7f7c042b1640 1 -- 172.21.15.22:0/86396554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be4004d50 msgr2=0x7f7be40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:41.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.244+0000 7f7c042b1640 1 --2- 172.21.15.22:0/86396554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be4004d50 0x7f7be40051d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00c240 tx=0x7f7bec01cc40 comp rx=0 tx=0).stop 2024-03-29T04:18:41.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.244+0000 7f7c0284e640 1 -- 172.21.15.22:0/86396554 reap_dead start 2024-03-29T04:18:41.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.244+0000 7f7c042b1640 1 -- 172.21.15.22:0/86396554 shutdown_connections 2024-03-29T04:18:41.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.244+0000 7f7c042b1640 1 -- 172.21.15.22:0/86396554 >> 172.21.15.22:0/86396554 conn(0x7f7bfc06e410 msgr2=0x7f7bfc06e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:41.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.244+0000 7f7c042b1640 1 -- 172.21.15.22:0/86396554 shutdown_connections 2024-03-29T04:18:41.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.244+0000 7f7c042b1640 1 -- 172.21.15.22:0/86396554 wait complete. 2024-03-29T04:18:41.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.244+0000 7f7c042b1640 1 Processor -- start 2024-03-29T04:18:41.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.244+0000 7f7c042b1640 1 -- start start 2024-03-29T04:18:41.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7c042b1640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7bfc142ec0 0x7f7bfc156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:41.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7c042b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7bfc158820 0x7f7bfc1f2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:41.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7c042b1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bfc1f26a0 0x7f7bfc15a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:41.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7c042b1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc0b4010 con 0x7f7bfc158820 2024-03-29T04:18:41.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7c042b1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc06dcd0 con 0x7f7bfc1f26a0 2024-03-29T04:18:41.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7c042b1640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7bfc06d130 con 0x7f7bfc142ec0 2024-03-29T04:18:41.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7c0184c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7bfc158820 0x7f7bfc1f2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:41.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7c0184c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7bfc158820 0x7f7bfc1f2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51368/0 (socket says 172.21.15.22:51368) 2024-03-29T04:18:41.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7c0184c640 1 -- 172.21.15.22:0/2574492312 learned_addr learned my addr 172.21.15.22:0/2574492312 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:41.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7c0284e640 1 --2- 172.21.15.22:0/2574492312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bfc1f26a0 0x7f7bfc15a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:41.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7c0204d640 1 --2- 172.21.15.22:0/2574492312 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7bfc142ec0 0x7f7bfc156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:41.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7c0184c640 1 -- 172.21.15.22:0/2574492312 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7bfc142ec0 msgr2=0x7f7bfc156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:41.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7c0184c640 1 --2- 172.21.15.22:0/2574492312 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7bfc142ec0 0x7f7bfc156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7c0184c640 1 -- 172.21.15.22:0/2574492312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bfc1f26a0 msgr2=0x7f7bfc15a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:41.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7c0184c640 1 --2- 172.21.15.22:0/2574492312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bfc1f26a0 0x7f7bfc15a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7c0184c640 1 -- 172.21.15.22:0/2574492312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bfc157220 con 0x7f7bfc158820 2024-03-29T04:18:41.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7c0184c640 1 --2- 172.21.15.22:0/2574492312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7bfc158820 0x7f7bfc1f2160 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000f400 tx=0x7f7bf000f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:41.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7bfaffd640 1 -- 172.21.15.22:0/2574492312 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf00039a0 con 0x7f7bfc158820 2024-03-29T04:18:41.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7bfaffd640 1 -- 172.21.15.22:0/2574492312 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bf0003b40 con 0x7f7bfc158820 2024-03-29T04:18:41.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7bfaffd640 1 -- 172.21.15.22:0/2574492312 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0004020 con 0x7f7bfc158820 2024-03-29T04:18:41.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7c042b1640 1 -- 172.21.15.22:0/2574492312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bfc1f1980 con 0x7f7bfc158820 2024-03-29T04:18:41.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7c042b1640 1 -- 172.21.15.22:0/2574492312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bfc1577a0 con 0x7f7bfc158820 2024-03-29T04:18:41.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7c042b1640 1 -- 172.21.15.22:0/2574492312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bfc068890 con 0x7f7bfc158820 2024-03-29T04:18:41.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7bfaffd640 1 -- 172.21.15.22:0/2574492312 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f7bf001d030 con 0x7f7bfc158820 2024-03-29T04:18:41.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7bfaffd640 1 --2- 172.21.15.22:0/2574492312 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7be8067fc0 0x7f7be806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:41.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7bfaffd640 1 -- 172.21.15.22:0/2574492312 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f7bf0087530 con 0x7f7bfc158820 2024-03-29T04:18:41.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.248+0000 7f7c0204d640 1 --2- 172.21.15.22:0/2574492312 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7be8067fc0 0x7f7be806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:41.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.252+0000 7f7c0204d640 1 --2- 172.21.15.22:0/2574492312 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7be8067fc0 0x7f7be806a480 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009a30 tx=0x7f7bf4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:41.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.252+0000 7f7bfaffd640 1 -- 172.21.15.22:0/2574492312 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f7bf0024020 con 0x7f7bfc158820 2024-03-29T04:18:41.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.424+0000 7f7c042b1640 1 -- 172.21.15.22:0/2574492312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7bfc070de0 con 0x7f7bfc158820 2024-03-29T04:18:41.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.424+0000 7f7bfaffd640 1 -- 172.21.15.22:0/2574492312 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f7bf00578d0 con 0x7f7bfc158820 2024-03-29T04:18:41.406 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-03-29T04:18:41.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.428+0000 7f7c042b1640 1 -- 172.21.15.22:0/2574492312 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7be8067fc0 msgr2=0x7f7be806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:41.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.428+0000 7f7c042b1640 1 --2- 172.21.15.22:0/2574492312 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7be8067fc0 0x7f7be806a480 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009a30 tx=0x7f7bf4008040 comp rx=0 tx=0).stop 2024-03-29T04:18:41.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.428+0000 7f7c042b1640 1 -- 172.21.15.22:0/2574492312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7bfc158820 msgr2=0x7f7bfc1f2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:41.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.428+0000 7f7c042b1640 1 --2- 172.21.15.22:0/2574492312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7bfc158820 0x7f7bfc1f2160 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000f400 tx=0x7f7bf000f8c0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.428+0000 7f7c042b1640 1 -- 172.21.15.22:0/2574492312 shutdown_connections 2024-03-29T04:18:41.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.428+0000 7f7c042b1640 1 --2- 172.21.15.22:0/2574492312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bfc1f26a0 0x7f7bfc15a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.428+0000 7f7c042b1640 1 --2- 172.21.15.22:0/2574492312 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7be8067fc0 0x7f7be806a480 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.428+0000 7f7c042b1640 1 --2- 172.21.15.22:0/2574492312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7bfc158820 0x7f7bfc1f2160 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.428+0000 7f7c042b1640 1 --2- 172.21.15.22:0/2574492312 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7bfc142ec0 0x7f7bfc156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.428+0000 7f7c042b1640 1 -- 172.21.15.22:0/2574492312 >> 172.21.15.22:0/2574492312 conn(0x7f7bfc06e410 msgr2=0x7f7bfc06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:41.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.428+0000 7f7c042b1640 1 -- 172.21.15.22:0/2574492312 shutdown_connections 2024-03-29T04:18:41.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.428+0000 7f7c042b1640 1 -- 172.21.15.22:0/2574492312 wait complete. 2024-03-29T04:18:41.423 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:18:41.424 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:18:41.430 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:41.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.572+0000 7fa72b6c9640 1 Processor -- start 2024-03-29T04:18:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.572+0000 7fa72b6c9640 1 -- start start 2024-03-29T04:18:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.572+0000 7fa72b6c9640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa72406c670 0x7fa7240cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.572+0000 7fa72b6c9640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fa72406ca50 0x7fa7240ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.572+0000 7fa72b6c9640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7240ccef0 con 0x7fa7240b41f0 2024-03-29T04:18:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.572+0000 7fa72b6c9640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fa7240b34c0 con 0x7fa72406ca50 2024-03-29T04:18:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.572+0000 7fa72b6c9640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa7240b3710 con 0x7fa72406c670 2024-03-29T04:18:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.572+0000 7fa729c66640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fa72406ca50 0x7fa7240ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.572+0000 7fa728c64640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7fa7240b41f0 0x7fa72406e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.22:48290/0 (socket says 172.21.15.22:48290) 2024-03-29T04:18:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.572+0000 7fa728c64640 1 -- 172.21.15.22:0/301835417 learned_addr learned my addr 172.21.15.22:0/301835417 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.572+0000 7fa729465640 1 --2- 172.21.15.22:0/301835417 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa72406c670 0x7fa7240cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.572+0000 7fa729c66640 1 -- 172.21.15.22:0/301835417 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa72406c670 msgr2=0x7fa7240cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.572+0000 7fa729c66640 1 --2- 172.21.15.22:0/301835417 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa72406c670 0x7fa7240cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.572+0000 7fa729c66640 1 -- 172.21.15.22:0/301835417 >> v1:172.21.15.151:6789/0 conn(0x7fa7240b41f0 legacy=0x7fa72406e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.572+0000 7fa729c66640 1 -- 172.21.15.22:0/301835417 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7240b39f0 con 0x7fa72406ca50 2024-03-29T04:18:41.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.576+0000 7fa729c66640 1 --2- 172.21.15.22:0/301835417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa72406ca50 0x7fa7240ce5b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009d90 tx=0x7fa71c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4314c9cb076aec server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:41.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.576+0000 7fa71bfff640 1 -- 172.21.15.22:0/301835417 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71c0b2070 con 0x7fa72406ca50 2024-03-29T04:18:41.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.576+0000 7fa71bfff640 1 -- 172.21.15.22:0/301835417 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa71c0a5940 con 0x7fa72406ca50 2024-03-29T04:18:41.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.576+0000 7fa71bfff640 1 -- 172.21.15.22:0/301835417 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71c0a5d10 con 0x7fa72406ca50 2024-03-29T04:18:41.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.576+0000 7fa72b6c9640 1 -- 172.21.15.22:0/301835417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa72406ca50 msgr2=0x7fa7240ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:41.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.576+0000 7fa72b6c9640 1 --2- 172.21.15.22:0/301835417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa72406ca50 0x7fa7240ce5b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009d90 tx=0x7fa71c01c970 comp rx=0 tx=0).stop 2024-03-29T04:18:41.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.576+0000 7fa72b6c9640 1 -- 172.21.15.22:0/301835417 shutdown_connections 2024-03-29T04:18:41.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.576+0000 7fa72b6c9640 1 --2- 172.21.15.22:0/301835417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa72406ca50 0x7fa7240ce5b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.576+0000 7fa72b6c9640 1 --2- 172.21.15.22:0/301835417 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa72406c670 0x7fa7240cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.576+0000 7fa72b6c9640 1 -- 172.21.15.22:0/301835417 >> 172.21.15.22:0/301835417 conn(0x7fa7240cfee0 msgr2=0x7fa7240d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:41.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.576+0000 7fa72b6c9640 1 -- 172.21.15.22:0/301835417 shutdown_connections 2024-03-29T04:18:41.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.576+0000 7fa72b6c9640 1 -- 172.21.15.22:0/301835417 wait complete. 2024-03-29T04:18:41.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.576+0000 7fa72b6c9640 1 Processor -- start 2024-03-29T04:18:41.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.576+0000 7fa72b6c9640 1 -- start start 2024-03-29T04:18:41.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa72b6c9640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa72406c670 0x7fa72413d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:41.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa72b6c9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa72406ca50 0x7fa724134360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:41.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa72b6c9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7240b41f0 0x7fa72413efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:41.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa72b6c9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa724169070 con 0x7fa72406ca50 2024-03-29T04:18:41.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa72b6c9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa724168ef0 con 0x7fa7240b41f0 2024-03-29T04:18:41.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa72b6c9640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa7241691f0 con 0x7fa72406c670 2024-03-29T04:18:41.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa728c64640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa72406ca50 0x7fa724134360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:41.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa728c64640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa72406ca50 0x7fa724134360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51372/0 (socket says 172.21.15.22:51372) 2024-03-29T04:18:41.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa728c64640 1 -- 172.21.15.22:0/1480234759 learned_addr learned my addr 172.21.15.22:0/1480234759 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:41.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa729c66640 1 --2- 172.21.15.22:0/1480234759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7240b41f0 0x7fa72413efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:41.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa729465640 1 --2- 172.21.15.22:0/1480234759 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa72406c670 0x7fa72413d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:41.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa728c64640 1 -- 172.21.15.22:0/1480234759 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa72406c670 msgr2=0x7fa72413d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:41.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa728c64640 1 --2- 172.21.15.22:0/1480234759 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa72406c670 0x7fa72413d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa728c64640 1 -- 172.21.15.22:0/1480234759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7240b41f0 msgr2=0x7fa72413efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:41.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa728c64640 1 --2- 172.21.15.22:0/1480234759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7240b41f0 0x7fa72413efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa728c64640 1 -- 172.21.15.22:0/1480234759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72413c8e0 con 0x7fa72406ca50 2024-03-29T04:18:41.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa729465640 1 --2- 172.21.15.22:0/1480234759 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa72406c670 0x7fa72413d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:41.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa729c66640 1 --2- 172.21.15.22:0/1480234759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7240b41f0 0x7fa72413efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:41.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa728c64640 1 --2- 172.21.15.22:0/1480234759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa72406ca50 0x7fa724134360 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa71400c450 tx=0x7fa71400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:41.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa719ffb640 1 -- 172.21.15.22:0/1480234759 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714017070 con 0x7fa72406ca50 2024-03-29T04:18:41.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa719ffb640 1 -- 172.21.15.22:0/1480234759 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa714005cf0 con 0x7fa72406ca50 2024-03-29T04:18:41.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa719ffb640 1 -- 172.21.15.22:0/1480234759 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714007290 con 0x7fa72406ca50 2024-03-29T04:18:41.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa72b6c9640 1 -- 172.21.15.22:0/1480234759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa72413cbc0 con 0x7fa72406ca50 2024-03-29T04:18:41.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa72b6c9640 1 -- 172.21.15.22:0/1480234759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa72413d180 con 0x7fa72406ca50 2024-03-29T04:18:41.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa72b6c9640 1 -- 172.21.15.22:0/1480234759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa724068890 con 0x7fa72406ca50 2024-03-29T04:18:41.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa719ffb640 1 -- 172.21.15.22:0/1480234759 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fa71401a070 con 0x7fa72406ca50 2024-03-29T04:18:41.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa719ffb640 1 --2- 172.21.15.22:0/1480234759 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa708067fc0 0x7fa70806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:41.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa719ffb640 1 -- 172.21.15.22:0/1480234759 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fa714085c10 con 0x7fa72406ca50 2024-03-29T04:18:41.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.580+0000 7fa729465640 1 --2- 172.21.15.22:0/1480234759 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa708067fc0 0x7fa70806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:41.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.584+0000 7fa729465640 1 --2- 172.21.15.22:0/1480234759 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa708067fc0 0x7fa70806a480 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa72415f120 tx=0x7fa720008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:41.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.584+0000 7fa719ffb640 1 -- 172.21.15.22:0/1480234759 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fa714055fc0 con 0x7fa72406ca50 2024-03-29T04:18:41.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.744+0000 7fa72b6c9640 1 -- 172.21.15.22:0/1480234759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa724160350 con 0x7fa72406ca50 2024-03-29T04:18:41.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.748+0000 7fa719ffb640 1 -- 172.21.15.22:0/1480234759 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18847 (secure 0 0 0) 0x7fa7140558c0 con 0x7fa72406ca50 2024-03-29T04:18:41.730 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:41.730 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:18:16.893439+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:41.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.752+0000 7fa72b6c9640 1 -- 172.21.15.22:0/1480234759 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa708067fc0 msgr2=0x7fa70806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:41.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.752+0000 7fa72b6c9640 1 --2- 172.21.15.22:0/1480234759 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa708067fc0 0x7fa70806a480 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa72415f120 tx=0x7fa720008040 comp rx=0 tx=0).stop 2024-03-29T04:18:41.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.752+0000 7fa72b6c9640 1 -- 172.21.15.22:0/1480234759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa72406ca50 msgr2=0x7fa724134360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:41.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.752+0000 7fa72b6c9640 1 --2- 172.21.15.22:0/1480234759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa72406ca50 0x7fa724134360 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa71400c450 tx=0x7fa71400c910 comp rx=0 tx=0).stop 2024-03-29T04:18:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.752+0000 7fa72b6c9640 1 -- 172.21.15.22:0/1480234759 shutdown_connections 2024-03-29T04:18:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.752+0000 7fa72b6c9640 1 --2- 172.21.15.22:0/1480234759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7240b41f0 0x7fa72413efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.752+0000 7fa72b6c9640 1 --2- 172.21.15.22:0/1480234759 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa708067fc0 0x7fa70806a480 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.752+0000 7fa72b6c9640 1 --2- 172.21.15.22:0/1480234759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa72406ca50 0x7fa724134360 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.752+0000 7fa72b6c9640 1 --2- 172.21.15.22:0/1480234759 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa72406c670 0x7fa72413d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.752+0000 7fa72b6c9640 1 -- 172.21.15.22:0/1480234759 >> 172.21.15.22:0/1480234759 conn(0x7fa7240cfee0 msgr2=0x7fa72406dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.752+0000 7fa72b6c9640 1 -- 172.21.15.22:0/1480234759 shutdown_connections 2024-03-29T04:18:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.752+0000 7fa72b6c9640 1 -- 172.21.15.22:0/1480234759 wait complete. 2024-03-29T04:18:41.749 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:17:34.034142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:17:34.886659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:17:38.628632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:18:41.750 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:18:41.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.896+0000 7f18aedba640 1 Processor -- start 2024-03-29T04:18:41.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.896+0000 7f18aedba640 1 -- start start 2024-03-29T04:18:41.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.896+0000 7f18aedba640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f18a813f140 0x7f18a81606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:41.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.896+0000 7f18aedba640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18a806cd80 0x7f18a80ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:41.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.896+0000 7f18aedba640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18a806d710 con 0x7f18a815ff40 2024-03-29T04:18:41.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.896+0000 7f18aedba640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f18a806e320 con 0x7f18a813f140 2024-03-29T04:18:41.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.896+0000 7f18aedba640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f18a806d9f0 con 0x7f18a806cd80 2024-03-29T04:18:41.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.896+0000 7f18acb56640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f18a813f140 0x7f18a81606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:41.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.896+0000 7f18acb56640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f18a813f140 0x7f18a81606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51390/0 (socket says 172.21.15.22:51390) 2024-03-29T04:18:41.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.896+0000 7f18acb56640 1 -- 172.21.15.22:0/1396955469 learned_addr learned my addr 172.21.15.22:0/1396955469 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:41.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.896+0000 7f18ad357640 1 --2- 172.21.15.22:0/1396955469 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18a806cd80 0x7f18a80ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:41.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.896+0000 7f18acb56640 1 -- 172.21.15.22:0/1396955469 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18a806cd80 msgr2=0x7f18a80ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:41.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.896+0000 7f18acb56640 1 --2- 172.21.15.22:0/1396955469 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18a806cd80 0x7f18a80ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.896+0000 7f18acb56640 1 -- 172.21.15.22:0/1396955469 >> v1:172.21.15.151:6789/0 conn(0x7f18a815ff40 legacy=0x7f18a806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:41.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.896+0000 7f18acb56640 1 -- 172.21.15.22:0/1396955469 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a806dcd0 con 0x7f18a813f140 2024-03-29T04:18:41.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.896+0000 7f18ad357640 1 --2- 172.21.15.22:0/1396955469 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18a806cd80 0x7f18a80ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:41.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.900+0000 7f18acb56640 1 --2- 172.21.15.22:0/1396955469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18a813f140 0x7f18a81606d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009e50 tx=0x7f18a001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0bf592201eeec51 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:41.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.900+0000 7f18a77fe640 1 -- 172.21.15.22:0/1396955469 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a00a58f0 con 0x7f18a813f140 2024-03-29T04:18:41.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.900+0000 7f18a77fe640 1 -- 172.21.15.22:0/1396955469 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a00a5a90 con 0x7f18a813f140 2024-03-29T04:18:41.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.900+0000 7f18a77fe640 1 -- 172.21.15.22:0/1396955469 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a00a5e60 con 0x7f18a813f140 2024-03-29T04:18:41.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.900+0000 7f18aedba640 1 -- 172.21.15.22:0/1396955469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18a813f140 msgr2=0x7f18a81606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:41.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.900+0000 7f18aedba640 1 --2- 172.21.15.22:0/1396955469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18a813f140 0x7f18a81606d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009e50 tx=0x7f18a001ca00 comp rx=0 tx=0).stop 2024-03-29T04:18:41.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.900+0000 7f18aedba640 1 -- 172.21.15.22:0/1396955469 shutdown_connections 2024-03-29T04:18:41.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.900+0000 7f18aedba640 1 --2- 172.21.15.22:0/1396955469 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18a806cd80 0x7f18a80ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.900+0000 7f18aedba640 1 --2- 172.21.15.22:0/1396955469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18a813f140 0x7f18a81606d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.900+0000 7f18aedba640 1 -- 172.21.15.22:0/1396955469 >> 172.21.15.22:0/1396955469 conn(0x7f18a80d00a0 msgr2=0x7f18a80d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:41.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.900+0000 7f18aedba640 1 -- 172.21.15.22:0/1396955469 shutdown_connections 2024-03-29T04:18:41.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.900+0000 7f18aedba640 1 -- 172.21.15.22:0/1396955469 wait complete. 2024-03-29T04:18:41.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.900+0000 7f18aedba640 1 Processor -- start 2024-03-29T04:18:41.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.900+0000 7f18aedba640 1 -- start start 2024-03-29T04:18:41.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18aedba640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18a806cd80 0x7f18a8137320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18aedba640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18a813f140 0x7f18a8151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18aedba640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18a815ff40 0x7f18a8133c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18aedba640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f18a80cf810 con 0x7f18a815ff40 2024-03-29T04:18:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18aedba640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f18a80cf690 con 0x7f18a806cd80 2024-03-29T04:18:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18aedba640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f18a80cf990 con 0x7f18a813f140 2024-03-29T04:18:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18ad357640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18a815ff40 0x7f18a8133c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18ad357640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18a815ff40 0x7f18a8133c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51392/0 (socket says 172.21.15.22:51392) 2024-03-29T04:18:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18ad357640 1 -- 172.21.15.22:0/3103996173 learned_addr learned my addr 172.21.15.22:0/3103996173 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18a7fff640 1 --2- 172.21.15.22:0/3103996173 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18a813f140 0x7f18a8151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18acb56640 1 --2- 172.21.15.22:0/3103996173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18a806cd80 0x7f18a8137320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18ad357640 1 -- 172.21.15.22:0/3103996173 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18a813f140 msgr2=0x7f18a8151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18ad357640 1 --2- 172.21.15.22:0/3103996173 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18a813f140 0x7f18a8151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18ad357640 1 -- 172.21.15.22:0/3103996173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18a806cd80 msgr2=0x7f18a8137320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:41.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18ad357640 1 --2- 172.21.15.22:0/3103996173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18a806cd80 0x7f18a8137320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:41.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18ad357640 1 -- 172.21.15.22:0/3103996173 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a8137b70 con 0x7f18a815ff40 2024-03-29T04:18:41.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18a7fff640 1 --2- 172.21.15.22:0/3103996173 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18a813f140 0x7f18a8151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:41.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18acb56640 1 --2- 172.21.15.22:0/3103996173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18a806cd80 0x7f18a8137320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:41.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18ad357640 1 --2- 172.21.15.22:0/3103996173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18a815ff40 0x7f18a8133c10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f189800c7f0 tx=0x7f189800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:41.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18a57fa640 1 -- 172.21.15.22:0/3103996173 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898015070 con 0x7f18a815ff40 2024-03-29T04:18:41.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18a57fa640 1 -- 172.21.15.22:0/3103996173 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1898007820 con 0x7f18a815ff40 2024-03-29T04:18:41.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18a57fa640 1 -- 172.21.15.22:0/3103996173 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898007c50 con 0x7f18a815ff40 2024-03-29T04:18:41.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18aedba640 1 -- 172.21.15.22:0/3103996173 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a8137e50 con 0x7f18a815ff40 2024-03-29T04:18:41.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18aedba640 1 -- 172.21.15.22:0/3103996173 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a8137860 con 0x7f18a815ff40 2024-03-29T04:18:41.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18aedba640 1 -- 172.21.15.22:0/3103996173 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a8068900 con 0x7f18a815ff40 2024-03-29T04:18:41.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18a57fa640 1 -- 172.21.15.22:0/3103996173 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1898018070 con 0x7f18a815ff40 2024-03-29T04:18:41.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18a57fa640 1 --2- 172.21.15.22:0/3103996173 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1894067fc0 0x7f189406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:41.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18a57fa640 1 -- 172.21.15.22:0/3103996173 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f1898086b60 con 0x7f18a815ff40 2024-03-29T04:18:41.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.904+0000 7f18acb56640 1 --2- 172.21.15.22:0/3103996173 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1894067fc0 0x7f189406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:41.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.908+0000 7f18acb56640 1 --2- 172.21.15.22:0/3103996173 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1894067fc0 0x7f189406a480 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f18a00ae000 tx=0x7f18a00a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:41.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:41.908+0000 7f18a57fa640 1 -- 172.21.15.22:0/3103996173 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1898056e70 con 0x7f18a815ff40 2024-03-29T04:18:42.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.072+0000 7f18aedba640 1 -- 172.21.15.22:0/3103996173 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f18a81334b0 con 0x7f18a815ff40 2024-03-29T04:18:42.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.072+0000 7f18a57fa640 1 -- 172.21.15.22:0/3103996173 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f1898056770 con 0x7f18a815ff40 2024-03-29T04:18:42.053 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:18:42.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.072+0000 7f18aedba640 1 -- 172.21.15.22:0/3103996173 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1894067fc0 msgr2=0x7f189406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:42.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.072+0000 7f18aedba640 1 --2- 172.21.15.22:0/3103996173 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1894067fc0 0x7f189406a480 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f18a00ae000 tx=0x7f18a00a7070 comp rx=0 tx=0).stop 2024-03-29T04:18:42.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.072+0000 7f18aedba640 1 -- 172.21.15.22:0/3103996173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18a815ff40 msgr2=0x7f18a8133c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:42.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.072+0000 7f18aedba640 1 --2- 172.21.15.22:0/3103996173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18a815ff40 0x7f18a8133c10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f189800c7f0 tx=0x7f189800ccb0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.072+0000 7f18aedba640 1 -- 172.21.15.22:0/3103996173 shutdown_connections 2024-03-29T04:18:42.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.072+0000 7f18aedba640 1 --2- 172.21.15.22:0/3103996173 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f1894067fc0 0x7f189406a480 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.072+0000 7f18aedba640 1 --2- 172.21.15.22:0/3103996173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18a815ff40 0x7f18a8133c10 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.072+0000 7f18aedba640 1 --2- 172.21.15.22:0/3103996173 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f18a813f140 0x7f18a8151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.072+0000 7f18aedba640 1 --2- 172.21.15.22:0/3103996173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18a806cd80 0x7f18a8137320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.072+0000 7f18aedba640 1 -- 172.21.15.22:0/3103996173 >> 172.21.15.22:0/3103996173 conn(0x7f18a80d00a0 msgr2=0x7f18a806b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:42.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.072+0000 7f18aedba640 1 -- 172.21.15.22:0/3103996173 shutdown_connections 2024-03-29T04:18:42.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.072+0000 7f18aedba640 1 -- 172.21.15.22:0/3103996173 wait complete. 2024-03-29T04:18:42.071 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:18:42.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.216+0000 7f8e08ac1640 1 Processor -- start 2024-03-29T04:18:42.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.216+0000 7f8e08ac1640 1 -- start start 2024-03-29T04:18:42.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.216+0000 7f8e08ac1640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8e040afc00 0x7f8e040ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.216+0000 7f8e08ac1640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e0406fd40 0x7f8e040b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.216+0000 7f8e08ac1640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e040b3820 con 0x7f8e0415b190 2024-03-29T04:18:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.216+0000 7f8e08ac1640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f8e040b4110 con 0x7f8e040afc00 2024-03-29T04:18:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.216+0000 7f8e08ac1640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8e0406ef80 con 0x7f8e0406fd40 2024-03-29T04:18:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.216+0000 7f8e01d9b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8e040afc00 0x7f8e040ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.216+0000 7f8e01d9b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8e040afc00 0x7f8e040ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51402/0 (socket says 172.21.15.22:51402) 2024-03-29T04:18:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.216+0000 7f8e01d9b640 1 -- 172.21.15.22:0/1957105762 learned_addr learned my addr 172.21.15.22:0/1957105762 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.216+0000 7f8e02d9d640 1 --2- 172.21.15.22:0/1957105762 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e0406fd40 0x7f8e040b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.216+0000 7f8e01d9b640 1 -- 172.21.15.22:0/1957105762 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e0406fd40 msgr2=0x7f8e040b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.216+0000 7f8e01d9b640 1 --2- 172.21.15.22:0/1957105762 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e0406fd40 0x7f8e040b45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.216+0000 7f8e01d9b640 1 -- 172.21.15.22:0/1957105762 >> v1:172.21.15.151:6789/0 conn(0x7f8e0415b190 legacy=0x7f8e0406f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.216+0000 7f8e01d9b640 1 -- 172.21.15.22:0/1957105762 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e040b3ac0 con 0x7f8e040afc00 2024-03-29T04:18:42.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e01d9b640 1 --2- 172.21.15.22:0/1957105762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e040afc00 0x7f8e040ce220 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8e040b4290 tx=0x7f8df801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a476cf20566a6be server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:42.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e0159a640 1 -- 172.21.15.22:0/1957105762 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df80aa610 con 0x7f8e040afc00 2024-03-29T04:18:42.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e0159a640 1 -- 172.21.15.22:0/1957105762 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df80af070 con 0x7f8e040afc00 2024-03-29T04:18:42.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e0159a640 1 -- 172.21.15.22:0/1957105762 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df80aa9d0 con 0x7f8e040afc00 2024-03-29T04:18:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e08ac1640 1 -- 172.21.15.22:0/1957105762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e040afc00 msgr2=0x7f8e040ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e08ac1640 1 --2- 172.21.15.22:0/1957105762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e040afc00 0x7f8e040ce220 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8e040b4290 tx=0x7f8df801c510 comp rx=0 tx=0).stop 2024-03-29T04:18:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e08ac1640 1 -- 172.21.15.22:0/1957105762 shutdown_connections 2024-03-29T04:18:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e08ac1640 1 --2- 172.21.15.22:0/1957105762 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e0406fd40 0x7f8e040b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e08ac1640 1 --2- 172.21.15.22:0/1957105762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e040afc00 0x7f8e040ce220 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e08ac1640 1 -- 172.21.15.22:0/1957105762 >> 172.21.15.22:0/1957105762 conn(0x7f8e040d0c90 msgr2=0x7f8e040d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e08ac1640 1 -- 172.21.15.22:0/1957105762 shutdown_connections 2024-03-29T04:18:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e08ac1640 1 -- 172.21.15.22:0/1957105762 wait complete. 2024-03-29T04:18:42.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e08ac1640 1 Processor -- start 2024-03-29T04:18:42.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e08ac1640 1 -- start start 2024-03-29T04:18:42.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e08ac1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e0406fd40 0x7f8e04150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:42.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e08ac1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e040afc00 0x7f8e04165280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:42.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e08ac1640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e0415b190 0x7f8e041657c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:42.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e08ac1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8e04168090 con 0x7f8e040afc00 2024-03-29T04:18:42.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e08ac1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e04167f10 con 0x7f8e0406fd40 2024-03-29T04:18:42.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e08ac1640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8e04168210 con 0x7f8e0415b190 2024-03-29T04:18:42.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e01d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e040afc00 0x7f8e04165280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:42.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e01d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e040afc00 0x7f8e04165280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51416/0 (socket says 172.21.15.22:51416) 2024-03-29T04:18:42.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e01d9b640 1 -- 172.21.15.22:0/1471964809 learned_addr learned my addr 172.21.15.22:0/1471964809 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:42.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e02d9d640 1 --2- 172.21.15.22:0/1471964809 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e0415b190 0x7f8e041657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:42.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e0259c640 1 --2- 172.21.15.22:0/1471964809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e0406fd40 0x7f8e04150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:42.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e01d9b640 1 -- 172.21.15.22:0/1471964809 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e0415b190 msgr2=0x7f8e041657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e01d9b640 1 --2- 172.21.15.22:0/1471964809 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e0415b190 0x7f8e041657c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e01d9b640 1 -- 172.21.15.22:0/1471964809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e0406fd40 msgr2=0x7f8e04150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e01d9b640 1 --2- 172.21.15.22:0/1471964809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e0406fd40 0x7f8e04150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.220+0000 7f8e01d9b640 1 -- 172.21.15.22:0/1471964809 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e04165d00 con 0x7f8e040afc00 2024-03-29T04:18:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.224+0000 7f8e01d9b640 1 --2- 172.21.15.22:0/1471964809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e040afc00 0x7f8e04165280 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8df801a140 tx=0x7f8df801ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.224+0000 7f8df2ffd640 1 -- 172.21.15.22:0/1471964809 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df80bd5f0 con 0x7f8e040afc00 2024-03-29T04:18:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.224+0000 7f8df2ffd640 1 -- 172.21.15.22:0/1471964809 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df80bd790 con 0x7f8e040afc00 2024-03-29T04:18:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.224+0000 7f8e08ac1640 1 -- 172.21.15.22:0/1471964809 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0413d280 con 0x7f8e040afc00 2024-03-29T04:18:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.224+0000 7f8df2ffd640 1 -- 172.21.15.22:0/1471964809 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df80bdb40 con 0x7f8e040afc00 2024-03-29T04:18:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.224+0000 7f8e08ac1640 1 -- 172.21.15.22:0/1471964809 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0413d840 con 0x7f8e040afc00 2024-03-29T04:18:42.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.224+0000 7f8e08ac1640 1 -- 172.21.15.22:0/1471964809 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e04068900 con 0x7f8e040afc00 2024-03-29T04:18:42.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.224+0000 7f8df2ffd640 1 -- 172.21.15.22:0/1471964809 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f8df80b33f0 con 0x7f8e040afc00 2024-03-29T04:18:42.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.224+0000 7f8df2ffd640 1 --2- 172.21.15.22:0/1471964809 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8de4067fc0 0x7f8de406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:42.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.224+0000 7f8df2ffd640 1 -- 172.21.15.22:0/1471964809 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f8df81216c0 con 0x7f8e040afc00 2024-03-29T04:18:42.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.224+0000 7f8e0259c640 1 --2- 172.21.15.22:0/1471964809 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8de4067fc0 0x7f8de406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:42.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.224+0000 7f8e0259c640 1 --2- 172.21.15.22:0/1471964809 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8de4067fc0 0x7f8de406a480 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8e041517b0 tx=0x7f8df4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:42.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.228+0000 7f8df2ffd640 1 -- 172.21.15.22:0/1471964809 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f8df80c3060 con 0x7f8e040afc00 2024-03-29T04:18:42.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.392+0000 7f8e08ac1640 1 -- 172.21.15.22:0/1471964809 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8e0406c1b0 con 0x7f8e040afc00 2024-03-29T04:18:42.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.392+0000 7f8df2ffd640 1 -- 172.21.15.22:0/1471964809 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f8df80a7070 con 0x7f8e040afc00 2024-03-29T04:18:42.373 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:42.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.392+0000 7f8e08ac1640 1 -- 172.21.15.22:0/1471964809 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8de4067fc0 msgr2=0x7f8de406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:42.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.392+0000 7f8e08ac1640 1 --2- 172.21.15.22:0/1471964809 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8de4067fc0 0x7f8de406a480 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8e041517b0 tx=0x7f8df4009040 comp rx=0 tx=0).stop 2024-03-29T04:18:42.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.392+0000 7f8e08ac1640 1 -- 172.21.15.22:0/1471964809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e040afc00 msgr2=0x7f8e04165280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:42.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.392+0000 7f8e08ac1640 1 --2- 172.21.15.22:0/1471964809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e040afc00 0x7f8e04165280 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8df801a140 tx=0x7f8df801ad50 comp rx=0 tx=0).stop 2024-03-29T04:18:42.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.396+0000 7f8e08ac1640 1 -- 172.21.15.22:0/1471964809 shutdown_connections 2024-03-29T04:18:42.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.396+0000 7f8e08ac1640 1 --2- 172.21.15.22:0/1471964809 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e0415b190 0x7f8e041657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.396+0000 7f8e08ac1640 1 --2- 172.21.15.22:0/1471964809 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8de4067fc0 0x7f8de406a480 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.396+0000 7f8e08ac1640 1 --2- 172.21.15.22:0/1471964809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e040afc00 0x7f8e04165280 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.396+0000 7f8e08ac1640 1 --2- 172.21.15.22:0/1471964809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e0406fd40 0x7f8e04150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.396+0000 7f8e08ac1640 1 -- 172.21.15.22:0/1471964809 >> 172.21.15.22:0/1471964809 conn(0x7f8e040d0c90 msgr2=0x7f8e040b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:42.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.396+0000 7f8e08ac1640 1 -- 172.21.15.22:0/1471964809 shutdown_connections 2024-03-29T04:18:42.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.396+0000 7f8e08ac1640 1 -- 172.21.15.22:0/1471964809 wait complete. 2024-03-29T04:18:42.392 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:18:42.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.544+0000 7f944e719640 1 Processor -- start 2024-03-29T04:18:42.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.544+0000 7f944e719640 1 -- start start 2024-03-29T04:18:42.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.544+0000 7f944e719640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f944815ff40 0x7f944806c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:42.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.544+0000 7f944e719640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f944806d710 con 0x7f944813f140 2024-03-29T04:18:42.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.544+0000 7f944e719640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f944806d9f0 con 0x7f944806cbb0 2024-03-29T04:18:42.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.544+0000 7f944e719640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f944806d000 con 0x7f944815ff40 2024-03-29T04:18:42.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.544+0000 7f9447fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f944813f140 0x7f94481606d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58578/0 (socket says 172.21.15.22:58578) 2024-03-29T04:18:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.544+0000 7f9447fff640 1 -- 172.21.15.22:0/2085602061 learned_addr learned my addr 172.21.15.22:0/2085602061 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.544+0000 7f943f7fe640 1 --2- 172.21.15.22:0/2085602061 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f944815ff40 0x7f944806c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.544+0000 7f94477fe640 1 -- 172.21.15.22:0/2085602061 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3932847015 0 0) 0x7f944806d710 con 0x7f944813f140 2024-03-29T04:18:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.544+0000 7f94477fe640 1 -- 172.21.15.22:0/2085602061 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9434003530 con 0x7f944813f140 2024-03-29T04:18:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.544+0000 7f94477fe640 1 -- 172.21.15.22:0/2085602061 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 899060023 0 0) 0x7f944806d9f0 con 0x7f944806cbb0 2024-03-29T04:18:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.544+0000 7f94477fe640 1 -- 172.21.15.22:0/2085602061 --> v1:172.21.15.151:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f944806d710 con 0x7f944806cbb0 2024-03-29T04:18:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.544+0000 7f94477fe640 1 -- 172.21.15.22:0/2085602061 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3254380005 0 0) 0x7f9434003530 con 0x7f944813f140 2024-03-29T04:18:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.544+0000 7f943f7fe640 1 -- 172.21.15.22:0/2085602061 >> v1:172.21.15.151:6789/0 conn(0x7f944806cbb0 legacy=0x7f94480ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.544+0000 7f943f7fe640 1 -- 172.21.15.22:0/2085602061 >> v1:172.21.15.22:6789/0 conn(0x7f944813f140 legacy=0x7f94481606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:42.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.544+0000 7f943f7fe640 1 -- 172.21.15.22:0/2085602061 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944806dcd0 con 0x7f944815ff40 2024-03-29T04:18:42.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f943f7fe640 1 --2- 172.21.15.22:0/2085602061 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f944815ff40 0x7f944806c670 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f94300097f0 tx=0x7f943001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=940df9b972d67f88 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:42.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f94477fe640 1 -- 172.21.15.22:0/2085602061 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94300b0070 con 0x7f944815ff40 2024-03-29T04:18:42.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f94477fe640 1 -- 172.21.15.22:0/2085602061 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94300ab400 con 0x7f944815ff40 2024-03-29T04:18:42.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f94477fe640 1 -- 172.21.15.22:0/2085602061 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9438001680 con 0x7f944815ff40 2024-03-29T04:18:42.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f944e719640 1 -- 172.21.15.22:0/2085602061 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f944815ff40 msgr2=0x7f944806c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:42.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f944e719640 1 --2- 172.21.15.22:0/2085602061 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f944815ff40 0x7f944806c670 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f94300097f0 tx=0x7f943001c400 comp rx=0 tx=0).stop 2024-03-29T04:18:42.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f944e719640 1 -- 172.21.15.22:0/2085602061 shutdown_connections 2024-03-29T04:18:42.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f944e719640 1 --2- 172.21.15.22:0/2085602061 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f944815ff40 0x7f944806c670 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f944e719640 1 -- 172.21.15.22:0/2085602061 >> 172.21.15.22:0/2085602061 conn(0x7f94480d00a0 msgr2=0x7f94480d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:42.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f944e719640 1 -- 172.21.15.22:0/2085602061 shutdown_connections 2024-03-29T04:18:42.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f944e719640 1 -- 172.21.15.22:0/2085602061 wait complete. 2024-03-29T04:18:42.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f944e719640 1 Processor -- start 2024-03-29T04:18:42.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f944e719640 1 -- start start 2024-03-29T04:18:42.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f944e719640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f944806cbb0 0x7f944813a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:42.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f944e719640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f944813f140 0x7f9448158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:42.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f944e719640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f944815ff40 0x7f9448155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:42.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f944e719640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f94480cfc30 con 0x7f944813f140 2024-03-29T04:18:42.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f944e719640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f94480cfab0 con 0x7f944815ff40 2024-03-29T04:18:42.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f944e719640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f944816a000 con 0x7f944806cbb0 2024-03-29T04:18:42.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f943f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f944813f140 0x7f9448158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:42.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f943f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f944813f140 0x7f9448158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51428/0 (socket says 172.21.15.22:51428) 2024-03-29T04:18:42.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f943f7fe640 1 -- 172.21.15.22:0/1490473559 learned_addr learned my addr 172.21.15.22:0/1490473559 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:42.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f944ccb6640 1 --2- 172.21.15.22:0/1490473559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f944815ff40 0x7f9448155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:42.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f9447fff640 1 --2- 172.21.15.22:0/1490473559 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f944806cbb0 0x7f944813a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:42.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f943f7fe640 1 -- 172.21.15.22:0/1490473559 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f944806cbb0 msgr2=0x7f944813a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:42.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.548+0000 7f943f7fe640 1 --2- 172.21.15.22:0/1490473559 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f944806cbb0 0x7f944813a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.552+0000 7f943f7fe640 1 -- 172.21.15.22:0/1490473559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f944815ff40 msgr2=0x7f9448155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:42.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.552+0000 7f943f7fe640 1 --2- 172.21.15.22:0/1490473559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f944815ff40 0x7f9448155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.552+0000 7f943f7fe640 1 -- 172.21.15.22:0/1490473559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9448160bd0 con 0x7f944813f140 2024-03-29T04:18:42.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.552+0000 7f943f7fe640 1 --2- 172.21.15.22:0/1490473559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f944813f140 0x7f9448158e40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9430000c10 tx=0x7f94300191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:42.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.552+0000 7f94457fa640 1 -- 172.21.15.22:0/1490473559 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94300bb070 con 0x7f944813f140 2024-03-29T04:18:42.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.552+0000 7f94457fa640 1 -- 172.21.15.22:0/1490473559 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94300b7460 con 0x7f944813f140 2024-03-29T04:18:42.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.552+0000 7f944e719640 1 -- 172.21.15.22:0/1490473559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9448157660 con 0x7f944813f140 2024-03-29T04:18:42.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.552+0000 7f94457fa640 1 -- 172.21.15.22:0/1490473559 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94300b0040 con 0x7f944813f140 2024-03-29T04:18:42.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.552+0000 7f944e719640 1 -- 172.21.15.22:0/1490473559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9448157c20 con 0x7f944813f140 2024-03-29T04:18:42.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.552+0000 7f944e719640 1 -- 172.21.15.22:0/1490473559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9448068900 con 0x7f944813f140 2024-03-29T04:18:42.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.552+0000 7f94457fa640 1 -- 172.21.15.22:0/1490473559 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f94300be070 con 0x7f944813f140 2024-03-29T04:18:42.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.552+0000 7f94457fa640 1 --2- 172.21.15.22:0/1490473559 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9428067fc0 0x7f942806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:42.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.552+0000 7f94457fa640 1 -- 172.21.15.22:0/1490473559 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f9430120c00 con 0x7f944813f140 2024-03-29T04:18:42.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.552+0000 7f9447fff640 1 --2- 172.21.15.22:0/1490473559 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9428067fc0 0x7f942806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:42.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.552+0000 7f9447fff640 1 --2- 172.21.15.22:0/1490473559 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9428067fc0 0x7f942806a480 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f94380011f0 tx=0x7f9438009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:42.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.556+0000 7f94457fa640 1 -- 172.21.15.22:0/1490473559 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9430127080 con 0x7f944813f140 2024-03-29T04:18:42.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.716+0000 7f944e719640 1 -- 172.21.15.22:0/1490473559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f944815ab00 con 0x7f944813f140 2024-03-29T04:18:42.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.716+0000 7f94457fa640 1 -- 172.21.15.22:0/1490473559 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f9430015ff0 con 0x7f944813f140 2024-03-29T04:18:42.697 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-29T04:18:42.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.720+0000 7f944e719640 1 -- 172.21.15.22:0/1490473559 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9428067fc0 msgr2=0x7f942806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:42.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.720+0000 7f944e719640 1 --2- 172.21.15.22:0/1490473559 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9428067fc0 0x7f942806a480 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f94380011f0 tx=0x7f9438009040 comp rx=0 tx=0).stop 2024-03-29T04:18:42.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.720+0000 7f944e719640 1 -- 172.21.15.22:0/1490473559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f944813f140 msgr2=0x7f9448158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:42.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.720+0000 7f944e719640 1 --2- 172.21.15.22:0/1490473559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f944813f140 0x7f9448158e40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9430000c10 tx=0x7f94300191a0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.720+0000 7f944e719640 1 -- 172.21.15.22:0/1490473559 shutdown_connections 2024-03-29T04:18:42.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.720+0000 7f944e719640 1 --2- 172.21.15.22:0/1490473559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f944815ff40 0x7f9448155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.720+0000 7f944e719640 1 --2- 172.21.15.22:0/1490473559 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9428067fc0 0x7f942806a480 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.720+0000 7f944e719640 1 --2- 172.21.15.22:0/1490473559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f944813f140 0x7f9448158e40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.720+0000 7f944e719640 1 --2- 172.21.15.22:0/1490473559 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f944806cbb0 0x7f944813a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.720+0000 7f944e719640 1 -- 172.21.15.22:0/1490473559 >> 172.21.15.22:0/1490473559 conn(0x7f94480d00a0 msgr2=0x7f94480b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:42.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.720+0000 7f944e719640 1 -- 172.21.15.22:0/1490473559 shutdown_connections 2024-03-29T04:18:42.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.720+0000 7f944e719640 1 -- 172.21.15.22:0/1490473559 wait complete. 2024-03-29T04:18:42.715 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:18:42.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.860+0000 7fe77413a640 1 Processor -- start 2024-03-29T04:18:42.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.860+0000 7fe77413a640 1 -- start start 2024-03-29T04:18:42.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.860+0000 7fe77413a640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fe76c0cf870 0x7fe76c15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:42.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.860+0000 7fe77413a640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe76c0726d0 0x7fe76c0b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:42.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.860+0000 7fe77413a640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe76c0b3c40 con 0x7fe76c0b4090 2024-03-29T04:18:42.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.860+0000 7fe77413a640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fe76c071910 con 0x7fe76c0cf870 2024-03-29T04:18:42.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.860+0000 7fe77413a640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe76c071bf0 con 0x7fe76c0726d0 2024-03-29T04:18:42.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.860+0000 7fe7726d7640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe76c0b4090 0x7fe76c06d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58586/0 (socket says 172.21.15.22:58586) 2024-03-29T04:18:42.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.860+0000 7fe771ed6640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fe76c0cf870 0x7fe76c15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:42.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.860+0000 7fe7726d7640 1 -- 172.21.15.22:0/2559942450 learned_addr learned my addr 172.21.15.22:0/2559942450 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:42.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.860+0000 7fe7716d5640 1 --2- 172.21.15.22:0/2559942450 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe76c0726d0 0x7fe76c0b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:42.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.860+0000 7fe770ed4640 1 -- 172.21.15.22:0/2559942450 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2208669218 0 0) 0x7fe76c0b3c40 con 0x7fe76c0b4090 2024-03-29T04:18:42.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.860+0000 7fe770ed4640 1 -- 172.21.15.22:0/2559942450 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe758003530 con 0x7fe76c0b4090 2024-03-29T04:18:42.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.860+0000 7fe771ed6640 1 -- 172.21.15.22:0/2559942450 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe76c0726d0 msgr2=0x7fe76c0b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:42.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.860+0000 7fe771ed6640 1 --2- 172.21.15.22:0/2559942450 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe76c0726d0 0x7fe76c0b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.860+0000 7fe771ed6640 1 -- 172.21.15.22:0/2559942450 >> v1:172.21.15.22:6789/0 conn(0x7fe76c0b4090 legacy=0x7fe76c06d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:42.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.860+0000 7fe771ed6640 1 -- 172.21.15.22:0/2559942450 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76c071ed0 con 0x7fe76c0cf870 2024-03-29T04:18:42.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.864+0000 7fe771ed6640 1 --2- 172.21.15.22:0/2559942450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe76c0cf870 0x7fe76c15baa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe76800a960 tx=0x7fe76801e2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96af6c1053041461 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:42.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.864+0000 7fe770ed4640 1 -- 172.21.15.22:0/2559942450 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7680b3070 con 0x7fe76c0cf870 2024-03-29T04:18:42.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.864+0000 7fe770ed4640 1 -- 172.21.15.22:0/2559942450 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7680a6930 con 0x7fe76c0cf870 2024-03-29T04:18:42.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.864+0000 7fe770ed4640 1 -- 172.21.15.22:0/2559942450 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76800b040 con 0x7fe76c0cf870 2024-03-29T04:18:42.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.864+0000 7fe77413a640 1 -- 172.21.15.22:0/2559942450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe76c0cf870 msgr2=0x7fe76c15baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:42.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.864+0000 7fe77413a640 1 --2- 172.21.15.22:0/2559942450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe76c0cf870 0x7fe76c15baa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe76800a960 tx=0x7fe76801e2a0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.864+0000 7fe77413a640 1 -- 172.21.15.22:0/2559942450 shutdown_connections 2024-03-29T04:18:42.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.864+0000 7fe77413a640 1 --2- 172.21.15.22:0/2559942450 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe76c0726d0 0x7fe76c0b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.864+0000 7fe77413a640 1 --2- 172.21.15.22:0/2559942450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe76c0cf870 0x7fe76c15baa0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.864+0000 7fe77413a640 1 -- 172.21.15.22:0/2559942450 >> 172.21.15.22:0/2559942450 conn(0x7fe76c0d00a0 msgr2=0x7fe76c0d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:42.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.864+0000 7fe77413a640 1 -- 172.21.15.22:0/2559942450 shutdown_connections 2024-03-29T04:18:42.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.864+0000 7fe77413a640 1 -- 172.21.15.22:0/2559942450 wait complete. 2024-03-29T04:18:42.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.864+0000 7fe77413a640 1 Processor -- start 2024-03-29T04:18:42.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.864+0000 7fe77413a640 1 -- start start 2024-03-29T04:18:42.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe77413a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe76c0726d0 0x7fe76c147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:42.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe77413a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe76c0b4090 0x7fe76c14eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:42.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe77413a640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe76c0cf870 0x7fe76c1581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:42.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe77413a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c06e310 con 0x7fe76c0b4090 2024-03-29T04:18:42.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe77413a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c06e190 con 0x7fe76c0726d0 2024-03-29T04:18:42.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe77413a640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe76c072ab0 con 0x7fe76c0cf870 2024-03-29T04:18:42.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe7716d5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe76c0b4090 0x7fe76c14eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:42.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe7716d5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe76c0b4090 0x7fe76c14eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51438/0 (socket says 172.21.15.22:51438) 2024-03-29T04:18:42.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe7716d5640 1 -- 172.21.15.22:0/2051073791 learned_addr learned my addr 172.21.15.22:0/2051073791 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:42.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe771ed6640 1 --2- 172.21.15.22:0/2051073791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe76c0726d0 0x7fe76c147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:42.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe7726d7640 1 --2- 172.21.15.22:0/2051073791 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe76c0cf870 0x7fe76c1581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:42.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe7716d5640 1 -- 172.21.15.22:0/2051073791 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe76c0cf870 msgr2=0x7fe76c1581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:42.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe7716d5640 1 --2- 172.21.15.22:0/2051073791 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe76c0cf870 0x7fe76c1581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe7716d5640 1 -- 172.21.15.22:0/2051073791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe76c0726d0 msgr2=0x7fe76c147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:42.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe7716d5640 1 --2- 172.21.15.22:0/2051073791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe76c0726d0 0x7fe76c147110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:42.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe7716d5640 1 -- 172.21.15.22:0/2051073791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76c147650 con 0x7fe76c0b4090 2024-03-29T04:18:42.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe7716d5640 1 --2- 172.21.15.22:0/2051073791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe76c0b4090 0x7fe76c14eec0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00e9b0 tx=0x7fe75c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:42.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe7627fc640 1 -- 172.21.15.22:0/2051073791 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c017070 con 0x7fe76c0b4090 2024-03-29T04:18:42.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe7627fc640 1 -- 172.21.15.22:0/2051073791 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe75c00c9a0 con 0x7fe76c0b4090 2024-03-29T04:18:42.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe77413a640 1 -- 172.21.15.22:0/2051073791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe76c14de20 con 0x7fe76c0b4090 2024-03-29T04:18:42.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe7627fc640 1 -- 172.21.15.22:0/2051073791 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00cd70 con 0x7fe76c0b4090 2024-03-29T04:18:42.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe77413a640 1 -- 172.21.15.22:0/2051073791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe76c14e3e0 con 0x7fe76c0b4090 2024-03-29T04:18:42.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe77413a640 1 -- 172.21.15.22:0/2051073791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe76c068900 con 0x7fe76c0b4090 2024-03-29T04:18:42.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe7627fc640 1 -- 172.21.15.22:0/2051073791 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe75c01a070 con 0x7fe76c0b4090 2024-03-29T04:18:42.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe7627fc640 1 --2- 172.21.15.22:0/2051073791 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe750067ff0 0x7fe75006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:42.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe7627fc640 1 -- 172.21.15.22:0/2051073791 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fe75c086970 con 0x7fe76c0b4090 2024-03-29T04:18:42.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.868+0000 7fe771ed6640 1 --2- 172.21.15.22:0/2051073791 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe750067ff0 0x7fe75006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:42.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.872+0000 7fe771ed6640 1 --2- 172.21.15.22:0/2051073791 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe750067ff0 0x7fe75006a4b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe7680b0000 tx=0x7fe768008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:42.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:42.872+0000 7fe7627fc640 1 -- 172.21.15.22:0/2051073791 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fe75c056d20 con 0x7fe76c0b4090 2024-03-29T04:18:43.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.036+0000 7fe77413a640 1 -- 172.21.15.22:0/2051073791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe76c150100 con 0x7fe76c0b4090 2024-03-29T04:18:43.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.036+0000 7fe7627fc640 1 -- 172.21.15.22:0/2051073791 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7fe75c056620 con 0x7fe76c0b4090 2024-03-29T04:18:43.016 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:43.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.036+0000 7fe77413a640 1 -- 172.21.15.22:0/2051073791 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe750067ff0 msgr2=0x7fe75006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:43.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.036+0000 7fe77413a640 1 --2- 172.21.15.22:0/2051073791 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe750067ff0 0x7fe75006a4b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe7680b0000 tx=0x7fe768008040 comp rx=0 tx=0).stop 2024-03-29T04:18:43.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.036+0000 7fe77413a640 1 -- 172.21.15.22:0/2051073791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe76c0b4090 msgr2=0x7fe76c14eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:43.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.036+0000 7fe77413a640 1 --2- 172.21.15.22:0/2051073791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe76c0b4090 0x7fe76c14eec0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00e9b0 tx=0x7fe75c00ee70 comp rx=0 tx=0).stop 2024-03-29T04:18:43.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.036+0000 7fe77413a640 1 -- 172.21.15.22:0/2051073791 shutdown_connections 2024-03-29T04:18:43.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.036+0000 7fe77413a640 1 --2- 172.21.15.22:0/2051073791 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe76c0cf870 0x7fe76c1581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.036+0000 7fe77413a640 1 --2- 172.21.15.22:0/2051073791 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe750067ff0 0x7fe75006a4b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.036+0000 7fe77413a640 1 --2- 172.21.15.22:0/2051073791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe76c0b4090 0x7fe76c14eec0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.036+0000 7fe77413a640 1 --2- 172.21.15.22:0/2051073791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe76c0726d0 0x7fe76c147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.036+0000 7fe77413a640 1 -- 172.21.15.22:0/2051073791 >> 172.21.15.22:0/2051073791 conn(0x7fe76c0d00a0 msgr2=0x7fe76c06d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:43.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.036+0000 7fe77413a640 1 -- 172.21.15.22:0/2051073791 shutdown_connections 2024-03-29T04:18:43.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.036+0000 7fe77413a640 1 -- 172.21.15.22:0/2051073791 wait complete. 2024-03-29T04:18:43.034 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:18:43.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.172+0000 7fabe0936640 1 Processor -- start 2024-03-29T04:18:43.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.172+0000 7fabe0936640 1 -- start start 2024-03-29T04:18:43.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.172+0000 7fabe0936640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fabdc0b2ef0 0x7fabdc144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:43.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.172+0000 7fabe0936640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fabdc15f7a0 0x7fabdc06c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:43.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.172+0000 7fabe0936640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabdc0b4ba0 con 0x7fabdc144270 2024-03-29T04:18:43.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.172+0000 7fabe0936640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fabdc144fa0 con 0x7fabdc15f7a0 2024-03-29T04:18:43.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.172+0000 7fabe0936640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fabdc0b33f0 con 0x7fabdc0b2ef0 2024-03-29T04:18:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.172+0000 7fabda59c640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fabdc144270 0x7fabdc0b4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58600/0 (socket says 172.21.15.22:58600) 2024-03-29T04:18:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.172+0000 7fabda59c640 1 -- 172.21.15.22:0/1118749102 learned_addr learned my addr 172.21.15.22:0/1118749102 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.172+0000 7fabdad9d640 1 --2- 172.21.15.22:0/1118749102 >> v2:172.21.15.22:3300/0 conn(0x7fabdc15f7a0 0x7fabdc06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.176+0000 7fabd9d9b640 1 -- 172.21.15.22:0/1118749102 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2902805066 0 0) 0x7fabdc0b4ba0 con 0x7fabdc144270 2024-03-29T04:18:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.176+0000 7fabd3fff640 1 --2- 172.21.15.22:0/1118749102 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fabdc0b2ef0 0x7fabdc144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.176+0000 7fabd9d9b640 1 -- 172.21.15.22:0/1118749102 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabc8005740 con 0x7fabdc144270 2024-03-29T04:18:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.176+0000 7fabdad9d640 1 -- 172.21.15.22:0/1118749102 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fabdc0b2ef0 msgr2=0x7fabdc144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.176+0000 7fabdad9d640 1 --2- 172.21.15.22:0/1118749102 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fabdc0b2ef0 0x7fabdc144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.176+0000 7fabdad9d640 1 -- 172.21.15.22:0/1118749102 >> v1:172.21.15.22:6789/0 conn(0x7fabdc144270 legacy=0x7fabdc0b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.176+0000 7fabdad9d640 1 -- 172.21.15.22:0/1118749102 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabdc0b36d0 con 0x7fabdc15f7a0 2024-03-29T04:18:43.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.176+0000 7fabdad9d640 1 --2- 172.21.15.22:0/1118749102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabdc15f7a0 0x7fabdc06c670 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fabd4009d90 tx=0x7fabd401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29bbaf409aef5840 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:43.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.176+0000 7fabd9d9b640 1 -- 172.21.15.22:0/1118749102 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd40b2070 con 0x7fabdc15f7a0 2024-03-29T04:18:43.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.176+0000 7fabd9d9b640 1 -- 172.21.15.22:0/1118749102 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabd40a5910 con 0x7fabdc15f7a0 2024-03-29T04:18:43.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.176+0000 7fabd9d9b640 1 -- 172.21.15.22:0/1118749102 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd40a5cc0 con 0x7fabdc15f7a0 2024-03-29T04:18:43.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.176+0000 7fabe0936640 1 -- 172.21.15.22:0/1118749102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabdc15f7a0 msgr2=0x7fabdc06c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:43.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.176+0000 7fabe0936640 1 --2- 172.21.15.22:0/1118749102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabdc15f7a0 0x7fabdc06c670 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fabd4009d90 tx=0x7fabd401c940 comp rx=0 tx=0).stop 2024-03-29T04:18:43.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.176+0000 7fabe0936640 1 -- 172.21.15.22:0/1118749102 shutdown_connections 2024-03-29T04:18:43.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.176+0000 7fabe0936640 1 --2- 172.21.15.22:0/1118749102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabdc15f7a0 0x7fabdc06c670 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.176+0000 7fabe0936640 1 --2- 172.21.15.22:0/1118749102 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fabdc0b2ef0 0x7fabdc144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.176+0000 7fabe0936640 1 -- 172.21.15.22:0/1118749102 >> 172.21.15.22:0/1118749102 conn(0x7fabdc0cfee0 msgr2=0x7fabdc0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:43.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.176+0000 7fabe0936640 1 -- 172.21.15.22:0/1118749102 shutdown_connections 2024-03-29T04:18:43.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.176+0000 7fabe0936640 1 -- 172.21.15.22:0/1118749102 wait complete. 2024-03-29T04:18:43.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabe0936640 1 Processor -- start 2024-03-29T04:18:43.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabe0936640 1 -- start start 2024-03-29T04:18:43.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabe0936640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabdc0b2ef0 0x7fabdc15a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:43.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabe0936640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fabdc144270 0x7fabdc15d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:43.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabe0936640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabdc15f7a0 0x7fabdc15d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:43.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabe0936640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc0cdd10 con 0x7fabdc0b2ef0 2024-03-29T04:18:43.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabe0936640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc15fc30 con 0x7fabdc15f7a0 2024-03-29T04:18:43.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabe0936640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fabdc0cde90 con 0x7fabdc144270 2024-03-29T04:18:43.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabda59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabdc0b2ef0 0x7fabdc15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:43.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabda59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabdc0b2ef0 0x7fabdc15a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51456/0 (socket says 172.21.15.22:51456) 2024-03-29T04:18:43.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabda59c640 1 -- 172.21.15.22:0/2921556056 learned_addr learned my addr 172.21.15.22:0/2921556056 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:43.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabdad9d640 1 --2- 172.21.15.22:0/2921556056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabdc15f7a0 0x7fabdc15d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:43.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabd3fff640 1 --2- 172.21.15.22:0/2921556056 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fabdc144270 0x7fabdc15d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:43.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabda59c640 1 -- 172.21.15.22:0/2921556056 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fabdc144270 msgr2=0x7fabdc15d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:43.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabda59c640 1 --2- 172.21.15.22:0/2921556056 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fabdc144270 0x7fabdc15d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabda59c640 1 -- 172.21.15.22:0/2921556056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabdc15f7a0 msgr2=0x7fabdc15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:43.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabda59c640 1 --2- 172.21.15.22:0/2921556056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabdc15f7a0 0x7fabdc15d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabda59c640 1 -- 172.21.15.22:0/2921556056 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabdc15c3b0 con 0x7fabdc0b2ef0 2024-03-29T04:18:43.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabdad9d640 1 --2- 172.21.15.22:0/2921556056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabdc15f7a0 0x7fabdc15d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:43.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabda59c640 1 --2- 172.21.15.22:0/2921556056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabdc0b2ef0 0x7fabdc15a930 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fabc400c3c0 tx=0x7fabc400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:43.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabd2ffd640 1 -- 172.21.15.22:0/2921556056 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4017070 con 0x7fabdc0b2ef0 2024-03-29T04:18:43.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabd2ffd640 1 -- 172.21.15.22:0/2921556056 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabc4005cf0 con 0x7fabdc0b2ef0 2024-03-29T04:18:43.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabe0936640 1 -- 172.21.15.22:0/2921556056 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabdc15c690 con 0x7fabdc0b2ef0 2024-03-29T04:18:43.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabd2ffd640 1 -- 172.21.15.22:0/2921556056 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc40072f0 con 0x7fabdc0b2ef0 2024-03-29T04:18:43.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabe0936640 1 -- 172.21.15.22:0/2921556056 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabdc15cc50 con 0x7fabdc0b2ef0 2024-03-29T04:18:43.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabe0936640 1 -- 172.21.15.22:0/2921556056 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabdc068890 con 0x7fabdc0b2ef0 2024-03-29T04:18:43.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.180+0000 7fabd2ffd640 1 -- 172.21.15.22:0/2921556056 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fabc401a070 con 0x7fabdc0b2ef0 2024-03-29T04:18:43.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.184+0000 7fabd2ffd640 1 --2- 172.21.15.22:0/2921556056 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fabbc067fc0 0x7fabbc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:43.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.184+0000 7fabd2ffd640 1 -- 172.21.15.22:0/2921556056 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fabc4085c60 con 0x7fabdc0b2ef0 2024-03-29T04:18:43.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.184+0000 7fabd3fff640 1 --2- 172.21.15.22:0/2921556056 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fabbc067fc0 0x7fabbc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:43.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.184+0000 7fabd3fff640 1 --2- 172.21.15.22:0/2921556056 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fabbc067fc0 0x7fabbc06a480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0cc2a0 tx=0x7fabcc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:43.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.188+0000 7fabd2ffd640 1 -- 172.21.15.22:0/2921556056 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fabc40560a0 con 0x7fabdc0b2ef0 2024-03-29T04:18:43.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.360+0000 7fabe0936640 1 -- 172.21.15.22:0/2921556056 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fabdc0722d0 con 0x7fabdc0b2ef0 2024-03-29T04:18:43.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.360+0000 7fabd2ffd640 1 -- 172.21.15.22:0/2921556056 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7fabc4085070 con 0x7fabdc0b2ef0 2024-03-29T04:18:43.342 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-03-29T04:18:43.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.364+0000 7fabe0936640 1 -- 172.21.15.22:0/2921556056 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fabbc067fc0 msgr2=0x7fabbc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:43.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.364+0000 7fabe0936640 1 --2- 172.21.15.22:0/2921556056 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fabbc067fc0 0x7fabbc06a480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0cc2a0 tx=0x7fabcc008040 comp rx=0 tx=0).stop 2024-03-29T04:18:43.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.364+0000 7fabe0936640 1 -- 172.21.15.22:0/2921556056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabdc0b2ef0 msgr2=0x7fabdc15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:43.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.364+0000 7fabe0936640 1 --2- 172.21.15.22:0/2921556056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabdc0b2ef0 0x7fabdc15a930 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fabc400c3c0 tx=0x7fabc400c880 comp rx=0 tx=0).stop 2024-03-29T04:18:43.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.364+0000 7fabe0936640 1 -- 172.21.15.22:0/2921556056 shutdown_connections 2024-03-29T04:18:43.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.364+0000 7fabe0936640 1 --2- 172.21.15.22:0/2921556056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabdc15f7a0 0x7fabdc15d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.364+0000 7fabe0936640 1 --2- 172.21.15.22:0/2921556056 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fabdc144270 0x7fabdc15d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.364+0000 7fabe0936640 1 --2- 172.21.15.22:0/2921556056 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fabbc067fc0 0x7fabbc06a480 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.364+0000 7fabe0936640 1 --2- 172.21.15.22:0/2921556056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabdc0b2ef0 0x7fabdc15a930 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.364+0000 7fabe0936640 1 -- 172.21.15.22:0/2921556056 >> 172.21.15.22:0/2921556056 conn(0x7fabdc0cfee0 msgr2=0x7fabdc06d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:43.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.364+0000 7fabe0936640 1 -- 172.21.15.22:0/2921556056 shutdown_connections 2024-03-29T04:18:43.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.364+0000 7fabe0936640 1 -- 172.21.15.22:0/2921556056 wait complete. 2024-03-29T04:18:43.363 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi151.front.sepia.ceph.com... 2024-03-29T04:18:43.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:43.363 DEBUG:teuthology.orchestra.run.smithi151:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-03-29T04:18:43.377 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:18:43.377 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-03-29T04:18:43.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:18:43.377 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-29T04:18:43.426 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-29T04:18:43.427 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:18:43.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd07d861640 1 Processor -- start 2024-03-29T04:18:43.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd07d861640 1 -- start start 2024-03-29T04:18:43.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd07d861640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd07806f390 0x7fd0781604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:43.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd07d861640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd07815c610 con 0x7fd07815c230 2024-03-29T04:18:43.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd07d861640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0780cc9f0 con 0x7fd0780cfd20 2024-03-29T04:18:43.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd07d861640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd0780b4fa0 con 0x7fd07806f390 2024-03-29T04:18:43.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd076ffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fd07815c230 0x7fd07815fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58604/0 (socket says 172.21.15.22:58604) 2024-03-29T04:18:43.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd076ffd640 1 -- 172.21.15.22:0/2158293610 learned_addr learned my addr 172.21.15.22:0/2158293610 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:43.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd0767fc640 1 --2- 172.21.15.22:0/2158293610 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd07806f390 0x7fd0781604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:43.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd075ffb640 1 -- 172.21.15.22:0/2158293610 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 708815985 0 0) 0x7fd07815c610 con 0x7fd07815c230 2024-03-29T04:18:43.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd075ffb640 1 -- 172.21.15.22:0/2158293610 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd060003530 con 0x7fd07815c230 2024-03-29T04:18:43.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd075ffb640 1 -- 172.21.15.22:0/2158293610 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1165357999 0 0) 0x7fd060003530 con 0x7fd07815c230 2024-03-29T04:18:43.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd075ffb640 1 -- 172.21.15.22:0/2158293610 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd07806f390 msgr2=0x7fd0781604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:43.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd075ffb640 1 --2- 172.21.15.22:0/2158293610 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd07806f390 0x7fd0781604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd075ffb640 1 -- 172.21.15.22:0/2158293610 >> v1:172.21.15.151:6789/0 conn(0x7fd0780cfd20 legacy=0x7fd0780d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:43.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd075ffb640 1 -- 172.21.15.22:0/2158293610 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0780cccd0 con 0x7fd07815c230 2024-03-29T04:18:43.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd075ffb640 1 -- 172.21.15.22:0/2158293610 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7fd068001680 con 0x7fd07815c230 2024-03-29T04:18:43.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd075ffb640 1 -- 172.21.15.22:0/2158293610 >> v1:172.21.15.22:6789/0 conn(0x7fd07815c230 legacy=0x7fd07815fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:43.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd075ffb640 1 --2- 172.21.15.22:0/2158293610 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd07806f390 0x7fd0600039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:43.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd075ffb640 1 --2- 172.21.15.22:0/2158293610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd060003f70 0x7fd060004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:43.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd075ffb640 1 --2- 172.21.15.22:0/2158293610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd060004d50 0x7fd0600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:43.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd075ffb640 1 -- 172.21.15.22:0/2158293610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd06000bb60 con 0x7fd060004d50 2024-03-29T04:18:43.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd075ffb640 1 -- 172.21.15.22:0/2158293610 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd06000bce0 con 0x7fd060003f70 2024-03-29T04:18:43.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd075ffb640 1 -- 172.21.15.22:0/2158293610 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd06000bee0 con 0x7fd07806f390 2024-03-29T04:18:43.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd0777fe640 1 --2- 172.21.15.22:0/2158293610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd060004d50 0x7fd0600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:43.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd076ffd640 1 --2- 172.21.15.22:0/2158293610 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd07806f390 0x7fd0600039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:43.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd0767fc640 1 --2- 172.21.15.22:0/2158293610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd060003f70 0x7fd060004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:43.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd0777fe640 1 -- 172.21.15.22:0/2158293610 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd07806f390 msgr2=0x7fd0600039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:43.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd0777fe640 1 --2- 172.21.15.22:0/2158293610 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd07806f390 0x7fd0600039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd0777fe640 1 -- 172.21.15.22:0/2158293610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd060003f70 msgr2=0x7fd060004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:43.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd0777fe640 1 --2- 172.21.15.22:0/2158293610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd060003f70 0x7fd060004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.576+0000 7fd0777fe640 1 -- 172.21.15.22:0/2158293610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd060003530 con 0x7fd060004d50 2024-03-29T04:18:43.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.580+0000 7fd0777fe640 1 --2- 172.21.15.22:0/2158293610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd060004d50 0x7fd0600051d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd070001e70 tx=0x7fd07001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1062d4574f3a7a41 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:43.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.580+0000 7fd075ffb640 1 -- 172.21.15.22:0/2158293610 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070001a90 con 0x7fd060004d50 2024-03-29T04:18:43.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.580+0000 7fd075ffb640 1 -- 172.21.15.22:0/2158293610 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd070003b80 con 0x7fd060004d50 2024-03-29T04:18:43.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.580+0000 7fd075ffb640 1 -- 172.21.15.22:0/2158293610 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070005080 con 0x7fd060004d50 2024-03-29T04:18:43.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.580+0000 7fd07d861640 1 -- 172.21.15.22:0/2158293610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd060004d50 msgr2=0x7fd0600051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:43.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.580+0000 7fd07d861640 1 --2- 172.21.15.22:0/2158293610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd060004d50 0x7fd0600051d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd070001e70 tx=0x7fd07001c780 comp rx=0 tx=0).stop 2024-03-29T04:18:43.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.580+0000 7fd0777fe640 1 -- 172.21.15.22:0/2158293610 reap_dead start 2024-03-29T04:18:43.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.580+0000 7fd07d861640 1 -- 172.21.15.22:0/2158293610 shutdown_connections 2024-03-29T04:18:43.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.580+0000 7fd07d861640 1 -- 172.21.15.22:0/2158293610 >> 172.21.15.22:0/2158293610 conn(0x7fd0780b3c80 msgr2=0x7fd0780b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:43.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.580+0000 7fd07d861640 1 -- 172.21.15.22:0/2158293610 shutdown_connections 2024-03-29T04:18:43.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.580+0000 7fd07d861640 1 -- 172.21.15.22:0/2158293610 wait complete. 2024-03-29T04:18:43.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.580+0000 7fd07d861640 1 Processor -- start 2024-03-29T04:18:43.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.580+0000 7fd07d861640 1 -- start start 2024-03-29T04:18:43.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd07d861640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd060004d50 0x7fd0781450f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:43.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd07d861640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd07815a280 0x7fd0780c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:43.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd07d861640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd07815a700 0x7fd0780c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:43.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd07d861640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd0780cc7a0 con 0x7fd07815a280 2024-03-29T04:18:43.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd07d861640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd0780cf4f0 con 0x7fd060004d50 2024-03-29T04:18:43.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd07d861640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd07806f820 con 0x7fd07815a700 2024-03-29T04:18:43.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd0767fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd07815a280 0x7fd0780c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:43.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd0767fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd07815a280 0x7fd0780c7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51472/0 (socket says 172.21.15.22:51472) 2024-03-29T04:18:43.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd0767fc640 1 -- 172.21.15.22:0/518513271 learned_addr learned my addr 172.21.15.22:0/518513271 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:43.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd076ffd640 1 --2- 172.21.15.22:0/518513271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd060004d50 0x7fd0781450f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:43.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd0777fe640 1 --2- 172.21.15.22:0/518513271 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd07815a700 0x7fd0780c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:43.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd0767fc640 1 -- 172.21.15.22:0/518513271 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd07815a700 msgr2=0x7fd0780c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:43.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd0767fc640 1 --2- 172.21.15.22:0/518513271 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd07815a700 0x7fd0780c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd0767fc640 1 -- 172.21.15.22:0/518513271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd060004d50 msgr2=0x7fd0781450f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:43.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd0767fc640 1 --2- 172.21.15.22:0/518513271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd060004d50 0x7fd0781450f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd0767fc640 1 -- 172.21.15.22:0/518513271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd078145630 con 0x7fd07815a280 2024-03-29T04:18:43.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd076ffd640 1 --2- 172.21.15.22:0/518513271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd060004d50 0x7fd0781450f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:43.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd0767fc640 1 --2- 172.21.15.22:0/518513271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd07815a280 0x7fd0780c7c40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0115e0 tx=0x7fd06c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:43.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd05ffff640 1 -- 172.21.15.22:0/518513271 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c00f8f0 con 0x7fd07815a280 2024-03-29T04:18:43.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd05ffff640 1 -- 172.21.15.22:0/518513271 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd06c00fa90 con 0x7fd07815a280 2024-03-29T04:18:43.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd07d861640 1 -- 172.21.15.22:0/518513271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd078153610 con 0x7fd07815a280 2024-03-29T04:18:43.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd05ffff640 1 -- 172.21.15.22:0/518513271 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c00fe40 con 0x7fd07815a280 2024-03-29T04:18:43.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd07d861640 1 -- 172.21.15.22:0/518513271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd078153bd0 con 0x7fd07815a280 2024-03-29T04:18:43.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd07d861640 1 -- 172.21.15.22:0/518513271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd078068890 con 0x7fd07815a280 2024-03-29T04:18:43.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd05ffff640 1 -- 172.21.15.22:0/518513271 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fd06c015460 con 0x7fd07815a280 2024-03-29T04:18:43.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd05ffff640 1 --2- 172.21.15.22:0/518513271 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd064067fc0 0x7fd06406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:43.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd05ffff640 1 -- 172.21.15.22:0/518513271 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fd06c0872a0 con 0x7fd07815a280 2024-03-29T04:18:43.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.584+0000 7fd076ffd640 1 --2- 172.21.15.22:0/518513271 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd064067fc0 0x7fd06406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:43.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.588+0000 7fd076ffd640 1 --2- 172.21.15.22:0/518513271 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd064067fc0 0x7fd06406a480 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd068001ec0 tx=0x7fd068009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:43.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.588+0000 7fd05ffff640 1 -- 172.21.15.22:0/518513271 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fd078145630 con 0x7fd07815a280 2024-03-29T04:18:43.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.780+0000 7fd07d861640 1 -- 172.21.15.22:0/518513271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd078070230 con 0x7fd07815a280 2024-03-29T04:18:43.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.780+0000 7fd05ffff640 1 -- 172.21.15.22:0/518513271 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7fd06c019030 con 0x7fd07815a280 2024-03-29T04:18:43.761 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:43.761 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3094991863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3094991863},{"type":"v1","addr":"172.21.15.22:6834","nonce":3094991863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2464050026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2464050026},{"type":"v1","addr":"172.21.15.151:6835","nonce":2464050026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2999052384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2999052384},{"type":"v1","addr":"172.21.15.22:6835","nonce":2999052384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:17:35.918069+0000","modified":"2024-03-29T04:17:53.156365+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6837/1466916671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":1466916671},{"type":"v1","addr":"172.21.15.151:6837","nonce":1466916671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-03-29T04:18:43.761 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2024-03-29T04:18:43.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.780+0000 7fd07d861640 1 -- 172.21.15.22:0/518513271 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd064067fc0 msgr2=0x7fd06406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:43.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.780+0000 7fd07d861640 1 --2- 172.21.15.22:0/518513271 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd064067fc0 0x7fd06406a480 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd068001ec0 tx=0x7fd068009040 comp rx=0 tx=0).stop 2024-03-29T04:18:43.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.780+0000 7fd07d861640 1 -- 172.21.15.22:0/518513271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd07815a280 msgr2=0x7fd0780c7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:43.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.780+0000 7fd07d861640 1 --2- 172.21.15.22:0/518513271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd07815a280 0x7fd0780c7c40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0115e0 tx=0x7fd06c011aa0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.780+0000 7fd07d861640 1 -- 172.21.15.22:0/518513271 shutdown_connections 2024-03-29T04:18:43.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.780+0000 7fd07d861640 1 --2- 172.21.15.22:0/518513271 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd07815a700 0x7fd0780c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.780+0000 7fd07d861640 1 --2- 172.21.15.22:0/518513271 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd064067fc0 0x7fd06406a480 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.780+0000 7fd07d861640 1 --2- 172.21.15.22:0/518513271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd07815a280 0x7fd0780c7c40 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.780+0000 7fd07d861640 1 --2- 172.21.15.22:0/518513271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd060004d50 0x7fd0781450f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.780+0000 7fd07d861640 1 -- 172.21.15.22:0/518513271 >> 172.21.15.22:0/518513271 conn(0x7fd0780b3c80 msgr2=0x7fd0780b5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:43.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.780+0000 7fd07d861640 1 -- 172.21.15.22:0/518513271 shutdown_connections 2024-03-29T04:18:43.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.780+0000 7fd07d861640 1 -- 172.21.15.22:0/518513271 wait complete. 2024-03-29T04:18:43.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:43.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.912+0000 7efebe455640 1 Processor -- start 2024-03-29T04:18:43.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.912+0000 7efebe455640 1 -- start start 2024-03-29T04:18:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.912+0000 7efebe455640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efeb8070490 0x7efeb815c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.912+0000 7efebe455640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7efeb806de90 0x7efeb806e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.912+0000 7efebe455640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeb815c670 con 0x7efeb80b5b20 2024-03-29T04:18:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.912+0000 7efebe455640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7efeb815edd0 con 0x7efeb8070490 2024-03-29T04:18:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.912+0000 7efebe455640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7efeb815f0b0 con 0x7efeb806de90 2024-03-29T04:18:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.916+0000 7efeb7fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efeb8070490 0x7efeb815c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.916+0000 7efeb7fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efeb8070490 0x7efeb815c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51484/0 (socket says 172.21.15.22:51484) 2024-03-29T04:18:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.916+0000 7efeb7fff640 1 -- 172.21.15.22:0/3268056085 learned_addr learned my addr 172.21.15.22:0/3268056085 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.916+0000 7efebc9f2640 1 --2- 172.21.15.22:0/3268056085 >> v2:172.21.15.151:3300/0 conn(0x7efeb806de90 0x7efeb806e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.916+0000 7efeb7fff640 1 -- 172.21.15.22:0/3268056085 >> v2:172.21.15.151:3300/0 conn(0x7efeb806de90 msgr2=0x7efeb806e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.916+0000 7efeb7fff640 1 --2- 172.21.15.22:0/3268056085 >> v2:172.21.15.151:3300/0 conn(0x7efeb806de90 0x7efeb806e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.916+0000 7efeb7fff640 1 -- 172.21.15.22:0/3268056085 >> v1:172.21.15.151:6789/0 conn(0x7efeb80b5b20 legacy=0x7efeb806d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.916+0000 7efeb7fff640 1 -- 172.21.15.22:0/3268056085 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb815f390 con 0x7efeb8070490 2024-03-29T04:18:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.916+0000 7efeb7fff640 1 --2- 172.21.15.22:0/3268056085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efeb8070490 0x7efeb815c130 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efeb80cd1a0 tx=0x7efea801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197c5ffe03131537 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.916+0000 7efeb77fe640 1 -- 172.21.15.22:0/3268056085 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea80b1070 con 0x7efeb8070490 2024-03-29T04:18:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.916+0000 7efeb77fe640 1 -- 172.21.15.22:0/3268056085 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea80ac640 con 0x7efeb8070490 2024-03-29T04:18:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.916+0000 7efeb77fe640 1 -- 172.21.15.22:0/3268056085 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea80aca10 con 0x7efeb8070490 2024-03-29T04:18:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.916+0000 7efebe455640 1 -- 172.21.15.22:0/3268056085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efeb8070490 msgr2=0x7efeb815c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.916+0000 7efebe455640 1 --2- 172.21.15.22:0/3268056085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efeb8070490 0x7efeb815c130 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efeb80cd1a0 tx=0x7efea801c400 comp rx=0 tx=0).stop 2024-03-29T04:18:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.916+0000 7efebe455640 1 -- 172.21.15.22:0/3268056085 shutdown_connections 2024-03-29T04:18:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.916+0000 7efebe455640 1 --2- 172.21.15.22:0/3268056085 >> v2:172.21.15.151:3300/0 conn(0x7efeb806de90 0x7efeb806e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.916+0000 7efebe455640 1 --2- 172.21.15.22:0/3268056085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efeb8070490 0x7efeb815c130 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.916+0000 7efebe455640 1 -- 172.21.15.22:0/3268056085 >> 172.21.15.22:0/3268056085 conn(0x7efeb8072500 msgr2=0x7efeb8072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.916+0000 7efebe455640 1 -- 172.21.15.22:0/3268056085 shutdown_connections 2024-03-29T04:18:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.916+0000 7efebe455640 1 -- 172.21.15.22:0/3268056085 wait complete. 2024-03-29T04:18:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efebe455640 1 Processor -- start 2024-03-29T04:18:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efebe455640 1 -- start start 2024-03-29T04:18:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efebe455640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeb806de90 0x7efeb814f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efebe455640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efeb8070490 0x7efeb8148a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efebe455640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efeb80b5b20 0x7efeb8155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efebe455640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efeb806f8f0 con 0x7efeb8070490 2024-03-29T04:18:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efebe455640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efeb806f770 con 0x7efeb806de90 2024-03-29T04:18:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efebe455640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7efeb806fa70 con 0x7efeb80b5b20 2024-03-29T04:18:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efeaf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efeb8070490 0x7efeb8148a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efeaf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efeb8070490 0x7efeb8148a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51500/0 (socket says 172.21.15.22:51500) 2024-03-29T04:18:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efeaf7fe640 1 -- 172.21.15.22:0/1602463798 learned_addr learned my addr 172.21.15.22:0/1602463798 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efeb7fff640 1 --2- 172.21.15.22:0/1602463798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeb806de90 0x7efeb814f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efeaf7fe640 1 -- 172.21.15.22:0/1602463798 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efeb80b5b20 msgr2=0x7efeb8155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efebc9f2640 1 --2- 172.21.15.22:0/1602463798 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efeb80b5b20 0x7efeb8155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efeaf7fe640 1 --2- 172.21.15.22:0/1602463798 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efeb80b5b20 0x7efeb8155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efeaf7fe640 1 -- 172.21.15.22:0/1602463798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeb806de90 msgr2=0x7efeb814f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efeaf7fe640 1 --2- 172.21.15.22:0/1602463798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeb806de90 0x7efeb814f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efeaf7fe640 1 -- 172.21.15.22:0/1602463798 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb8151ef0 con 0x7efeb8070490 2024-03-29T04:18:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efeaf7fe640 1 --2- 172.21.15.22:0/1602463798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efeb8070490 0x7efeb8148a60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7efea000c370 tx=0x7efea000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efeb57fa640 1 -- 172.21.15.22:0/1602463798 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea00052d0 con 0x7efeb8070490 2024-03-29T04:18:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efeb57fa640 1 -- 172.21.15.22:0/1602463798 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea0005470 con 0x7efeb8070490 2024-03-29T04:18:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efebe455640 1 -- 172.21.15.22:0/1602463798 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb8151c10 con 0x7efeb8070490 2024-03-29T04:18:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efeb57fa640 1 -- 172.21.15.22:0/1602463798 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea0007260 con 0x7efeb8070490 2024-03-29T04:18:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efebe455640 1 -- 172.21.15.22:0/1602463798 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb814ed10 con 0x7efeb8070490 2024-03-29T04:18:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.920+0000 7efebe455640 1 -- 172.21.15.22:0/1602463798 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb8068890 con 0x7efeb8070490 2024-03-29T04:18:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.924+0000 7efeb57fa640 1 -- 172.21.15.22:0/1602463798 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7efea0007400 con 0x7efeb8070490 2024-03-29T04:18:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.924+0000 7efeb57fa640 1 --2- 172.21.15.22:0/1602463798 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efe98067ff0 0x7efe9806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.924+0000 7efeb57fa640 1 -- 172.21.15.22:0/1602463798 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7efea0085ce0 con 0x7efeb8070490 2024-03-29T04:18:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.924+0000 7efeb7fff640 1 --2- 172.21.15.22:0/1602463798 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efe98067ff0 0x7efe9806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:43.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.924+0000 7efeb7fff640 1 --2- 172.21.15.22:0/1602463798 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efe98067ff0 0x7efe9806a4b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efea80ae000 tx=0x7efea80a6070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:43.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:43.928+0000 7efeb57fa640 1 -- 172.21.15.22:0/1602463798 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7efeb815c670 con 0x7efeb8070490 2024-03-29T04:18:44.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.084+0000 7efebe455640 1 -- 172.21.15.22:0/1602463798 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efeb80b0540 con 0x7efeb8070490 2024-03-29T04:18:44.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.088+0000 7efeb57fa640 1 -- 172.21.15.22:0/1602463798 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18847 (secure 0 0 0) 0x7efea0056120 con 0x7efeb8070490 2024-03-29T04:18:44.068 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:44.068 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:18:16.893439+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:44.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.088+0000 7efebe455640 1 -- 172.21.15.22:0/1602463798 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efe98067ff0 msgr2=0x7efe9806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:44.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.088+0000 7efebe455640 1 --2- 172.21.15.22:0/1602463798 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efe98067ff0 0x7efe9806a4b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efea80ae000 tx=0x7efea80a6070 comp rx=0 tx=0).stop 2024-03-29T04:18:44.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.088+0000 7efebe455640 1 -- 172.21.15.22:0/1602463798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efeb8070490 msgr2=0x7efeb8148a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:44.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.088+0000 7efebe455640 1 --2- 172.21.15.22:0/1602463798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efeb8070490 0x7efeb8148a60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7efea000c370 tx=0x7efea000c830 comp rx=0 tx=0).stop 2024-03-29T04:18:44.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.088+0000 7efebe455640 1 -- 172.21.15.22:0/1602463798 shutdown_connections 2024-03-29T04:18:44.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.088+0000 7efebe455640 1 --2- 172.21.15.22:0/1602463798 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7efeb80b5b20 0x7efeb8155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.088+0000 7efebe455640 1 --2- 172.21.15.22:0/1602463798 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7efe98067ff0 0x7efe9806a4b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.088+0000 7efebe455640 1 --2- 172.21.15.22:0/1602463798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efeb8070490 0x7efeb8148a60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.088+0000 7efebe455640 1 --2- 172.21.15.22:0/1602463798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeb806de90 0x7efeb814f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.088+0000 7efebe455640 1 -- 172.21.15.22:0/1602463798 >> 172.21.15.22:0/1602463798 conn(0x7efeb8072500 msgr2=0x7efeb8158b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:44.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.088+0000 7efebe455640 1 -- 172.21.15.22:0/1602463798 shutdown_connections 2024-03-29T04:18:44.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.088+0000 7efebe455640 1 -- 172.21.15.22:0/1602463798 wait complete. 2024-03-29T04:18:44.086 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:17:34.034142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:17:34.886659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:17:38.628632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:18:44.087 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:18:44.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.232+0000 7f6b6f921640 1 Processor -- start 2024-03-29T04:18:44.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.232+0000 7f6b6f921640 1 -- start start 2024-03-29T04:18:44.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.232+0000 7f6b6f921640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6b6813f140 0x7f6b681606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:44.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.232+0000 7f6b6f921640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f6b6815ff40 0x7f6b6806c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:44.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.232+0000 7f6b6f921640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b6806d710 con 0x7f6b6806cbb0 2024-03-29T04:18:44.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.232+0000 7f6b6f921640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f6b6806e320 con 0x7f6b6813f140 2024-03-29T04:18:44.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.232+0000 7f6b6f921640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f6b6806d9f0 con 0x7f6b6815ff40 2024-03-29T04:18:44.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.232+0000 7f6b6d6bd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6b6813f140 0x7f6b681606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:44.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.232+0000 7f6b6d6bd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6b6813f140 0x7f6b681606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51514/0 (socket says 172.21.15.22:51514) 2024-03-29T04:18:44.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.232+0000 7f6b6d6bd640 1 -- 172.21.15.22:0/4268782030 learned_addr learned my addr 172.21.15.22:0/4268782030 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:44.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.232+0000 7f6b6cebc640 1 --2- 172.21.15.22:0/4268782030 >> v2:172.21.15.151:3300/0 conn(0x7f6b6815ff40 0x7f6b6806c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:44.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.232+0000 7f6b6d6bd640 1 -- 172.21.15.22:0/4268782030 >> v2:172.21.15.151:3300/0 conn(0x7f6b6815ff40 msgr2=0x7f6b6806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:44.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.232+0000 7f6b6d6bd640 1 --2- 172.21.15.22:0/4268782030 >> v2:172.21.15.151:3300/0 conn(0x7f6b6815ff40 0x7f6b6806c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.232+0000 7f6b6d6bd640 1 -- 172.21.15.22:0/4268782030 >> v1:172.21.15.151:6789/0 conn(0x7f6b6806cbb0 legacy=0x7f6b68141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:44.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.232+0000 7f6b6d6bd640 1 -- 172.21.15.22:0/4268782030 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6806dcd0 con 0x7f6b6813f140 2024-03-29T04:18:44.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.232+0000 7f6b6cebc640 1 --2- 172.21.15.22:0/4268782030 >> v2:172.21.15.151:3300/0 conn(0x7f6b6815ff40 0x7f6b6806c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:44.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.236+0000 7f6b6d6bd640 1 --2- 172.21.15.22:0/4268782030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b6813f140 0x7f6b681606d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6b68153930 tx=0x7f6b6401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42f4492c5c4d81a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:44.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.236+0000 7f6b5ffff640 1 -- 172.21.15.22:0/4268782030 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b640ab5f0 con 0x7f6b6813f140 2024-03-29T04:18:44.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.236+0000 7f6b5ffff640 1 -- 172.21.15.22:0/4268782030 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b640b0070 con 0x7f6b6813f140 2024-03-29T04:18:44.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.236+0000 7f6b5ffff640 1 -- 172.21.15.22:0/4268782030 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b640ab9b0 con 0x7f6b6813f140 2024-03-29T04:18:44.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.236+0000 7f6b6f921640 1 -- 172.21.15.22:0/4268782030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b6813f140 msgr2=0x7f6b681606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:44.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.236+0000 7f6b6f921640 1 --2- 172.21.15.22:0/4268782030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b6813f140 0x7f6b681606d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6b68153930 tx=0x7f6b6401c4e0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.236+0000 7f6b6f921640 1 -- 172.21.15.22:0/4268782030 shutdown_connections 2024-03-29T04:18:44.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.236+0000 7f6b6f921640 1 --2- 172.21.15.22:0/4268782030 >> v2:172.21.15.151:3300/0 conn(0x7f6b6815ff40 0x7f6b6806c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.236+0000 7f6b6f921640 1 --2- 172.21.15.22:0/4268782030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b6813f140 0x7f6b681606d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.236+0000 7f6b6f921640 1 -- 172.21.15.22:0/4268782030 >> 172.21.15.22:0/4268782030 conn(0x7f6b680d00a0 msgr2=0x7f6b680d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:44.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.236+0000 7f6b6f921640 1 -- 172.21.15.22:0/4268782030 shutdown_connections 2024-03-29T04:18:44.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.236+0000 7f6b6f921640 1 -- 172.21.15.22:0/4268782030 wait complete. 2024-03-29T04:18:44.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.236+0000 7f6b6f921640 1 Processor -- start 2024-03-29T04:18:44.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.236+0000 7f6b6f921640 1 -- start start 2024-03-29T04:18:44.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b6f921640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b6806cbb0 0x7f6b6813e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:44.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b6f921640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6b6813f140 0x7f6b68154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:44.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b6f921640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b6815ff40 0x7f6b6815ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:44.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b6f921640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6b68169170 con 0x7f6b6806cbb0 2024-03-29T04:18:44.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b6f921640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6b68168ff0 con 0x7f6b6815ff40 2024-03-29T04:18:44.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b6f921640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f6b681692f0 con 0x7f6b6813f140 2024-03-29T04:18:44.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b6d6bd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b6806cbb0 0x7f6b6813e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:44.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b6d6bd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b6806cbb0 0x7f6b6813e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51520/0 (socket says 172.21.15.22:51520) 2024-03-29T04:18:44.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b6d6bd640 1 -- 172.21.15.22:0/147817408 learned_addr learned my addr 172.21.15.22:0/147817408 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:44.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b6debe640 1 --2- 172.21.15.22:0/147817408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b6815ff40 0x7f6b6815ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:44.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b6cebc640 1 --2- 172.21.15.22:0/147817408 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6b6813f140 0x7f6b68154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:44.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b6d6bd640 1 -- 172.21.15.22:0/147817408 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6b6813f140 msgr2=0x7f6b68154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:44.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b6d6bd640 1 --2- 172.21.15.22:0/147817408 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6b6813f140 0x7f6b68154710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b6d6bd640 1 -- 172.21.15.22:0/147817408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b6815ff40 msgr2=0x7f6b6815ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:44.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b6d6bd640 1 --2- 172.21.15.22:0/147817408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b6815ff40 0x7f6b6815ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b6d6bd640 1 -- 172.21.15.22:0/147817408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b68155010 con 0x7f6b6806cbb0 2024-03-29T04:18:44.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b6d6bd640 1 --2- 172.21.15.22:0/147817408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b6806cbb0 0x7f6b6813e250 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6b640ad040 tx=0x7f6b6401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:44.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b5dffb640 1 -- 172.21.15.22:0/147817408 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b640be5f0 con 0x7f6b6806cbb0 2024-03-29T04:18:44.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b5dffb640 1 -- 172.21.15.22:0/147817408 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b640be7b0 con 0x7f6b6806cbb0 2024-03-29T04:18:44.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b5dffb640 1 -- 172.21.15.22:0/147817408 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b640beba0 con 0x7f6b6806cbb0 2024-03-29T04:18:44.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b6f921640 1 -- 172.21.15.22:0/147817408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b681552f0 con 0x7f6b6806cbb0 2024-03-29T04:18:44.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b6f921640 1 -- 172.21.15.22:0/147817408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b681558b0 con 0x7f6b6806cbb0 2024-03-29T04:18:44.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b6f921640 1 -- 172.21.15.22:0/147817408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b68068900 con 0x7f6b6806cbb0 2024-03-29T04:18:44.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b5dffb640 1 -- 172.21.15.22:0/147817408 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f6b640b43f0 con 0x7f6b6806cbb0 2024-03-29T04:18:44.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b5dffb640 1 --2- 172.21.15.22:0/147817408 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6b4c067fc0 0x7f6b4c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:44.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.240+0000 7f6b5dffb640 1 -- 172.21.15.22:0/147817408 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f6b64121530 con 0x7f6b6806cbb0 2024-03-29T04:18:44.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.244+0000 7f6b6cebc640 1 --2- 172.21.15.22:0/147817408 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6b4c067fc0 0x7f6b4c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:44.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.244+0000 7f6b6cebc640 1 --2- 172.21.15.22:0/147817408 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6b4c067fc0 0x7f6b4c06a480 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6b58006fd0 tx=0x7f6b58008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:44.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.248+0000 7f6b5dffb640 1 -- 172.21.15.22:0/147817408 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f6b640c4060 con 0x7f6b6806cbb0 2024-03-29T04:18:44.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.408+0000 7f6b6f921640 1 -- 172.21.15.22:0/147817408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6b680cf410 con 0x7f6b6806cbb0 2024-03-29T04:18:44.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.408+0000 7f6b5dffb640 1 -- 172.21.15.22:0/147817408 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f6b640160d0 con 0x7f6b6806cbb0 2024-03-29T04:18:44.389 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:18:44.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.412+0000 7f6b6f921640 1 -- 172.21.15.22:0/147817408 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6b4c067fc0 msgr2=0x7f6b4c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:44.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.412+0000 7f6b6f921640 1 --2- 172.21.15.22:0/147817408 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6b4c067fc0 0x7f6b4c06a480 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6b58006fd0 tx=0x7f6b58008040 comp rx=0 tx=0).stop 2024-03-29T04:18:44.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.412+0000 7f6b6f921640 1 -- 172.21.15.22:0/147817408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b6806cbb0 msgr2=0x7f6b6813e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:44.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.412+0000 7f6b6f921640 1 --2- 172.21.15.22:0/147817408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b6806cbb0 0x7f6b6813e250 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6b640ad040 tx=0x7f6b6401ad20 comp rx=0 tx=0).stop 2024-03-29T04:18:44.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.412+0000 7f6b6f921640 1 -- 172.21.15.22:0/147817408 shutdown_connections 2024-03-29T04:18:44.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.412+0000 7f6b6f921640 1 --2- 172.21.15.22:0/147817408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b6815ff40 0x7f6b6815ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.412+0000 7f6b6f921640 1 --2- 172.21.15.22:0/147817408 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6b6813f140 0x7f6b68154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.412+0000 7f6b6f921640 1 --2- 172.21.15.22:0/147817408 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6b4c067fc0 0x7f6b4c06a480 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.412+0000 7f6b6f921640 1 --2- 172.21.15.22:0/147817408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b6806cbb0 0x7f6b6813e250 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.412+0000 7f6b6f921640 1 -- 172.21.15.22:0/147817408 >> 172.21.15.22:0/147817408 conn(0x7f6b680d00a0 msgr2=0x7f6b68071840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:44.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.412+0000 7f6b6f921640 1 -- 172.21.15.22:0/147817408 shutdown_connections 2024-03-29T04:18:44.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.412+0000 7f6b6f921640 1 -- 172.21.15.22:0/147817408 wait complete. 2024-03-29T04:18:44.409 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:18:44.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.548+0000 7f6ed8d05640 1 Processor -- start 2024-03-29T04:18:44.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.548+0000 7f6ed8d05640 1 -- start start 2024-03-29T04:18:44.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.552+0000 7f6ed8d05640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6ed41605d0 0x7f6ed40b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:44.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.552+0000 7f6ed8d05640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ed406e040 con 0x7f6ed40cccb0 2024-03-29T04:18:44.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.552+0000 7f6ed8d05640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ed406e990 con 0x7f6ed41459b0 2024-03-29T04:18:44.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.552+0000 7f6ed8d05640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f6ed406ec70 con 0x7f6ed41605d0 2024-03-29T04:18:44.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.552+0000 7f6ed1d9b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6ed41605d0 0x7f6ed40b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:44.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.552+0000 7f6ed2d9d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f6ed40cccb0 0x7f6ed406f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58608/0 (socket says 172.21.15.22:58608) 2024-03-29T04:18:44.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.552+0000 7f6ed2d9d640 1 -- 172.21.15.22:0/4143891637 learned_addr learned my addr 172.21.15.22:0/4143891637 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:44.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.552+0000 7f6ed1d9b640 1 -- 172.21.15.22:0/4143891637 >> v1:172.21.15.151:6789/0 conn(0x7f6ed41459b0 legacy=0x7f6ed415fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:44.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.552+0000 7f6ed1d9b640 1 -- 172.21.15.22:0/4143891637 >> v1:172.21.15.22:6789/0 conn(0x7f6ed40cccb0 legacy=0x7f6ed406f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:44.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.552+0000 7f6ed1d9b640 1 -- 172.21.15.22:0/4143891637 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed406ef50 con 0x7f6ed41605d0 2024-03-29T04:18:44.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.552+0000 7f6ed159a640 1 -- 172.21.15.22:0/4143891637 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2370347157 0 0) 0x7f6ed406e040 con 0x7f6ed40cccb0 2024-03-29T04:18:44.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.552+0000 7f6ed1d9b640 1 --2- 172.21.15.22:0/4143891637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ed41605d0 0x7f6ed40b3f70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0097c0 tx=0x7f6ebc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b06366b7da72822 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:44.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.552+0000 7f6ed159a640 1 -- 172.21.15.22:0/4143891637 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0b0070 con 0x7f6ed41605d0 2024-03-29T04:18:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.552+0000 7f6ed159a640 1 -- 172.21.15.22:0/4143891637 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ebc0ab600 con 0x7f6ed41605d0 2024-03-29T04:18:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.552+0000 7f6ed159a640 1 -- 172.21.15.22:0/4143891637 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0ab9d0 con 0x7f6ed41605d0 2024-03-29T04:18:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed8d05640 1 -- 172.21.15.22:0/4143891637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ed41605d0 msgr2=0x7f6ed40b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed8d05640 1 --2- 172.21.15.22:0/4143891637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ed41605d0 0x7f6ed40b3f70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0097c0 tx=0x7f6ebc01c460 comp rx=0 tx=0).stop 2024-03-29T04:18:44.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed8d05640 1 -- 172.21.15.22:0/4143891637 shutdown_connections 2024-03-29T04:18:44.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed8d05640 1 --2- 172.21.15.22:0/4143891637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ed41605d0 0x7f6ed40b3f70 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed8d05640 1 -- 172.21.15.22:0/4143891637 >> 172.21.15.22:0/4143891637 conn(0x7f6ed40b2c70 msgr2=0x7f6ed40b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:44.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed8d05640 1 -- 172.21.15.22:0/4143891637 shutdown_connections 2024-03-29T04:18:44.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed8d05640 1 -- 172.21.15.22:0/4143891637 wait complete. 2024-03-29T04:18:44.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed8d05640 1 Processor -- start 2024-03-29T04:18:44.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed8d05640 1 -- start start 2024-03-29T04:18:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed8d05640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6ed40cccb0 0x7f6ed4146aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed8d05640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ed41459b0 0x7f6ed41442a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed8d05640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ed41605d0 0x7f6ed40c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed8d05640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed4145f10 con 0x7f6ed41459b0 2024-03-29T04:18:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed8d05640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed4145d90 con 0x7f6ed41605d0 2024-03-29T04:18:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed8d05640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed406dbd0 con 0x7f6ed40cccb0 2024-03-29T04:18:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed1d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ed41459b0 0x7f6ed41442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed1d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ed41459b0 0x7f6ed41442a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51530/0 (socket says 172.21.15.22:51530) 2024-03-29T04:18:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed1d9b640 1 -- 172.21.15.22:0/440298850 learned_addr learned my addr 172.21.15.22:0/440298850 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed2d9d640 1 --2- 172.21.15.22:0/440298850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ed41605d0 0x7f6ed40c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed259c640 1 --2- 172.21.15.22:0/440298850 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6ed40cccb0 0x7f6ed4146aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed1d9b640 1 -- 172.21.15.22:0/440298850 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6ed40cccb0 msgr2=0x7f6ed4146aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed1d9b640 1 --2- 172.21.15.22:0/440298850 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6ed40cccb0 0x7f6ed4146aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed1d9b640 1 -- 172.21.15.22:0/440298850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ed41605d0 msgr2=0x7f6ed40c5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed1d9b640 1 --2- 172.21.15.22:0/440298850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ed41605d0 0x7f6ed40c5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed1d9b640 1 -- 172.21.15.22:0/440298850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed41447e0 con 0x7f6ed41459b0 2024-03-29T04:18:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed1d9b640 1 --2- 172.21.15.22:0/440298850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ed41459b0 0x7f6ed41442a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0097c0 tx=0x7f6ebc01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ecaffd640 1 -- 172.21.15.22:0/440298850 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0b0040 con 0x7f6ed41459b0 2024-03-29T04:18:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ecaffd640 1 -- 172.21.15.22:0/440298850 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ebc004230 con 0x7f6ed41459b0 2024-03-29T04:18:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed8d05640 1 -- 172.21.15.22:0/440298850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed4147130 con 0x7f6ed41459b0 2024-03-29T04:18:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ecaffd640 1 -- 172.21.15.22:0/440298850 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc003620 con 0x7f6ed41459b0 2024-03-29T04:18:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.556+0000 7f6ed8d05640 1 -- 172.21.15.22:0/440298850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed41476f0 con 0x7f6ed41459b0 2024-03-29T04:18:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.560+0000 7f6ed8d05640 1 -- 172.21.15.22:0/440298850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed4068900 con 0x7f6ed41459b0 2024-03-29T04:18:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.560+0000 7f6ecaffd640 1 -- 172.21.15.22:0/440298850 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f6ebc0ab4c0 con 0x7f6ed41459b0 2024-03-29T04:18:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.560+0000 7f6ecaffd640 1 --2- 172.21.15.22:0/440298850 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6eb4067fc0 0x7f6eb406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.560+0000 7f6ecaffd640 1 -- 172.21.15.22:0/440298850 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f6ebc120fc0 con 0x7f6ed41459b0 2024-03-29T04:18:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.560+0000 7f6ed259c640 1 --2- 172.21.15.22:0/440298850 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6eb4067fc0 0x7f6eb406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.560+0000 7f6ed259c640 1 --2- 172.21.15.22:0/440298850 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6eb4067fc0 0x7f6eb406a480 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4005530 tx=0x7f6ec4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:44.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.564+0000 7f6ecaffd640 1 -- 172.21.15.22:0/440298850 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f6ed41447e0 con 0x7f6ed41459b0 2024-03-29T04:18:44.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.728+0000 7f6ed8d05640 1 -- 172.21.15.22:0/440298850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6ed40cb650 con 0x7f6ed41459b0 2024-03-29T04:18:44.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.728+0000 7f6ecaffd640 1 -- 172.21.15.22:0/440298850 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f6ebc016050 con 0x7f6ed41459b0 2024-03-29T04:18:44.709 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.732+0000 7f6ed8d05640 1 -- 172.21.15.22:0/440298850 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6eb4067fc0 msgr2=0x7f6eb406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.732+0000 7f6ed8d05640 1 --2- 172.21.15.22:0/440298850 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6eb4067fc0 0x7f6eb406a480 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4005530 tx=0x7f6ec4009040 comp rx=0 tx=0).stop 2024-03-29T04:18:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.732+0000 7f6ed8d05640 1 -- 172.21.15.22:0/440298850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ed41459b0 msgr2=0x7f6ed41442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.732+0000 7f6ed8d05640 1 --2- 172.21.15.22:0/440298850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ed41459b0 0x7f6ed41442a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0097c0 tx=0x7f6ebc01aca0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.732+0000 7f6ed8d05640 1 -- 172.21.15.22:0/440298850 shutdown_connections 2024-03-29T04:18:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.732+0000 7f6ed8d05640 1 --2- 172.21.15.22:0/440298850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ed41605d0 0x7f6ed40c5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.732+0000 7f6ed8d05640 1 --2- 172.21.15.22:0/440298850 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6eb4067fc0 0x7f6eb406a480 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.732+0000 7f6ed8d05640 1 --2- 172.21.15.22:0/440298850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ed41459b0 0x7f6ed41442a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.732+0000 7f6ed8d05640 1 --2- 172.21.15.22:0/440298850 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6ed40cccb0 0x7f6ed4146aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.732+0000 7f6ed8d05640 1 -- 172.21.15.22:0/440298850 >> 172.21.15.22:0/440298850 conn(0x7f6ed40b2c70 msgr2=0x7f6ed40b4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.732+0000 7f6ed8d05640 1 -- 172.21.15.22:0/440298850 shutdown_connections 2024-03-29T04:18:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.732+0000 7f6ed8d05640 1 -- 172.21.15.22:0/440298850 wait complete. 2024-03-29T04:18:44.728 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:18:44.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.872+0000 7f7e52c9a640 1 Processor -- start 2024-03-29T04:18:44.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.872+0000 7f7e52c9a640 1 -- start start 2024-03-29T04:18:44.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.872+0000 7f7e52c9a640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f7e4c0b9e30 0x7f7e4c0cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:44.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.872+0000 7f7e52c9a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7e4c0cf4f0 0x7f7e4c0d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:44.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.872+0000 7f7e52c9a640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e4c06c850 con 0x7f7e4c15e4c0 2024-03-29T04:18:44.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.872+0000 7f7e52c9a640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f7e4c0ba290 con 0x7f7e4c0cf4f0 2024-03-29T04:18:44.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.872+0000 7f7e52c9a640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f7e4c15e8a0 con 0x7f7e4c0b9e30 2024-03-29T04:18:44.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.872+0000 7f7e51237640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7e4c0cf4f0 0x7f7e4c0d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:44.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.872+0000 7f7e51237640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7e4c0cf4f0 0x7f7e4c0d2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51538/0 (socket says 172.21.15.22:51538) 2024-03-29T04:18:44.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.872+0000 7f7e51237640 1 -- 172.21.15.22:0/1603961733 learned_addr learned my addr 172.21.15.22:0/1603961733 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:44.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.872+0000 7f7e4bfff640 1 --2- 172.21.15.22:0/1603961733 >> v2:172.21.15.151:3300/0 conn(0x7f7e4c0b9e30 0x7f7e4c0cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:44.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.872+0000 7f7e51237640 1 -- 172.21.15.22:0/1603961733 >> v2:172.21.15.151:3300/0 conn(0x7f7e4c0b9e30 msgr2=0x7f7e4c0cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:44.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.872+0000 7f7e51237640 1 --2- 172.21.15.22:0/1603961733 >> v2:172.21.15.151:3300/0 conn(0x7f7e4c0b9e30 0x7f7e4c0cefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.872+0000 7f7e51237640 1 -- 172.21.15.22:0/1603961733 >> v1:172.21.15.151:6789/0 conn(0x7f7e4c15e4c0 legacy=0x7f7e4c06c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:44.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.872+0000 7f7e51237640 1 -- 172.21.15.22:0/1603961733 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e4c06cb30 con 0x7f7e4c0cf4f0 2024-03-29T04:18:44.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.872+0000 7f7e4bfff640 1 --2- 172.21.15.22:0/1603961733 >> v2:172.21.15.151:3300/0 conn(0x7f7e4c0b9e30 0x7f7e4c0cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:44.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e51237640 1 --2- 172.21.15.22:0/1603961733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e4c0cf4f0 0x7f7e4c0d2d70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c0d2590 tx=0x7f7e4401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667beefe0b20c0ca server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:44.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e4b7fe640 1 -- 172.21.15.22:0/1603961733 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e440a56c0 con 0x7f7e4c0cf4f0 2024-03-29T04:18:44.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e4b7fe640 1 -- 172.21.15.22:0/1603961733 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e440a5860 con 0x7f7e4c0cf4f0 2024-03-29T04:18:44.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e4b7fe640 1 -- 172.21.15.22:0/1603961733 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e440a5c30 con 0x7f7e4c0cf4f0 2024-03-29T04:18:44.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e52c9a640 1 -- 172.21.15.22:0/1603961733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e4c0cf4f0 msgr2=0x7f7e4c0d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:44.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e52c9a640 1 --2- 172.21.15.22:0/1603961733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e4c0cf4f0 0x7f7e4c0d2d70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c0d2590 tx=0x7f7e4401c9f0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e52c9a640 1 -- 172.21.15.22:0/1603961733 shutdown_connections 2024-03-29T04:18:44.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e52c9a640 1 --2- 172.21.15.22:0/1603961733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e4c0cf4f0 0x7f7e4c0d2d70 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e52c9a640 1 --2- 172.21.15.22:0/1603961733 >> v2:172.21.15.151:3300/0 conn(0x7f7e4c0b9e30 0x7f7e4c0cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e52c9a640 1 -- 172.21.15.22:0/1603961733 >> 172.21.15.22:0/1603961733 conn(0x7f7e4c0b6410 msgr2=0x7f7e4c0d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:44.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e52c9a640 1 -- 172.21.15.22:0/1603961733 shutdown_connections 2024-03-29T04:18:44.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e52c9a640 1 -- 172.21.15.22:0/1603961733 wait complete. 2024-03-29T04:18:44.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e52c9a640 1 Processor -- start 2024-03-29T04:18:44.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e52c9a640 1 -- start start 2024-03-29T04:18:44.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e52c9a640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7e4c0b9e30 0x7f7e4c1507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:44.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e52c9a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e4c0cf4f0 0x7f7e4c0cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:44.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e52c9a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e4c15e4c0 0x7f7e4c15c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:44.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e52c9a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4c0d3ed0 con 0x7f7e4c0cf4f0 2024-03-29T04:18:44.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e52c9a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4c0d3d50 con 0x7f7e4c15e4c0 2024-03-29T04:18:44.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e52c9a640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7e4c0d4050 con 0x7f7e4c0b9e30 2024-03-29T04:18:44.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e4bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e4c0cf4f0 0x7f7e4c0cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:44.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e4bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e4c0cf4f0 0x7f7e4c0cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51542/0 (socket says 172.21.15.22:51542) 2024-03-29T04:18:44.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e4bfff640 1 -- 172.21.15.22:0/2595538834 learned_addr learned my addr 172.21.15.22:0/2595538834 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:44.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e51237640 1 --2- 172.21.15.22:0/2595538834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e4c15e4c0 0x7f7e4c15c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:44.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e50a36640 1 --2- 172.21.15.22:0/2595538834 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7e4c0b9e30 0x7f7e4c1507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:44.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e4bfff640 1 -- 172.21.15.22:0/2595538834 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7e4c0b9e30 msgr2=0x7f7e4c1507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:44.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e4bfff640 1 --2- 172.21.15.22:0/2595538834 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7e4c0b9e30 0x7f7e4c1507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e4bfff640 1 -- 172.21.15.22:0/2595538834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e4c15e4c0 msgr2=0x7f7e4c15c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:44.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e4bfff640 1 --2- 172.21.15.22:0/2595538834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e4c15e4c0 0x7f7e4c15c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:44.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.876+0000 7f7e4bfff640 1 -- 172.21.15.22:0/2595538834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e4c0cf930 con 0x7f7e4c0cf4f0 2024-03-29T04:18:44.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.880+0000 7f7e4bfff640 1 --2- 172.21.15.22:0/2595538834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e4c0cf4f0 0x7f7e4c0cb8b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000c7f0 tx=0x7f7e4000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:44.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.880+0000 7f7e497fa640 1 -- 172.21.15.22:0/2595538834 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40015070 con 0x7f7e4c0cf4f0 2024-03-29T04:18:44.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.880+0000 7f7e497fa640 1 -- 172.21.15.22:0/2595538834 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e40007820 con 0x7f7e4c0cf4f0 2024-03-29T04:18:44.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.880+0000 7f7e497fa640 1 -- 172.21.15.22:0/2595538834 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40007c10 con 0x7f7e4c0cf4f0 2024-03-29T04:18:44.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.880+0000 7f7e52c9a640 1 -- 172.21.15.22:0/2595538834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e4c150ce0 con 0x7f7e4c0cf4f0 2024-03-29T04:18:44.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.880+0000 7f7e52c9a640 1 -- 172.21.15.22:0/2595538834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e4c158590 con 0x7f7e4c0cf4f0 2024-03-29T04:18:44.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.880+0000 7f7e52c9a640 1 -- 172.21.15.22:0/2595538834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e4c068900 con 0x7f7e4c0cf4f0 2024-03-29T04:18:44.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.880+0000 7f7e497fa640 1 -- 172.21.15.22:0/2595538834 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f7e40018070 con 0x7f7e4c0cf4f0 2024-03-29T04:18:44.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.880+0000 7f7e497fa640 1 --2- 172.21.15.22:0/2595538834 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7e38067fc0 0x7f7e3806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:44.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.880+0000 7f7e497fa640 1 -- 172.21.15.22:0/2595538834 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f7e40086980 con 0x7f7e4c0cf4f0 2024-03-29T04:18:44.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.880+0000 7f7e50a36640 1 --2- 172.21.15.22:0/2595538834 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7e38067fc0 0x7f7e3806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:44.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.880+0000 7f7e50a36640 1 --2- 172.21.15.22:0/2595538834 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7e38067fc0 0x7f7e3806a480 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c0b88c0 tx=0x7f7e3c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:44.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:44.884+0000 7f7e497fa640 1 -- 172.21.15.22:0/2595538834 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f7e400566d0 con 0x7f7e4c0cf4f0 2024-03-29T04:18:45.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.048+0000 7f7e52c9a640 1 -- 172.21.15.22:0/2595538834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7e4c0b7bf0 con 0x7f7e4c0cf4f0 2024-03-29T04:18:45.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.048+0000 7f7e497fa640 1 -- 172.21.15.22:0/2595538834 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f7e4005a9d0 con 0x7f7e4c0cf4f0 2024-03-29T04:18:45.030 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-29T04:18:45.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.052+0000 7f7e52c9a640 1 -- 172.21.15.22:0/2595538834 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7e38067fc0 msgr2=0x7f7e3806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:45.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.052+0000 7f7e52c9a640 1 --2- 172.21.15.22:0/2595538834 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7e38067fc0 0x7f7e3806a480 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c0b88c0 tx=0x7f7e3c009040 comp rx=0 tx=0).stop 2024-03-29T04:18:45.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.052+0000 7f7e52c9a640 1 -- 172.21.15.22:0/2595538834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e4c0cf4f0 msgr2=0x7f7e4c0cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:45.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.052+0000 7f7e52c9a640 1 --2- 172.21.15.22:0/2595538834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e4c0cf4f0 0x7f7e4c0cb8b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000c7f0 tx=0x7f7e4000ccb0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.052+0000 7f7e52c9a640 1 -- 172.21.15.22:0/2595538834 shutdown_connections 2024-03-29T04:18:45.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.052+0000 7f7e52c9a640 1 --2- 172.21.15.22:0/2595538834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e4c15e4c0 0x7f7e4c15c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.052+0000 7f7e52c9a640 1 --2- 172.21.15.22:0/2595538834 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7e38067fc0 0x7f7e3806a480 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.052+0000 7f7e52c9a640 1 --2- 172.21.15.22:0/2595538834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e4c0cf4f0 0x7f7e4c0cb8b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.052+0000 7f7e52c9a640 1 --2- 172.21.15.22:0/2595538834 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7e4c0b9e30 0x7f7e4c1507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.052+0000 7f7e52c9a640 1 -- 172.21.15.22:0/2595538834 >> 172.21.15.22:0/2595538834 conn(0x7f7e4c0b6410 msgr2=0x7f7e4c0b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:45.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.052+0000 7f7e52c9a640 1 -- 172.21.15.22:0/2595538834 shutdown_connections 2024-03-29T04:18:45.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.052+0000 7f7e52c9a640 1 -- 172.21.15.22:0/2595538834 wait complete. 2024-03-29T04:18:45.050 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:18:45.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.192+0000 7f8e3d19d640 1 Processor -- start 2024-03-29T04:18:45.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.192+0000 7f8e3d19d640 1 -- start start 2024-03-29T04:18:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.192+0000 7f8e3d19d640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f8e3815e4c0 0x7f8e380b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.192+0000 7f8e3d19d640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e3806c850 con 0x7f8e3806c070 2024-03-29T04:18:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.192+0000 7f8e3d19d640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e3806cb30 con 0x7f8e380cf6c0 2024-03-29T04:18:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.192+0000 7f8e3d19d640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f8e3815e8a0 con 0x7f8e3815e4c0 2024-03-29T04:18:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.192+0000 7f8e3659c640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f8e3806c070 0x7f8e380cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42616/0 (socket says 172.21.15.22:42616) 2024-03-29T04:18:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.192+0000 7f8e3659c640 1 -- 172.21.15.22:0/3379538543 learned_addr learned my addr 172.21.15.22:0/3379538543 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.192+0000 7f8e36d9d640 1 --2- 172.21.15.22:0/3379538543 >> v2:172.21.15.151:3300/0 conn(0x7f8e3815e4c0 0x7f8e380b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.192+0000 7f8e35d9b640 1 -- 172.21.15.22:0/3379538543 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1772280232 0 0) 0x7f8e3806c850 con 0x7f8e3806c070 2024-03-29T04:18:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.192+0000 7f8e35d9b640 1 -- 172.21.15.22:0/3379538543 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e20003530 con 0x7f8e3806c070 2024-03-29T04:18:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.192+0000 7f8e35d9b640 1 -- 172.21.15.22:0/3379538543 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4128539562 0 0) 0x7f8e20003530 con 0x7f8e3806c070 2024-03-29T04:18:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.192+0000 7f8e36d9d640 1 -- 172.21.15.22:0/3379538543 >> v1:172.21.15.151:6789/0 conn(0x7f8e380cf6c0 legacy=0x7f8e380d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.192+0000 7f8e36d9d640 1 -- 172.21.15.22:0/3379538543 >> v1:172.21.15.22:6789/0 conn(0x7f8e3806c070 legacy=0x7f8e380cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.192+0000 7f8e36d9d640 1 -- 172.21.15.22:0/3379538543 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3806ce10 con 0x7f8e3815e4c0 2024-03-29T04:18:45.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.196+0000 7f8e36d9d640 1 --2- 172.21.15.22:0/3379538543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3815e4c0 0x7f8e380b9e30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8e280097c0 tx=0x7f8e2801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b06a406d5dbae75 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:45.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.196+0000 7f8e35d9b640 1 -- 172.21.15.22:0/3379538543 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e280b0070 con 0x7f8e3815e4c0 2024-03-29T04:18:45.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.196+0000 7f8e35d9b640 1 -- 172.21.15.22:0/3379538543 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e280ab400 con 0x7f8e3815e4c0 2024-03-29T04:18:45.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.196+0000 7f8e35d9b640 1 -- 172.21.15.22:0/3379538543 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c001680 con 0x7f8e3815e4c0 2024-03-29T04:18:45.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.196+0000 7f8e3d19d640 1 -- 172.21.15.22:0/3379538543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3815e4c0 msgr2=0x7f8e380b9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:45.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.196+0000 7f8e3d19d640 1 --2- 172.21.15.22:0/3379538543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3815e4c0 0x7f8e380b9e30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8e280097c0 tx=0x7f8e2801c3c0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.196+0000 7f8e3d19d640 1 -- 172.21.15.22:0/3379538543 shutdown_connections 2024-03-29T04:18:45.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.196+0000 7f8e3d19d640 1 --2- 172.21.15.22:0/3379538543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3815e4c0 0x7f8e380b9e30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.196+0000 7f8e3d19d640 1 -- 172.21.15.22:0/3379538543 >> 172.21.15.22:0/3379538543 conn(0x7f8e380b6410 msgr2=0x7f8e380d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:45.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.196+0000 7f8e3d19d640 1 -- 172.21.15.22:0/3379538543 shutdown_connections 2024-03-29T04:18:45.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.196+0000 7f8e3d19d640 1 -- 172.21.15.22:0/3379538543 wait complete. 2024-03-29T04:18:45.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.196+0000 7f8e3d19d640 1 Processor -- start 2024-03-29T04:18:45.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.196+0000 7f8e3d19d640 1 -- start start 2024-03-29T04:18:45.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.196+0000 7f8e3d19d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e3806c070 0x7f8e380b8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:45.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.196+0000 7f8e3d19d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e380cf6c0 0x7f8e381497a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:45.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.196+0000 7f8e3d19d640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e3815e4c0 0x7f8e381533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:45.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.196+0000 7f8e3d19d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3816c050 con 0x7f8e3806c070 2024-03-29T04:18:45.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.196+0000 7f8e3d19d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e380d3f60 con 0x7f8e380cf6c0 2024-03-29T04:18:45.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.196+0000 7f8e3d19d640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f8e3816c1d0 con 0x7f8e3815e4c0 2024-03-29T04:18:45.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e36d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e3806c070 0x7f8e380b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:45.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e36d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e3806c070 0x7f8e380b8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43666/0 (socket says 172.21.15.22:43666) 2024-03-29T04:18:45.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e36d9d640 1 -- 172.21.15.22:0/1356408375 learned_addr learned my addr 172.21.15.22:0/1356408375 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:45.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e3759e640 1 --2- 172.21.15.22:0/1356408375 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e3815e4c0 0x7f8e381533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:45.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e3659c640 1 --2- 172.21.15.22:0/1356408375 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e380cf6c0 0x7f8e381497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:45.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e36d9d640 1 -- 172.21.15.22:0/1356408375 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e3815e4c0 msgr2=0x7f8e381533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:45.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e36d9d640 1 --2- 172.21.15.22:0/1356408375 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e3815e4c0 0x7f8e381533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e36d9d640 1 -- 172.21.15.22:0/1356408375 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e380cf6c0 msgr2=0x7f8e381497a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:45.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e36d9d640 1 --2- 172.21.15.22:0/1356408375 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e380cf6c0 0x7f8e381497a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e36d9d640 1 -- 172.21.15.22:0/1356408375 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e380c65d0 con 0x7f8e3806c070 2024-03-29T04:18:45.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e3759e640 1 --2- 172.21.15.22:0/1356408375 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e3815e4c0 0x7f8e381533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:45.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e36d9d640 1 --2- 172.21.15.22:0/1356408375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e3806c070 0x7f8e380b8470 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8e28018fd0 tx=0x7f8e28019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:45.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e1f7fe640 1 -- 172.21.15.22:0/1356408375 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e280b9020 con 0x7f8e3806c070 2024-03-29T04:18:45.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e1f7fe640 1 -- 172.21.15.22:0/1356408375 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e28003ab0 con 0x7f8e3806c070 2024-03-29T04:18:45.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e3d19d640 1 -- 172.21.15.22:0/1356408375 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e380c68b0 con 0x7f8e3806c070 2024-03-29T04:18:45.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e1f7fe640 1 -- 172.21.15.22:0/1356408375 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e280b0040 con 0x7f8e3806c070 2024-03-29T04:18:45.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e3d19d640 1 -- 172.21.15.22:0/1356408375 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e380c6e70 con 0x7f8e3806c070 2024-03-29T04:18:45.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e3d19d640 1 -- 172.21.15.22:0/1356408375 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e38068900 con 0x7f8e3806c070 2024-03-29T04:18:45.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e1f7fe640 1 -- 172.21.15.22:0/1356408375 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f8e280bc020 con 0x7f8e3806c070 2024-03-29T04:18:45.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e1f7fe640 1 --2- 172.21.15.22:0/1356408375 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8e24067fc0 0x7f8e2406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:45.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e1f7fe640 1 -- 172.21.15.22:0/1356408375 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f8e28120e90 con 0x7f8e3806c070 2024-03-29T04:18:45.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.200+0000 7f8e3659c640 1 --2- 172.21.15.22:0/1356408375 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8e24067fc0 0x7f8e2406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:45.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.204+0000 7f8e3659c640 1 --2- 172.21.15.22:0/1356408375 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8e24067fc0 0x7f8e2406a480 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8e380b96c0 tx=0x7f8e2c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:45.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.204+0000 7f8e1f7fe640 1 -- 172.21.15.22:0/1356408375 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f8e280c3030 con 0x7f8e3806c070 2024-03-29T04:18:45.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.364+0000 7f8e3d19d640 1 -- 172.21.15.22:0/1356408375 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8e38159a70 con 0x7f8e3806c070 2024-03-29T04:18:45.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.368+0000 7f8e1f7fe640 1 -- 172.21.15.22:0/1356408375 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f8e28015fb0 con 0x7f8e3806c070 2024-03-29T04:18:45.347 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:18:45.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.368+0000 7f8e3d19d640 1 -- 172.21.15.22:0/1356408375 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8e24067fc0 msgr2=0x7f8e2406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:45.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.368+0000 7f8e3d19d640 1 --2- 172.21.15.22:0/1356408375 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8e24067fc0 0x7f8e2406a480 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8e380b96c0 tx=0x7f8e2c009040 comp rx=0 tx=0).stop 2024-03-29T04:18:45.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.368+0000 7f8e3d19d640 1 -- 172.21.15.22:0/1356408375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e3806c070 msgr2=0x7f8e380b8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:45.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.368+0000 7f8e3d19d640 1 --2- 172.21.15.22:0/1356408375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e3806c070 0x7f8e380b8470 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8e28018fd0 tx=0x7f8e28019160 comp rx=0 tx=0).stop 2024-03-29T04:18:45.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.368+0000 7f8e3d19d640 1 -- 172.21.15.22:0/1356408375 shutdown_connections 2024-03-29T04:18:45.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.368+0000 7f8e3d19d640 1 --2- 172.21.15.22:0/1356408375 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f8e3815e4c0 0x7f8e381533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.368+0000 7f8e3d19d640 1 --2- 172.21.15.22:0/1356408375 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e380cf6c0 0x7f8e381497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.368+0000 7f8e3d19d640 1 --2- 172.21.15.22:0/1356408375 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8e24067fc0 0x7f8e2406a480 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.368+0000 7f8e3d19d640 1 --2- 172.21.15.22:0/1356408375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e3806c070 0x7f8e380b8470 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.368+0000 7f8e3d19d640 1 -- 172.21.15.22:0/1356408375 >> 172.21.15.22:0/1356408375 conn(0x7f8e380b6410 msgr2=0x7f8e380b4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:45.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.368+0000 7f8e3d19d640 1 -- 172.21.15.22:0/1356408375 shutdown_connections 2024-03-29T04:18:45.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.368+0000 7f8e3d19d640 1 -- 172.21.15.22:0/1356408375 wait complete. 2024-03-29T04:18:45.368 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:18:45.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.516+0000 7f30c353b640 1 Processor -- start 2024-03-29T04:18:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.516+0000 7f30c353b640 1 -- start start 2024-03-29T04:18:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.516+0000 7f30c353b640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc144270 0x7f30bc144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:45.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.516+0000 7f30c353b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f30bc0b2ef0 0x7f30bc15f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:45.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.516+0000 7f30c353b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30bc0b33f0 con 0x7f30bc0b4490 2024-03-29T04:18:45.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.516+0000 7f30c353b640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f30bc144fa0 con 0x7f30bc0b2ef0 2024-03-29T04:18:45.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.516+0000 7f30c353b640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f30bc15fce0 con 0x7f30bc144270 2024-03-29T04:18:45.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.516+0000 7f30c0ad6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f30bc0b2ef0 0x7f30bc15f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:45.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.516+0000 7f30c1ad8640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f30bc0b4490 0x7f30bc06c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42622/0 (socket says 172.21.15.22:42622) 2024-03-29T04:18:45.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.516+0000 7f30c1ad8640 1 -- 172.21.15.22:0/1362028354 learned_addr learned my addr 172.21.15.22:0/1362028354 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:45.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.516+0000 7f30c12d7640 1 --2- 172.21.15.22:0/1362028354 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc144270 0x7f30bc144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:45.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.516+0000 7f30c0ad6640 1 -- 172.21.15.22:0/1362028354 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc144270 msgr2=0x7f30bc144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:45.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.516+0000 7f30c0ad6640 1 --2- 172.21.15.22:0/1362028354 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc144270 0x7f30bc144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.516+0000 7f30c0ad6640 1 -- 172.21.15.22:0/1362028354 >> v1:172.21.15.22:6789/0 conn(0x7f30bc0b4490 legacy=0x7f30bc06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:45.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.516+0000 7f30c0ad6640 1 -- 172.21.15.22:0/1362028354 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30bc0b36d0 con 0x7f30bc0b2ef0 2024-03-29T04:18:45.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c0ad6640 1 --2- 172.21.15.22:0/1362028354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc0b2ef0 0x7f30bc15f7a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0097c0 tx=0x7f30ac01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b3f0ec5c4fc06c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:45.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30b3fff640 1 -- 172.21.15.22:0/1362028354 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0a7070 con 0x7f30bc0b2ef0 2024-03-29T04:18:45.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30b3fff640 1 -- 172.21.15.22:0/1362028354 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30ac0b0070 con 0x7f30bc0b2ef0 2024-03-29T04:18:45.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30b3fff640 1 -- 172.21.15.22:0/1362028354 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0ab890 con 0x7f30bc0b2ef0 2024-03-29T04:18:45.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c353b640 1 -- 172.21.15.22:0/1362028354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc0b2ef0 msgr2=0x7f30bc15f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:45.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c353b640 1 --2- 172.21.15.22:0/1362028354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc0b2ef0 0x7f30bc15f7a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0097c0 tx=0x7f30ac01c470 comp rx=0 tx=0).stop 2024-03-29T04:18:45.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c353b640 1 -- 172.21.15.22:0/1362028354 shutdown_connections 2024-03-29T04:18:45.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c353b640 1 --2- 172.21.15.22:0/1362028354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc0b2ef0 0x7f30bc15f7a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c353b640 1 --2- 172.21.15.22:0/1362028354 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc144270 0x7f30bc144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c353b640 1 -- 172.21.15.22:0/1362028354 >> 172.21.15.22:0/1362028354 conn(0x7f30bc0cfee0 msgr2=0x7f30bc0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:45.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c353b640 1 -- 172.21.15.22:0/1362028354 shutdown_connections 2024-03-29T04:18:45.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c353b640 1 -- 172.21.15.22:0/1362028354 wait complete. 2024-03-29T04:18:45.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c353b640 1 Processor -- start 2024-03-29T04:18:45.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c353b640 1 -- start start 2024-03-29T04:18:45.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c353b640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc0b2ef0 0x7f30bc148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:45.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c353b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30bc0b4490 0x7f30bc14aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:45.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c353b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc144270 0x7f30bc06d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:45.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c353b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f30bc0cdd10 con 0x7f30bc144270 2024-03-29T04:18:45.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c353b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f30bc06cf90 con 0x7f30bc0b4490 2024-03-29T04:18:45.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c353b640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f30bc0cde90 con 0x7f30bc0b2ef0 2024-03-29T04:18:45.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c1ad8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc144270 0x7f30bc06d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:45.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c1ad8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc144270 0x7f30bc06d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43684/0 (socket says 172.21.15.22:43684) 2024-03-29T04:18:45.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c1ad8640 1 -- 172.21.15.22:0/1041345244 learned_addr learned my addr 172.21.15.22:0/1041345244 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:45.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c12d7640 1 --2- 172.21.15.22:0/1041345244 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc0b2ef0 0x7f30bc148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:45.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c0ad6640 1 --2- 172.21.15.22:0/1041345244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30bc0b4490 0x7f30bc14aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:45.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c1ad8640 1 -- 172.21.15.22:0/1041345244 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc0b2ef0 msgr2=0x7f30bc148490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:45.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c1ad8640 1 --2- 172.21.15.22:0/1041345244 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc0b2ef0 0x7f30bc148490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c1ad8640 1 -- 172.21.15.22:0/1041345244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30bc0b4490 msgr2=0x7f30bc14aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:45.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c1ad8640 1 --2- 172.21.15.22:0/1041345244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30bc0b4490 0x7f30bc14aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c1ad8640 1 -- 172.21.15.22:0/1041345244 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30bc13e3d0 con 0x7f30bc144270 2024-03-29T04:18:45.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.520+0000 7f30c12d7640 1 --2- 172.21.15.22:0/1041345244 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc0b2ef0 0x7f30bc148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:45.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.524+0000 7f30c1ad8640 1 --2- 172.21.15.22:0/1041345244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc144270 0x7f30bc06d8d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f30b400caa0 tx=0x7f30b400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:45.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.524+0000 7f30b1ffb640 1 -- 172.21.15.22:0/1041345244 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4017070 con 0x7f30bc144270 2024-03-29T04:18:45.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.524+0000 7f30b1ffb640 1 -- 172.21.15.22:0/1041345244 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b400ac00 con 0x7f30bc144270 2024-03-29T04:18:45.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.524+0000 7f30b1ffb640 1 -- 172.21.15.22:0/1041345244 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4012620 con 0x7f30bc144270 2024-03-29T04:18:45.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.524+0000 7f30c353b640 1 -- 172.21.15.22:0/1041345244 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30bc13e6b0 con 0x7f30bc144270 2024-03-29T04:18:45.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.524+0000 7f30c353b640 1 -- 172.21.15.22:0/1041345244 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30bc13fd80 con 0x7f30bc144270 2024-03-29T04:18:45.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.524+0000 7f30c353b640 1 -- 172.21.15.22:0/1041345244 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30bc068890 con 0x7f30bc144270 2024-03-29T04:18:45.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.524+0000 7f30b1ffb640 1 -- 172.21.15.22:0/1041345244 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f30b401a070 con 0x7f30bc144270 2024-03-29T04:18:45.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.524+0000 7f30b1ffb640 1 --2- 172.21.15.22:0/1041345244 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f30a0067fc0 0x7f30a006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:45.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.524+0000 7f30b1ffb640 1 -- 172.21.15.22:0/1041345244 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f30b4086650 con 0x7f30bc144270 2024-03-29T04:18:45.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.524+0000 7f30c12d7640 1 --2- 172.21.15.22:0/1041345244 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f30a0067fc0 0x7f30a006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:45.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.528+0000 7f30c12d7640 1 --2- 172.21.15.22:0/1041345244 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f30a0067fc0 0x7f30a006a480 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f30b8007a70 tx=0x7f30b8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:45.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.528+0000 7f30b1ffb640 1 -- 172.21.15.22:0/1041345244 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f30b40569f0 con 0x7f30bc144270 2024-03-29T04:18:45.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.724+0000 7f30c353b640 1 -- 172.21.15.22:0/1041345244 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30bc0cf110 con 0x7f30bc144270 2024-03-29T04:18:45.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.728+0000 7f30b1ffb640 1 -- 172.21.15.22:0/1041345244 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f30b40562f0 con 0x7f30bc144270 2024-03-29T04:18:45.708 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:45.708 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3094991863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3094991863},{"type":"v1","addr":"172.21.15.22:6834","nonce":3094991863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2464050026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2464050026},{"type":"v1","addr":"172.21.15.151:6835","nonce":2464050026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2999052384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2999052384},{"type":"v1","addr":"172.21.15.22:6835","nonce":2999052384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:17:35.918069+0000","modified":"2024-03-29T04:17:53.156365+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6837/1466916671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":1466916671},{"type":"v1","addr":"172.21.15.151:6837","nonce":1466916671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-03-29T04:18:45.708 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2024-03-29T04:18:45.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.728+0000 7f30c353b640 1 -- 172.21.15.22:0/1041345244 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f30a0067fc0 msgr2=0x7f30a006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:45.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.728+0000 7f30c353b640 1 --2- 172.21.15.22:0/1041345244 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f30a0067fc0 0x7f30a006a480 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f30b8007a70 tx=0x7f30b8008040 comp rx=0 tx=0).stop 2024-03-29T04:18:45.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.728+0000 7f30c353b640 1 -- 172.21.15.22:0/1041345244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc144270 msgr2=0x7f30bc06d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:45.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.728+0000 7f30c353b640 1 --2- 172.21.15.22:0/1041345244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc144270 0x7f30bc06d8d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f30b400caa0 tx=0x7f30b400cf60 comp rx=0 tx=0).stop 2024-03-29T04:18:45.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.728+0000 7f30c353b640 1 -- 172.21.15.22:0/1041345244 shutdown_connections 2024-03-29T04:18:45.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.728+0000 7f30c353b640 1 --2- 172.21.15.22:0/1041345244 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f30a0067fc0 0x7f30a006a480 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.728+0000 7f30c353b640 1 --2- 172.21.15.22:0/1041345244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc144270 0x7f30bc06d8d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.728+0000 7f30c353b640 1 --2- 172.21.15.22:0/1041345244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30bc0b4490 0x7f30bc14aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.728+0000 7f30c353b640 1 --2- 172.21.15.22:0/1041345244 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f30bc0b2ef0 0x7f30bc148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.728+0000 7f30c353b640 1 -- 172.21.15.22:0/1041345244 >> 172.21.15.22:0/1041345244 conn(0x7f30bc0cfee0 msgr2=0x7f30bc0ce180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:45.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.728+0000 7f30c353b640 1 -- 172.21.15.22:0/1041345244 shutdown_connections 2024-03-29T04:18:45.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.728+0000 7f30c353b640 1 -- 172.21.15.22:0/1041345244 wait complete. 2024-03-29T04:18:45.725 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:45.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.852+0000 7f99c11e3640 1 Processor -- start 2024-03-29T04:18:45.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.852+0000 7f99c11e3640 1 -- start start 2024-03-29T04:18:45.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.856+0000 7f99c11e3640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f99bc06c670 0x7f99bc0cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:45.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.856+0000 7f99c11e3640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f99bc0b41f0 0x7f99bc06e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:45.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.856+0000 7f99c11e3640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99bc06e5c0 con 0x7f99bc06ca50 2024-03-29T04:18:45.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.856+0000 7f99c11e3640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f99bc0b34c0 con 0x7f99bc06c670 2024-03-29T04:18:45.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.856+0000 7f99c11e3640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f99bc0ccef0 con 0x7f99bc0b41f0 2024-03-29T04:18:45.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.856+0000 7f99bad9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f99bc06c670 0x7f99bc0cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:45.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.856+0000 7f99bad9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f99bc06c670 0x7f99bc0cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43694/0 (socket says 172.21.15.22:43694) 2024-03-29T04:18:45.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.856+0000 7f99bad9d640 1 -- 172.21.15.22:0/882674716 learned_addr learned my addr 172.21.15.22:0/882674716 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:45.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.856+0000 7f99ba59c640 1 --2- 172.21.15.22:0/882674716 >> v2:172.21.15.151:3300/0 conn(0x7f99bc0b41f0 0x7f99bc06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:45.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.856+0000 7f99bad9d640 1 -- 172.21.15.22:0/882674716 >> v2:172.21.15.151:3300/0 conn(0x7f99bc0b41f0 msgr2=0x7f99bc06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:45.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.856+0000 7f99bad9d640 1 --2- 172.21.15.22:0/882674716 >> v2:172.21.15.151:3300/0 conn(0x7f99bc0b41f0 0x7f99bc06e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.856+0000 7f99bad9d640 1 -- 172.21.15.22:0/882674716 >> v1:172.21.15.151:6789/0 conn(0x7f99bc06ca50 legacy=0x7f99bc15fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:45.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.856+0000 7f99bad9d640 1 -- 172.21.15.22:0/882674716 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99bc0b3710 con 0x7f99bc06c670 2024-03-29T04:18:45.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.856+0000 7f99bad9d640 1 --2- 172.21.15.22:0/882674716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99bc06c670 0x7f99bc0cc910 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f99bc06ebe0 tx=0x7f99ac01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616358e0fd34802e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:45.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.856+0000 7f99b9d9b640 1 -- 172.21.15.22:0/882674716 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac0b0070 con 0x7f99bc06c670 2024-03-29T04:18:45.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.856+0000 7f99b9d9b640 1 -- 172.21.15.22:0/882674716 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99ac0ab600 con 0x7f99bc06c670 2024-03-29T04:18:45.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.856+0000 7f99b9d9b640 1 -- 172.21.15.22:0/882674716 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac0ab9d0 con 0x7f99bc06c670 2024-03-29T04:18:45.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99c11e3640 1 -- 172.21.15.22:0/882674716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99bc06c670 msgr2=0x7f99bc0cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:45.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99c11e3640 1 --2- 172.21.15.22:0/882674716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99bc06c670 0x7f99bc0cc910 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f99bc06ebe0 tx=0x7f99ac01c4e0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99c11e3640 1 -- 172.21.15.22:0/882674716 shutdown_connections 2024-03-29T04:18:45.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99c11e3640 1 --2- 172.21.15.22:0/882674716 >> v2:172.21.15.151:3300/0 conn(0x7f99bc0b41f0 0x7f99bc06e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99c11e3640 1 --2- 172.21.15.22:0/882674716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99bc06c670 0x7f99bc0cc910 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99c11e3640 1 -- 172.21.15.22:0/882674716 >> 172.21.15.22:0/882674716 conn(0x7f99bc0cfee0 msgr2=0x7f99bc0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:45.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99c11e3640 1 -- 172.21.15.22:0/882674716 shutdown_connections 2024-03-29T04:18:45.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99c11e3640 1 -- 172.21.15.22:0/882674716 wait complete. 2024-03-29T04:18:45.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99c11e3640 1 Processor -- start 2024-03-29T04:18:45.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99c11e3640 1 -- start start 2024-03-29T04:18:45.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99c11e3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99bc06c670 0x7f99bc15ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:45.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99c11e3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99bc06ca50 0x7f99bc161880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:45.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99c11e3640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f99bc0b41f0 0x7f99bc0714a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:45.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99c11e3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc06cfe0 con 0x7f99bc06c670 2024-03-29T04:18:45.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99c11e3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc06ce60 con 0x7f99bc06ca50 2024-03-29T04:18:45.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99c11e3640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f99bc168ef0 con 0x7f99bc0b41f0 2024-03-29T04:18:45.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99bad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99bc06c670 0x7f99bc15ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:45.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99bad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99bc06c670 0x7f99bc15ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43704/0 (socket says 172.21.15.22:43704) 2024-03-29T04:18:45.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99bad9d640 1 -- 172.21.15.22:0/1394186474 learned_addr learned my addr 172.21.15.22:0/1394186474 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:45.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99ba59c640 1 --2- 172.21.15.22:0/1394186474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99bc06ca50 0x7f99bc161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:45.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99bb59e640 1 --2- 172.21.15.22:0/1394186474 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f99bc0b41f0 0x7f99bc0714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:45.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99bad9d640 1 -- 172.21.15.22:0/1394186474 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f99bc0b41f0 msgr2=0x7f99bc0714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:45.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99bad9d640 1 --2- 172.21.15.22:0/1394186474 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f99bc0b41f0 0x7f99bc0714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99bad9d640 1 -- 172.21.15.22:0/1394186474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99bc06ca50 msgr2=0x7f99bc161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:45.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99bad9d640 1 --2- 172.21.15.22:0/1394186474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99bc06ca50 0x7f99bc161880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:45.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99bad9d640 1 -- 172.21.15.22:0/1394186474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99bc1655e0 con 0x7f99bc06c670 2024-03-29T04:18:45.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99bad9d640 1 --2- 172.21.15.22:0/1394186474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99bc06c670 0x7f99bc15ab40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0ae040 tx=0x7f99ac019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:45.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99a37fe640 1 -- 172.21.15.22:0/1394186474 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac0b0040 con 0x7f99bc06c670 2024-03-29T04:18:45.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99a37fe640 1 -- 172.21.15.22:0/1394186474 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99ac0b9070 con 0x7f99bc06c670 2024-03-29T04:18:45.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99c11e3640 1 -- 172.21.15.22:0/1394186474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99bc1658c0 con 0x7f99bc06c670 2024-03-29T04:18:45.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99a37fe640 1 -- 172.21.15.22:0/1394186474 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac0b48a0 con 0x7f99bc06c670 2024-03-29T04:18:45.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.860+0000 7f99c11e3640 1 -- 172.21.15.22:0/1394186474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99bc1621f0 con 0x7f99bc06c670 2024-03-29T04:18:45.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.864+0000 7f99c11e3640 1 -- 172.21.15.22:0/1394186474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99bc068890 con 0x7f99bc06c670 2024-03-29T04:18:45.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.864+0000 7f99a37fe640 1 -- 172.21.15.22:0/1394186474 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f99ac0ab4c0 con 0x7f99bc06c670 2024-03-29T04:18:45.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.864+0000 7f99a37fe640 1 --2- 172.21.15.22:0/1394186474 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f99a8067fc0 0x7f99a806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:45.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.864+0000 7f99a37fe640 1 -- 172.21.15.22:0/1394186474 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f99ac120510 con 0x7f99bc06c670 2024-03-29T04:18:45.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.864+0000 7f99ba59c640 1 --2- 172.21.15.22:0/1394186474 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f99a8067fc0 0x7f99a806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:45.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.864+0000 7f99ba59c640 1 --2- 172.21.15.22:0/1394186474 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f99a8067fc0 0x7f99a806a480 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0cf560 tx=0x7f99b0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:45.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:45.868+0000 7f99a37fe640 1 -- 172.21.15.22:0/1394186474 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f99ac0160d0 con 0x7f99bc06c670 2024-03-29T04:18:46.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.024+0000 7f99c11e3640 1 -- 172.21.15.22:0/1394186474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99bc145470 con 0x7f99bc06c670 2024-03-29T04:18:46.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.028+0000 7f99a37fe640 1 -- 172.21.15.22:0/1394186474 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18847 (secure 0 0 0) 0x7f99ac0f08c0 con 0x7f99bc06c670 2024-03-29T04:18:46.010 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:46.010 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:18:16.893439+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:46.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.032+0000 7f99c11e3640 1 -- 172.21.15.22:0/1394186474 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f99a8067fc0 msgr2=0x7f99a806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:46.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.032+0000 7f99c11e3640 1 --2- 172.21.15.22:0/1394186474 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f99a8067fc0 0x7f99a806a480 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0cf560 tx=0x7f99b0008040 comp rx=0 tx=0).stop 2024-03-29T04:18:46.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.032+0000 7f99c11e3640 1 -- 172.21.15.22:0/1394186474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99bc06c670 msgr2=0x7f99bc15ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:46.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.032+0000 7f99c11e3640 1 --2- 172.21.15.22:0/1394186474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99bc06c670 0x7f99bc15ab40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0ae040 tx=0x7f99ac019280 comp rx=0 tx=0).stop 2024-03-29T04:18:46.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.032+0000 7f99c11e3640 1 -- 172.21.15.22:0/1394186474 shutdown_connections 2024-03-29T04:18:46.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.032+0000 7f99c11e3640 1 --2- 172.21.15.22:0/1394186474 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f99bc0b41f0 0x7f99bc0714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.032+0000 7f99c11e3640 1 --2- 172.21.15.22:0/1394186474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99bc06ca50 0x7f99bc161880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.032+0000 7f99c11e3640 1 --2- 172.21.15.22:0/1394186474 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f99a8067fc0 0x7f99a806a480 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.032+0000 7f99c11e3640 1 --2- 172.21.15.22:0/1394186474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99bc06c670 0x7f99bc15ab40 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.032+0000 7f99c11e3640 1 -- 172.21.15.22:0/1394186474 >> 172.21.15.22:0/1394186474 conn(0x7f99bc0cfee0 msgr2=0x7f99bc0b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:46.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.032+0000 7f99c11e3640 1 -- 172.21.15.22:0/1394186474 shutdown_connections 2024-03-29T04:18:46.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.032+0000 7f99c11e3640 1 -- 172.21.15.22:0/1394186474 wait complete. 2024-03-29T04:18:46.028 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T04:18:46.029 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:18:46.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.180+0000 7f454fbf1640 1 Processor -- start 2024-03-29T04:18:46.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.180+0000 7f454fbf1640 1 -- start start 2024-03-29T04:18:46.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.184+0000 7f454fbf1640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f454806c670 0x7f45480cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:46.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.184+0000 7f454fbf1640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f45480b41f0 0x7f454806e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:46.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.184+0000 7f454fbf1640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f454806e5c0 con 0x7f454806ca50 2024-03-29T04:18:46.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.184+0000 7f454fbf1640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f45480b34c0 con 0x7f454806c670 2024-03-29T04:18:46.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.184+0000 7f454fbf1640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f45480ccef0 con 0x7f45480b41f0 2024-03-29T04:18:46.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.184+0000 7f454d98d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f454806c670 0x7f45480cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:46.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.184+0000 7f454d98d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f454806c670 0x7f45480cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43714/0 (socket says 172.21.15.22:43714) 2024-03-29T04:18:46.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.184+0000 7f454d98d640 1 -- 172.21.15.22:0/2610018968 learned_addr learned my addr 172.21.15.22:0/2610018968 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:46.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.184+0000 7f454d18c640 1 --2- 172.21.15.22:0/2610018968 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f45480b41f0 0x7f454806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:46.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.184+0000 7f454d98d640 1 -- 172.21.15.22:0/2610018968 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f45480b41f0 msgr2=0x7f454806e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:46.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.184+0000 7f454d98d640 1 --2- 172.21.15.22:0/2610018968 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f45480b41f0 0x7f454806e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.184+0000 7f454d98d640 1 -- 172.21.15.22:0/2610018968 >> v1:172.21.15.151:6789/0 conn(0x7f454806ca50 legacy=0x7f454815fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:46.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.184+0000 7f454d98d640 1 -- 172.21.15.22:0/2610018968 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45480b3710 con 0x7f454806c670 2024-03-29T04:18:46.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.184+0000 7f454d98d640 1 --2- 172.21.15.22:0/2610018968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f454806c670 0x7f45480cc910 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4544009870 tx=0x7f454401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf26e08e2632a64 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:46.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.184+0000 7f454c98b640 1 -- 172.21.15.22:0/2610018968 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45440b0070 con 0x7f454806c670 2024-03-29T04:18:46.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.184+0000 7f454c98b640 1 -- 172.21.15.22:0/2610018968 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45440ab600 con 0x7f454806c670 2024-03-29T04:18:46.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.184+0000 7f454c98b640 1 -- 172.21.15.22:0/2610018968 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45440ab9d0 con 0x7f454806c670 2024-03-29T04:18:46.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454fbf1640 1 -- 172.21.15.22:0/2610018968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f454806c670 msgr2=0x7f45480cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454fbf1640 1 --2- 172.21.15.22:0/2610018968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f454806c670 0x7f45480cc910 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4544009870 tx=0x7f454401c520 comp rx=0 tx=0).stop 2024-03-29T04:18:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454fbf1640 1 -- 172.21.15.22:0/2610018968 shutdown_connections 2024-03-29T04:18:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454fbf1640 1 --2- 172.21.15.22:0/2610018968 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f45480b41f0 0x7f454806e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454fbf1640 1 --2- 172.21.15.22:0/2610018968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f454806c670 0x7f45480cc910 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454fbf1640 1 -- 172.21.15.22:0/2610018968 >> 172.21.15.22:0/2610018968 conn(0x7f45480cfee0 msgr2=0x7f45480d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454fbf1640 1 -- 172.21.15.22:0/2610018968 shutdown_connections 2024-03-29T04:18:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454fbf1640 1 -- 172.21.15.22:0/2610018968 wait complete. 2024-03-29T04:18:46.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454fbf1640 1 Processor -- start 2024-03-29T04:18:46.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454fbf1640 1 -- start start 2024-03-29T04:18:46.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454fbf1640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f454806c670 0x7f45481f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:46.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454fbf1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f454806ca50 0x7f4548159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:46.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454fbf1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45480b41f0 0x7f45481f6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:46.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454fbf1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4548146000 con 0x7f454806ca50 2024-03-29T04:18:46.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454fbf1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4548145e80 con 0x7f45480b41f0 2024-03-29T04:18:46.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454fbf1640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4548168ef0 con 0x7f454806c670 2024-03-29T04:18:46.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454d18c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f454806ca50 0x7f4548159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:46.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454d18c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f454806ca50 0x7f4548159990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43728/0 (socket says 172.21.15.22:43728) 2024-03-29T04:18:46.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454d18c640 1 -- 172.21.15.22:0/234476063 learned_addr learned my addr 172.21.15.22:0/234476063 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:46.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454d98d640 1 --2- 172.21.15.22:0/234476063 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f454806c670 0x7f45481f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:46.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454e18e640 1 --2- 172.21.15.22:0/234476063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45480b41f0 0x7f45481f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:46.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454d18c640 1 -- 172.21.15.22:0/234476063 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f454806c670 msgr2=0x7f45481f7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:46.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454d18c640 1 --2- 172.21.15.22:0/234476063 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f454806c670 0x7f45481f7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454d18c640 1 -- 172.21.15.22:0/234476063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45480b41f0 msgr2=0x7f45481f6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:46.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454d18c640 1 --2- 172.21.15.22:0/234476063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45480b41f0 0x7f45481f6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454d18c640 1 -- 172.21.15.22:0/234476063 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45481f52a0 con 0x7f454806ca50 2024-03-29T04:18:46.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454d18c640 1 --2- 172.21.15.22:0/234476063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f454806ca50 0x7f4548159990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f453800c7f0 tx=0x7f453800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:46.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f453e7fc640 1 -- 172.21.15.22:0/234476063 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4538015070 con 0x7f454806ca50 2024-03-29T04:18:46.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f453e7fc640 1 -- 172.21.15.22:0/234476063 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4538004240 con 0x7f454806ca50 2024-03-29T04:18:46.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454fbf1640 1 -- 172.21.15.22:0/234476063 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45481f5580 con 0x7f454806ca50 2024-03-29T04:18:46.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f453e7fc640 1 -- 172.21.15.22:0/234476063 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4538003620 con 0x7f454806ca50 2024-03-29T04:18:46.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.188+0000 7f454fbf1640 1 -- 172.21.15.22:0/234476063 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45481f5b40 con 0x7f454806ca50 2024-03-29T04:18:46.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.192+0000 7f454fbf1640 1 -- 172.21.15.22:0/234476063 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4548068890 con 0x7f454806ca50 2024-03-29T04:18:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.192+0000 7f453e7fc640 1 -- 172.21.15.22:0/234476063 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f45380037c0 con 0x7f454806ca50 2024-03-29T04:18:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.192+0000 7f453e7fc640 1 --2- 172.21.15.22:0/234476063 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f452c067fc0 0x7f452c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.192+0000 7f453e7fc640 1 -- 172.21.15.22:0/234476063 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f4538086770 con 0x7f454806ca50 2024-03-29T04:18:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.192+0000 7f454d98d640 1 --2- 172.21.15.22:0/234476063 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f452c067fc0 0x7f452c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.192+0000 7f454d98d640 1 --2- 172.21.15.22:0/234476063 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f452c067fc0 0x7f452c06a480 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f45440ad000 tx=0x7f45440a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:46.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.196+0000 7f453e7fc640 1 -- 172.21.15.22:0/234476063 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f4538056bb0 con 0x7f454806ca50 2024-03-29T04:18:46.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.384+0000 7f454fbf1640 1 -- 172.21.15.22:0/234476063 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45480cc650 con 0x7f454806ca50 2024-03-29T04:18:46.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.388+0000 7f453e7fc640 1 -- 172.21.15.22:0/234476063 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f45380564b0 con 0x7f454806ca50 2024-03-29T04:18:46.368 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:46.368 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3094991863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3094991863},{"type":"v1","addr":"172.21.15.22:6834","nonce":3094991863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2464050026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2464050026},{"type":"v1","addr":"172.21.15.151:6835","nonce":2464050026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2999052384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2999052384},{"type":"v1","addr":"172.21.15.22:6835","nonce":2999052384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:17:35.918069+0000","modified":"2024-03-29T04:17:53.156365+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6837/1466916671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":1466916671},{"type":"v1","addr":"172.21.15.151:6837","nonce":1466916671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-03-29T04:18:46.368 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2024-03-29T04:18:46.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.388+0000 7f454fbf1640 1 -- 172.21.15.22:0/234476063 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f452c067fc0 msgr2=0x7f452c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:46.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.388+0000 7f454fbf1640 1 --2- 172.21.15.22:0/234476063 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f452c067fc0 0x7f452c06a480 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f45440ad000 tx=0x7f45440a6000 comp rx=0 tx=0).stop 2024-03-29T04:18:46.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.388+0000 7f454fbf1640 1 -- 172.21.15.22:0/234476063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f454806ca50 msgr2=0x7f4548159990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:46.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.388+0000 7f454fbf1640 1 --2- 172.21.15.22:0/234476063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f454806ca50 0x7f4548159990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f453800c7f0 tx=0x7f453800ccb0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.388+0000 7f454fbf1640 1 -- 172.21.15.22:0/234476063 shutdown_connections 2024-03-29T04:18:46.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.388+0000 7f454fbf1640 1 --2- 172.21.15.22:0/234476063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45480b41f0 0x7f45481f6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.388+0000 7f454fbf1640 1 --2- 172.21.15.22:0/234476063 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f452c067fc0 0x7f452c06a480 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.388+0000 7f454fbf1640 1 --2- 172.21.15.22:0/234476063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f454806ca50 0x7f4548159990 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.388+0000 7f454fbf1640 1 --2- 172.21.15.22:0/234476063 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f454806c670 0x7f45481f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.388+0000 7f454fbf1640 1 -- 172.21.15.22:0/234476063 >> 172.21.15.22:0/234476063 conn(0x7f45480cfee0 msgr2=0x7f454806d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:46.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.388+0000 7f454fbf1640 1 -- 172.21.15.22:0/234476063 shutdown_connections 2024-03-29T04:18:46.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.388+0000 7f454fbf1640 1 -- 172.21.15.22:0/234476063 wait complete. 2024-03-29T04:18:46.385 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:18:46.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.524+0000 7f0a02be9640 1 Processor -- start 2024-03-29T04:18:46.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.524+0000 7f0a02be9640 1 -- start start 2024-03-29T04:18:46.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f0a02be9640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09fc06c670 0x7f09fc0cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:46.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f0a02be9640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09fc0ccef0 con 0x7f09fc06ca50 2024-03-29T04:18:46.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f0a02be9640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09fc0b3710 con 0x7f09fc0b41f0 2024-03-29T04:18:46.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f0a02be9640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f09fc0b34c0 con 0x7f09fc06c670 2024-03-29T04:18:46.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f0a01186640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f09fc06ca50 0x7f09fc0ce5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42632/0 (socket says 172.21.15.22:42632) 2024-03-29T04:18:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f0a01186640 1 -- 172.21.15.22:0/1371220595 learned_addr learned my addr 172.21.15.22:0/1371220595 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f0a00985640 1 --2- 172.21.15.22:0/1371220595 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09fc06c670 0x7f09fc0cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1371220595 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2019073781 0 0) 0x7f09fc0ccef0 con 0x7f09fc06ca50 2024-03-29T04:18:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1371220595 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09e4003530 con 0x7f09fc06ca50 2024-03-29T04:18:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1371220595 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3502021853 0 0) 0x7f09e4003530 con 0x7f09fc06ca50 2024-03-29T04:18:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1371220595 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09fc06c670 msgr2=0x7f09fc0cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f09fb7fe640 1 --2- 172.21.15.22:0/1371220595 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09fc06c670 0x7f09fc0cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1371220595 >> v1:172.21.15.151:6789/0 conn(0x7f09fc0b41f0 legacy=0x7f09fc06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1371220595 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09fc0b39f0 con 0x7f09fc06ca50 2024-03-29T04:18:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1371220595 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f09f4001a00 con 0x7f09fc06ca50 2024-03-29T04:18:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1371220595 >> v1:172.21.15.22:6789/0 conn(0x7f09fc06ca50 legacy=0x7f09fc0ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f09fb7fe640 1 --2- 172.21.15.22:0/1371220595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09e40039f0 0x7f09e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f09fb7fe640 1 --2- 172.21.15.22:0/1371220595 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09fc06c670 0x7f09e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f09fb7fe640 1 --2- 172.21.15.22:0/1371220595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09e4004d50 0x7f09e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1371220595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc0b34c0 con 0x7f09e4004d50 2024-03-29T04:18:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1371220595 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f09e400b9d0 con 0x7f09e40039f0 2024-03-29T04:18:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1371220595 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f09e400bbd0 con 0x7f09fc06c670 2024-03-29T04:18:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f0a01186640 1 --2- 172.21.15.22:0/1371220595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09e4004d50 0x7f09e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f0a00985640 1 --2- 172.21.15.22:0/1371220595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09e40039f0 0x7f09e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f09fbfff640 1 --2- 172.21.15.22:0/1371220595 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09fc06c670 0x7f09e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f0a01186640 1 -- 172.21.15.22:0/1371220595 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09fc06c670 msgr2=0x7f09e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f0a01186640 1 --2- 172.21.15.22:0/1371220595 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09fc06c670 0x7f09e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f0a01186640 1 -- 172.21.15.22:0/1371220595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09e40039f0 msgr2=0x7f09e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:46.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f0a01186640 1 --2- 172.21.15.22:0/1371220595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09e40039f0 0x7f09e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f0a01186640 1 -- 172.21.15.22:0/1371220595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e4003530 con 0x7f09e4004d50 2024-03-29T04:18:46.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.528+0000 7f09fbfff640 1 --2- 172.21.15.22:0/1371220595 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09fc06c670 0x7f09e4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:46.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f0a01186640 1 --2- 172.21.15.22:0/1371220595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09e4004d50 0x7f09e40051d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f09f400a880 tx=0x7f09f401fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e19f7adbd416be4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:46.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1371220595 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f400a4a0 con 0x7f09e4004d50 2024-03-29T04:18:46.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1371220595 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09f40aa730 con 0x7f09e4004d50 2024-03-29T04:18:46.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1371220595 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f40aab20 con 0x7f09e4004d50 2024-03-29T04:18:46.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f0a02be9640 1 -- 172.21.15.22:0/1371220595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09e4004d50 msgr2=0x7f09e40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:46.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f0a02be9640 1 --2- 172.21.15.22:0/1371220595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09e4004d50 0x7f09e40051d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f09f400a880 tx=0x7f09f401fae0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f0a01186640 1 -- 172.21.15.22:0/1371220595 reap_dead start 2024-03-29T04:18:46.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f0a02be9640 1 -- 172.21.15.22:0/1371220595 shutdown_connections 2024-03-29T04:18:46.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f0a02be9640 1 -- 172.21.15.22:0/1371220595 >> 172.21.15.22:0/1371220595 conn(0x7f09fc0cfee0 msgr2=0x7f09fc0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:46.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f0a02be9640 1 -- 172.21.15.22:0/1371220595 shutdown_connections 2024-03-29T04:18:46.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f0a02be9640 1 -- 172.21.15.22:0/1371220595 wait complete. 2024-03-29T04:18:46.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f0a02be9640 1 Processor -- start 2024-03-29T04:18:46.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f0a02be9640 1 -- start start 2024-03-29T04:18:46.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f0a02be9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09fc135960 0x7f09fc1f2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:46.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f0a02be9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09fc0c96d0 0x7f09fc0c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:46.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f0a02be9640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09fc0c7f50 0x7f09fc0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:46.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f0a02be9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc06d000 con 0x7f09fc0c96d0 2024-03-29T04:18:46.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f0a02be9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc06ce80 con 0x7f09fc135960 2024-03-29T04:18:46.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f0a02be9640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f09fc145e80 con 0x7f09fc0c7f50 2024-03-29T04:18:46.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f09fbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09fc0c96d0 0x7f09fc0c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:46.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f09fbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09fc0c96d0 0x7f09fc0c9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43740/0 (socket says 172.21.15.22:43740) 2024-03-29T04:18:46.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f09fbfff640 1 -- 172.21.15.22:0/4143326927 learned_addr learned my addr 172.21.15.22:0/4143326927 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:46.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f0a00985640 1 --2- 172.21.15.22:0/4143326927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09fc135960 0x7f09fc1f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:46.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f0a01186640 1 --2- 172.21.15.22:0/4143326927 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09fc0c7f50 0x7f09fc0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:46.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f09fbfff640 1 -- 172.21.15.22:0/4143326927 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09fc0c7f50 msgr2=0x7f09fc0c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:46.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f09fbfff640 1 --2- 172.21.15.22:0/4143326927 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09fc0c7f50 0x7f09fc0c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f09fbfff640 1 -- 172.21.15.22:0/4143326927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09fc135960 msgr2=0x7f09fc1f2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:46.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f09fbfff640 1 --2- 172.21.15.22:0/4143326927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09fc135960 0x7f09fc1f2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f09fbfff640 1 -- 172.21.15.22:0/4143326927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09fc0c7190 con 0x7f09fc0c96d0 2024-03-29T04:18:46.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f09fbfff640 1 --2- 172.21.15.22:0/4143326927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09fc0c96d0 0x7f09fc0c9b50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00d7f0 tx=0x7f09ec00dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:46.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f09f97fa640 1 -- 172.21.15.22:0/4143326927 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec005480 con 0x7f09fc0c96d0 2024-03-29T04:18:46.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f09f97fa640 1 -- 172.21.15.22:0/4143326927 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09ec017070 con 0x7f09fc0c96d0 2024-03-29T04:18:46.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f0a02be9640 1 -- 172.21.15.22:0/4143326927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09fc0c7470 con 0x7f09fc0c96d0 2024-03-29T04:18:46.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f09f97fa640 1 -- 172.21.15.22:0/4143326927 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec00b8f0 con 0x7f09fc0c96d0 2024-03-29T04:18:46.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.532+0000 7f0a02be9640 1 -- 172.21.15.22:0/4143326927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09fc1f2690 con 0x7f09fc0c96d0 2024-03-29T04:18:46.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.536+0000 7f0a02be9640 1 -- 172.21.15.22:0/4143326927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09fc068890 con 0x7f09fc0c96d0 2024-03-29T04:18:46.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.536+0000 7f09f97fa640 1 -- 172.21.15.22:0/4143326927 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f09ec00ba90 con 0x7f09fc0c96d0 2024-03-29T04:18:46.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.536+0000 7f09f97fa640 1 --2- 172.21.15.22:0/4143326927 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f09e8067ff0 0x7f09e806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:46.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.536+0000 7f09f97fa640 1 -- 172.21.15.22:0/4143326927 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f09ec086810 con 0x7f09fc0c96d0 2024-03-29T04:18:46.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.536+0000 7f0a00985640 1 --2- 172.21.15.22:0/4143326927 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f09e8067ff0 0x7f09e806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:46.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.536+0000 7f0a00985640 1 --2- 172.21.15.22:0/4143326927 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f09e8067ff0 0x7f09e806a4b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009a30 tx=0x7f09f0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:46.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.540+0000 7f09f97fa640 1 -- 172.21.15.22:0/4143326927 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f09ec019310 con 0x7f09fc0c96d0 2024-03-29T04:18:46.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.728+0000 7f0a02be9640 1 -- 172.21.15.22:0/4143326927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09fc0b32a0 con 0x7f09fc0c96d0 2024-03-29T04:18:46.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.728+0000 7f09f97fa640 1 -- 172.21.15.22:0/4143326927 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f09ec056c50 con 0x7f09fc0c96d0 2024-03-29T04:18:46.711 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:46.711 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3094991863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3094991863},{"type":"v1","addr":"172.21.15.22:6834","nonce":3094991863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2464050026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2464050026},{"type":"v1","addr":"172.21.15.151:6835","nonce":2464050026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2999052384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2999052384},{"type":"v1","addr":"172.21.15.22:6835","nonce":2999052384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:17:35.918069+0000","modified":"2024-03-29T04:17:53.156365+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6837/1466916671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":1466916671},{"type":"v1","addr":"172.21.15.151:6837","nonce":1466916671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-03-29T04:18:46.712 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2024-03-29T04:18:46.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.732+0000 7f0a02be9640 1 -- 172.21.15.22:0/4143326927 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f09e8067ff0 msgr2=0x7f09e806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:46.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.732+0000 7f0a02be9640 1 --2- 172.21.15.22:0/4143326927 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f09e8067ff0 0x7f09e806a4b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009a30 tx=0x7f09f0008040 comp rx=0 tx=0).stop 2024-03-29T04:18:46.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.732+0000 7f0a02be9640 1 -- 172.21.15.22:0/4143326927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09fc0c96d0 msgr2=0x7f09fc0c9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:46.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.732+0000 7f0a02be9640 1 --2- 172.21.15.22:0/4143326927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09fc0c96d0 0x7f09fc0c9b50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00d7f0 tx=0x7f09ec00dcb0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.732+0000 7f0a02be9640 1 -- 172.21.15.22:0/4143326927 shutdown_connections 2024-03-29T04:18:46.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.732+0000 7f0a02be9640 1 --2- 172.21.15.22:0/4143326927 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09fc0c7f50 0x7f09fc0c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.732+0000 7f0a02be9640 1 --2- 172.21.15.22:0/4143326927 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f09e8067ff0 0x7f09e806a4b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.732+0000 7f0a02be9640 1 --2- 172.21.15.22:0/4143326927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09fc0c96d0 0x7f09fc0c9b50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.732+0000 7f0a02be9640 1 --2- 172.21.15.22:0/4143326927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09fc135960 0x7f09fc1f2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.732+0000 7f0a02be9640 1 -- 172.21.15.22:0/4143326927 >> 172.21.15.22:0/4143326927 conn(0x7f09fc0cfee0 msgr2=0x7f09fc1405d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:46.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.732+0000 7f0a02be9640 1 -- 172.21.15.22:0/4143326927 shutdown_connections 2024-03-29T04:18:46.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.732+0000 7f0a02be9640 1 -- 172.21.15.22:0/4143326927 wait complete. 2024-03-29T04:18:46.729 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:46.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.872+0000 7f3b54526640 1 Processor -- start 2024-03-29T04:18:46.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.872+0000 7f3b54526640 1 -- start start 2024-03-29T04:18:46.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.876+0000 7f3b54526640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3b4c06c670 0x7f3b4c0cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:46.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.876+0000 7f3b54526640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f3b4c0b41f0 0x7f3b4c06e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:46.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.876+0000 7f3b54526640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b4c06ca50 0x7f3b4c15fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:46.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.876+0000 7f3b54526640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f3b4c0b34c0 con 0x7f3b4c06c670 2024-03-29T04:18:46.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.876+0000 7f3b54526640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f3b4c06e5c0 con 0x7f3b4c0b41f0 2024-03-29T04:18:46.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.876+0000 7f3b54526640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4c0ccef0 con 0x7f3b4c06ca50 2024-03-29T04:18:46.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.876+0000 7f3b522c2640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3b4c06c670 0x7f3b4c0cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:46.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.876+0000 7f3b522c2640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3b4c06c670 0x7f3b4c0cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43746/0 (socket says 172.21.15.22:43746) 2024-03-29T04:18:46.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.876+0000 7f3b522c2640 1 -- 172.21.15.22:0/1453735376 learned_addr learned my addr 172.21.15.22:0/1453735376 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:46.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.876+0000 7f3b51ac1640 1 --2- 172.21.15.22:0/1453735376 >> v2:172.21.15.151:3300/0 conn(0x7f3b4c0b41f0 0x7f3b4c06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:46.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.876+0000 7f3b52ac3640 1 --2- 172.21.15.22:0/1453735376 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b4c06ca50 0x7f3b4c15fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:46.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.876+0000 7f3b522c2640 1 -- 172.21.15.22:0/1453735376 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b4c06ca50 msgr2=0x7f3b4c15fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:46.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.876+0000 7f3b522c2640 1 --2- 172.21.15.22:0/1453735376 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b4c06ca50 0x7f3b4c15fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.876+0000 7f3b522c2640 1 -- 172.21.15.22:0/1453735376 >> v2:172.21.15.151:3300/0 conn(0x7f3b4c0b41f0 msgr2=0x7f3b4c06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:46.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.876+0000 7f3b522c2640 1 --2- 172.21.15.22:0/1453735376 >> v2:172.21.15.151:3300/0 conn(0x7f3b4c0b41f0 0x7f3b4c06e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.876+0000 7f3b522c2640 1 -- 172.21.15.22:0/1453735376 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c0b3710 con 0x7f3b4c06c670 2024-03-29T04:18:46.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.876+0000 7f3b522c2640 1 --2- 172.21.15.22:0/1453735376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4c06c670 0x7f3b4c0cc910 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c06ebe0 tx=0x7f3b4801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8185c0b662492702 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:46.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.876+0000 7f3b512c0640 1 -- 172.21.15.22:0/1453735376 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b480b0070 con 0x7f3b4c06c670 2024-03-29T04:18:46.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.876+0000 7f3b512c0640 1 -- 172.21.15.22:0/1453735376 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b480ab600 con 0x7f3b4c06c670 2024-03-29T04:18:46.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.876+0000 7f3b512c0640 1 -- 172.21.15.22:0/1453735376 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b480ab9d0 con 0x7f3b4c06c670 2024-03-29T04:18:46.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b54526640 1 -- 172.21.15.22:0/1453735376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4c06c670 msgr2=0x7f3b4c0cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:46.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b54526640 1 --2- 172.21.15.22:0/1453735376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4c06c670 0x7f3b4c0cc910 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c06ebe0 tx=0x7f3b4801c260 comp rx=0 tx=0).stop 2024-03-29T04:18:46.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b54526640 1 -- 172.21.15.22:0/1453735376 shutdown_connections 2024-03-29T04:18:46.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b54526640 1 --2- 172.21.15.22:0/1453735376 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b4c06ca50 0x7f3b4c15fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b54526640 1 --2- 172.21.15.22:0/1453735376 >> v2:172.21.15.151:3300/0 conn(0x7f3b4c0b41f0 0x7f3b4c06e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b54526640 1 --2- 172.21.15.22:0/1453735376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4c06c670 0x7f3b4c0cc910 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b54526640 1 -- 172.21.15.22:0/1453735376 >> 172.21.15.22:0/1453735376 conn(0x7f3b4c0cfee0 msgr2=0x7f3b4c0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:46.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b54526640 1 -- 172.21.15.22:0/1453735376 shutdown_connections 2024-03-29T04:18:46.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b54526640 1 -- 172.21.15.22:0/1453735376 wait complete. 2024-03-29T04:18:46.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b54526640 1 Processor -- start 2024-03-29T04:18:46.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b54526640 1 -- start start 2024-03-29T04:18:46.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b54526640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b4c06c670 0x7f3b4c1fb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:46.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b54526640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4c06ca50 0x7f3b4c15d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:46.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b54526640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b4c0b41f0 0x7f3b4c15f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:46.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b54526640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c06cfe0 con 0x7f3b4c06ca50 2024-03-29T04:18:46.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b54526640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c06ce60 con 0x7f3b4c0b41f0 2024-03-29T04:18:46.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b54526640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4c0ce5b0 con 0x7f3b4c06c670 2024-03-29T04:18:46.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b51ac1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4c06ca50 0x7f3b4c15d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:46.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b51ac1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4c06ca50 0x7f3b4c15d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43756/0 (socket says 172.21.15.22:43756) 2024-03-29T04:18:46.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b51ac1640 1 -- 172.21.15.22:0/1364008263 learned_addr learned my addr 172.21.15.22:0/1364008263 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:46.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b522c2640 1 --2- 172.21.15.22:0/1364008263 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b4c06c670 0x7f3b4c1fb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:46.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b52ac3640 1 --2- 172.21.15.22:0/1364008263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b4c0b41f0 0x7f3b4c15f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:46.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b51ac1640 1 -- 172.21.15.22:0/1364008263 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b4c06c670 msgr2=0x7f3b4c1fb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:46.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b51ac1640 1 --2- 172.21.15.22:0/1364008263 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b4c06c670 0x7f3b4c1fb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b51ac1640 1 -- 172.21.15.22:0/1364008263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b4c0b41f0 msgr2=0x7f3b4c15f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:46.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b51ac1640 1 --2- 172.21.15.22:0/1364008263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b4c0b41f0 0x7f3b4c15f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:46.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b51ac1640 1 -- 172.21.15.22:0/1364008263 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c15c6d0 con 0x7f3b4c06ca50 2024-03-29T04:18:46.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b51ac1640 1 --2- 172.21.15.22:0/1364008263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4c06ca50 0x7f3b4c15d770 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00e870 tx=0x7f3b3c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:46.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b42ffd640 1 -- 172.21.15.22:0/1364008263 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c017070 con 0x7f3b4c06ca50 2024-03-29T04:18:46.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b42ffd640 1 -- 172.21.15.22:0/1364008263 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b3c00c940 con 0x7f3b4c06ca50 2024-03-29T04:18:46.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b54526640 1 -- 172.21.15.22:0/1364008263 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4c15c9b0 con 0x7f3b4c06ca50 2024-03-29T04:18:46.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b42ffd640 1 -- 172.21.15.22:0/1364008263 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c00ccf0 con 0x7f3b4c06ca50 2024-03-29T04:18:46.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.880+0000 7f3b54526640 1 -- 172.21.15.22:0/1364008263 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4c15cf70 con 0x7f3b4c06ca50 2024-03-29T04:18:46.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.884+0000 7f3b54526640 1 -- 172.21.15.22:0/1364008263 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b4c068890 con 0x7f3b4c06ca50 2024-03-29T04:18:46.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.884+0000 7f3b42ffd640 1 -- 172.21.15.22:0/1364008263 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f3b3c01a070 con 0x7f3b4c06ca50 2024-03-29T04:18:46.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.884+0000 7f3b42ffd640 1 --2- 172.21.15.22:0/1364008263 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b30067ff0 0x7f3b3006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:46.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.884+0000 7f3b42ffd640 1 -- 172.21.15.22:0/1364008263 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f3b3c086af0 con 0x7f3b4c06ca50 2024-03-29T04:18:46.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.884+0000 7f3b522c2640 1 --2- 172.21.15.22:0/1364008263 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b30067ff0 0x7f3b3006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:46.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.884+0000 7f3b522c2640 1 --2- 172.21.15.22:0/1364008263 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b30067ff0 0x7f3b3006a4b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3b480ad000 tx=0x7f3b48008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:46.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:46.888+0000 7f3b42ffd640 1 -- 172.21.15.22:0/1364008263 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f3b3c056f30 con 0x7f3b4c06ca50 2024-03-29T04:18:47.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.056+0000 7f3b54526640 1 -- 172.21.15.22:0/1364008263 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b4c0cf8c0 con 0x7f3b4c06ca50 2024-03-29T04:18:47.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.056+0000 7f3b42ffd640 1 -- 172.21.15.22:0/1364008263 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18847 (secure 0 0 0) 0x7f3b3c056830 con 0x7f3b4c06ca50 2024-03-29T04:18:47.038 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:47.038 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:18:16.893439+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:17:34.034142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:17:34.886659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:17:38.628632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:47.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.060+0000 7f3b54526640 1 -- 172.21.15.22:0/1364008263 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b30067ff0 msgr2=0x7f3b3006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:47.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.060+0000 7f3b54526640 1 --2- 172.21.15.22:0/1364008263 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b30067ff0 0x7f3b3006a4b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3b480ad000 tx=0x7f3b48008040 comp rx=0 tx=0).stop 2024-03-29T04:18:47.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.060+0000 7f3b54526640 1 -- 172.21.15.22:0/1364008263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4c06ca50 msgr2=0x7f3b4c15d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:47.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.060+0000 7f3b54526640 1 --2- 172.21.15.22:0/1364008263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4c06ca50 0x7f3b4c15d770 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00e870 tx=0x7f3b3c00ec70 comp rx=0 tx=0).stop 2024-03-29T04:18:47.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.060+0000 7f3b54526640 1 -- 172.21.15.22:0/1364008263 shutdown_connections 2024-03-29T04:18:47.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.060+0000 7f3b54526640 1 --2- 172.21.15.22:0/1364008263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b4c0b41f0 0x7f3b4c15f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:47.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.060+0000 7f3b54526640 1 --2- 172.21.15.22:0/1364008263 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b30067ff0 0x7f3b3006a4b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:47.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.060+0000 7f3b54526640 1 --2- 172.21.15.22:0/1364008263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4c06ca50 0x7f3b4c15d770 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:47.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.060+0000 7f3b54526640 1 --2- 172.21.15.22:0/1364008263 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b4c06c670 0x7f3b4c1fb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:47.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.060+0000 7f3b54526640 1 -- 172.21.15.22:0/1364008263 >> 172.21.15.22:0/1364008263 conn(0x7f3b4c0cfee0 msgr2=0x7f3b4c13dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:47.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.060+0000 7f3b54526640 1 -- 172.21.15.22:0/1364008263 shutdown_connections 2024-03-29T04:18:47.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.060+0000 7f3b54526640 1 -- 172.21.15.22:0/1364008263 wait complete. 2024-03-29T04:18:47.058 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T04:18:47.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb711c9640 1 Processor -- start 2024-03-29T04:18:47.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb711c9640 1 -- start start 2024-03-29T04:18:47.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb711c9640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fbb6c141e20 0x7fbb6c06f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:47.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb711c9640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb6c0d0d90 con 0x7fbb6c13f120 2024-03-29T04:18:47.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb711c9640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb6c0d04f0 con 0x7fbb6c1419c0 2024-03-29T04:18:47.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb711c9640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fbb6c0d07d0 con 0x7fbb6c141e20 2024-03-29T04:18:47.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb6ad9d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fbb6c13f120 0x7fbb6c0cf9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42648/0 (socket says 172.21.15.22:42648) 2024-03-29T04:18:47.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb6ad9d640 1 -- 172.21.15.22:0/417098863 learned_addr learned my addr 172.21.15.22:0/417098863 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:47.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb6b59e640 1 --2- 172.21.15.22:0/417098863 >> v2:172.21.15.151:3300/0 conn(0x7fbb6c141e20 0x7fbb6c06f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:47.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb69d9b640 1 -- 172.21.15.22:0/417098863 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2329726289 0 0) 0x7fbb6c0d0d90 con 0x7fbb6c13f120 2024-03-29T04:18:47.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb69d9b640 1 -- 172.21.15.22:0/417098863 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb58003530 con 0x7fbb6c13f120 2024-03-29T04:18:47.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb69d9b640 1 -- 172.21.15.22:0/417098863 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2441739679 0 0) 0x7fbb58003530 con 0x7fbb6c13f120 2024-03-29T04:18:47.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb69d9b640 1 -- 172.21.15.22:0/417098863 >> v2:172.21.15.151:3300/0 conn(0x7fbb6c141e20 msgr2=0x7fbb6c06f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:47.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb69d9b640 1 --2- 172.21.15.22:0/417098863 >> v2:172.21.15.151:3300/0 conn(0x7fbb6c141e20 0x7fbb6c06f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:47.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb69d9b640 1 -- 172.21.15.22:0/417098863 >> v1:172.21.15.151:6789/0 conn(0x7fbb6c1419c0 legacy=0x7fbb6c06ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:47.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb69d9b640 1 -- 172.21.15.22:0/417098863 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6c0d0ab0 con 0x7fbb6c13f120 2024-03-29T04:18:47.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb69d9b640 1 -- 172.21.15.22:0/417098863 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7fbb64001a00 con 0x7fbb6c13f120 2024-03-29T04:18:47.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb69d9b640 1 -- 172.21.15.22:0/417098863 >> v1:172.21.15.22:6789/0 conn(0x7fbb6c13f120 legacy=0x7fbb6c0cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:47.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb69d9b640 1 --2- 172.21.15.22:0/417098863 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb580039c0 0x7fbb58003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:47.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb69d9b640 1 --2- 172.21.15.22:0/417098863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb58004810 0x7fbb58004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:47.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb69d9b640 1 --2- 172.21.15.22:0/417098863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb580051d0 0x7fbb5800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:47.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb69d9b640 1 -- 172.21.15.22:0/417098863 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c0d07d0 con 0x7fbb58004810 2024-03-29T04:18:47.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb69d9b640 1 -- 172.21.15.22:0/417098863 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5800c070 con 0x7fbb580051d0 2024-03-29T04:18:47.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb69d9b640 1 -- 172.21.15.22:0/417098863 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fbb5800c270 con 0x7fbb580039c0 2024-03-29T04:18:47.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb6a59c640 1 --2- 172.21.15.22:0/417098863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb58004810 0x7fbb58004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:47.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb6ad9d640 1 --2- 172.21.15.22:0/417098863 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb580039c0 0x7fbb58003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:47.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb6b59e640 1 --2- 172.21.15.22:0/417098863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb580051d0 0x7fbb5800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:47.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb6a59c640 1 -- 172.21.15.22:0/417098863 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb580039c0 msgr2=0x7fbb58003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:47.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb6a59c640 1 --2- 172.21.15.22:0/417098863 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb580039c0 0x7fbb58003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:47.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb6a59c640 1 -- 172.21.15.22:0/417098863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb580051d0 msgr2=0x7fbb5800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:47.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb6a59c640 1 --2- 172.21.15.22:0/417098863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb580051d0 0x7fbb5800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:47.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb6a59c640 1 -- 172.21.15.22:0/417098863 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb58003530 con 0x7fbb58004810 2024-03-29T04:18:47.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb6b59e640 1 --2- 172.21.15.22:0/417098863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb580051d0 0x7fbb5800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:47.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.200+0000 7fbb6b59e640 1 -- 172.21.15.22:0/417098863 reap_dead start 2024-03-29T04:18:47.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.204+0000 7fbb6a59c640 1 --2- 172.21.15.22:0/417098863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb58004810 0x7fbb58004c90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbb54001780 tx=0x7fbb5401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d6ff8c20eddbd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:47.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.204+0000 7fbb69d9b640 1 -- 172.21.15.22:0/417098863 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb540013a0 con 0x7fbb58004810 2024-03-29T04:18:47.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.204+0000 7fbb69d9b640 1 -- 172.21.15.22:0/417098863 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb540052f0 con 0x7fbb58004810 2024-03-29T04:18:47.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.204+0000 7fbb69d9b640 1 -- 172.21.15.22:0/417098863 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb54003840 con 0x7fbb58004810 2024-03-29T04:18:47.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.204+0000 7fbb711c9640 1 -- 172.21.15.22:0/417098863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb58004810 msgr2=0x7fbb58004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:47.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.204+0000 7fbb711c9640 1 --2- 172.21.15.22:0/417098863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb58004810 0x7fbb58004c90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbb54001780 tx=0x7fbb5401c400 comp rx=0 tx=0).stop 2024-03-29T04:18:47.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.204+0000 7fbb711c9640 1 -- 172.21.15.22:0/417098863 shutdown_connections 2024-03-29T04:18:47.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.204+0000 7fbb711c9640 1 --2- 172.21.15.22:0/417098863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb58004810 0x7fbb58004c90 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:47.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.204+0000 7fbb711c9640 1 -- 172.21.15.22:0/417098863 >> 172.21.15.22:0/417098863 conn(0x7fbb6c0b2c70 msgr2=0x7fbb6c0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:47.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.204+0000 7fbb711c9640 1 -- 172.21.15.22:0/417098863 shutdown_connections 2024-03-29T04:18:47.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.204+0000 7fbb711c9640 1 -- 172.21.15.22:0/417098863 wait complete. 2024-03-29T04:18:47.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.204+0000 7fbb711c9640 1 Processor -- start 2024-03-29T04:18:47.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.204+0000 7fbb711c9640 1 -- start start 2024-03-29T04:18:47.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb711c9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb58004810 0x7fbb6c13a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:47.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb711c9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb6c155360 0x7fbb6c143a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:47.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb711c9640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb6c143f90 0x7fbb6c15b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:47.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb711c9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c13f680 con 0x7fbb6c155360 2024-03-29T04:18:47.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb711c9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c13f500 con 0x7fbb58004810 2024-03-29T04:18:47.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb711c9640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fbb6c1426d0 con 0x7fbb6c143f90 2024-03-29T04:18:47.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb6a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb6c155360 0x7fbb6c143a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:47.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb6a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb6c155360 0x7fbb6c143a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43778/0 (socket says 172.21.15.22:43778) 2024-03-29T04:18:47.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb6a59c640 1 -- 172.21.15.22:0/372291434 learned_addr learned my addr 172.21.15.22:0/372291434 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:47.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb6ad9d640 1 --2- 172.21.15.22:0/372291434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb58004810 0x7fbb6c13a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:47.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb6b59e640 1 --2- 172.21.15.22:0/372291434 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb6c143f90 0x7fbb6c15b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:47.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb6a59c640 1 -- 172.21.15.22:0/372291434 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb6c143f90 msgr2=0x7fbb6c15b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:47.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb6a59c640 1 --2- 172.21.15.22:0/372291434 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb6c143f90 0x7fbb6c15b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:47.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb6a59c640 1 -- 172.21.15.22:0/372291434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb58004810 msgr2=0x7fbb6c13a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:47.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb6a59c640 1 --2- 172.21.15.22:0/372291434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb58004810 0x7fbb6c13a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:47.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb6a59c640 1 -- 172.21.15.22:0/372291434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6c0b4280 con 0x7fbb6c155360 2024-03-29T04:18:47.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb6a59c640 1 --2- 172.21.15.22:0/372291434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb6c155360 0x7fbb6c143a50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbb54005040 tx=0x7fbb58005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:47.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb637fe640 1 -- 172.21.15.22:0/372291434 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb540013a0 con 0x7fbb6c155360 2024-03-29T04:18:47.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb637fe640 1 -- 172.21.15.22:0/372291434 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb540ba070 con 0x7fbb6c155360 2024-03-29T04:18:47.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb711c9640 1 -- 172.21.15.22:0/372291434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb6c0b4560 con 0x7fbb6c155360 2024-03-29T04:18:47.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb637fe640 1 -- 172.21.15.22:0/372291434 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb54007a40 con 0x7fbb6c155360 2024-03-29T04:18:47.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb711c9640 1 -- 172.21.15.22:0/372291434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb6c0b4b20 con 0x7fbb6c155360 2024-03-29T04:18:47.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb711c9640 1 -- 172.21.15.22:0/372291434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb6c068890 con 0x7fbb6c155360 2024-03-29T04:18:47.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb637fe640 1 -- 172.21.15.22:0/372291434 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fbb54007450 con 0x7fbb6c155360 2024-03-29T04:18:47.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb637fe640 1 --2- 172.21.15.22:0/372291434 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbb4c067fc0 0x7fbb4c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:47.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb637fe640 1 -- 172.21.15.22:0/372291434 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fbb54121a30 con 0x7fbb6c155360 2024-03-29T04:18:47.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.208+0000 7fbb6ad9d640 1 --2- 172.21.15.22:0/372291434 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbb4c067fc0 0x7fbb4c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:47.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.212+0000 7fbb6ad9d640 1 --2- 172.21.15.22:0/372291434 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbb4c067fc0 0x7fbb4c06a480 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbb64001990 tx=0x7fbb64001920 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:47.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.212+0000 7fbb637fe640 1 -- 172.21.15.22:0/372291434 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fbb54016110 con 0x7fbb6c155360 2024-03-29T04:18:47.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.408+0000 7fbb711c9640 1 -- 172.21.15.22:0/372291434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbb6c06ba30 con 0x7fbb6c155360 2024-03-29T04:18:47.389 INFO:tasks.ceph.mon.a.smithi022.stderr:2024-03-29T04:18:47.408+0000 7fd3d21cd640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T04:18:47.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.412+0000 7fbb637fe640 1 -- 172.21.15.22:0/372291434 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7fbb540f1d30 con 0x7fbb6c155360 2024-03-29T04:18:47.392 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T04:18:47.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.412+0000 7fbb711c9640 1 -- 172.21.15.22:0/372291434 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbb4c067fc0 msgr2=0x7fbb4c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:47.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.412+0000 7fbb711c9640 1 --2- 172.21.15.22:0/372291434 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbb4c067fc0 0x7fbb4c06a480 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbb64001990 tx=0x7fbb64001920 comp rx=0 tx=0).stop 2024-03-29T04:18:47.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.412+0000 7fbb711c9640 1 -- 172.21.15.22:0/372291434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb6c155360 msgr2=0x7fbb6c143a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:47.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.412+0000 7fbb711c9640 1 --2- 172.21.15.22:0/372291434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb6c155360 0x7fbb6c143a50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbb54005040 tx=0x7fbb58005840 comp rx=0 tx=0).stop 2024-03-29T04:18:47.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.412+0000 7fbb711c9640 1 -- 172.21.15.22:0/372291434 shutdown_connections 2024-03-29T04:18:47.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.412+0000 7fbb711c9640 1 --2- 172.21.15.22:0/372291434 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbb6c143f90 0x7fbb6c15b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:47.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.412+0000 7fbb711c9640 1 --2- 172.21.15.22:0/372291434 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbb4c067fc0 0x7fbb4c06a480 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:47.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.412+0000 7fbb711c9640 1 --2- 172.21.15.22:0/372291434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb6c155360 0x7fbb6c143a50 secure :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbb54005040 tx=0x7fbb58005840 comp rx=0 tx=0).stop 2024-03-29T04:18:47.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.412+0000 7fbb711c9640 1 --2- 172.21.15.22:0/372291434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb58004810 0x7fbb6c13a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:47.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.412+0000 7fbb711c9640 1 -- 172.21.15.22:0/372291434 >> 172.21.15.22:0/372291434 conn(0x7fbb6c0b2c70 msgr2=0x7fbb6c072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:47.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.416+0000 7fbb711c9640 1 -- 172.21.15.22:0/372291434 shutdown_connections 2024-03-29T04:18:47.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.416+0000 7fbb711c9640 1 -- 172.21.15.22:0/372291434 wait complete. 2024-03-29T04:18:47.407 INFO:tasks.ceph.mds.b.smithi151.stderr: -4968> 2024-03-29T04:17:39.507+0000 7f68f6146640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T04:18:47.412 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-29T04:18:47.474 INFO:tasks.ceph.mds.b.smithi151.stdout:starting mds.b at 2024-03-29T04:18:47.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.548+0000 7f6bf574a640 1 Processor -- start 2024-03-29T04:18:47.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.548+0000 7f6bf574a640 1 -- start start 2024-03-29T04:18:47.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.548+0000 7f6bf574a640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f6bf013f140 0x7f6bf01606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:47.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.548+0000 7f6bf574a640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6bf015ff40 0x7f6bf006c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:47.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.548+0000 7f6bf574a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6bf006cbb0 0x7f6bf0141a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:47.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.548+0000 7f6bf574a640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f6bf006d000 con 0x7f6bf006cbb0 2024-03-29T04:18:47.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.548+0000 7f6bf574a640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f6bf006e320 con 0x7f6bf013f140 2024-03-29T04:18:47.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.548+0000 7f6bf574a640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f6bf006d710 con 0x7f6bf015ff40 2024-03-29T04:18:47.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.548+0000 7f6bef7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6bf006cbb0 0x7f6bf0141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:47.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6bef7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6bf006cbb0 0x7f6bf0141a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43780/0 (socket says 172.21.15.22:43780) 2024-03-29T04:18:47.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6bef7fe640 1 -- 172.21.15.22:0/208510587 learned_addr learned my addr 172.21.15.22:0/208510587 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:47.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6beeffd640 1 --2- 172.21.15.22:0/208510587 >> v2:172.21.15.151:3300/0 conn(0x7f6bf013f140 0x7f6bf01606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:47.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6be67fc640 1 --2- 172.21.15.22:0/208510587 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6bf015ff40 0x7f6bf006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:47.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6bef7fe640 1 -- 172.21.15.22:0/208510587 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6bf015ff40 msgr2=0x7f6bf006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:47.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6bef7fe640 1 --2- 172.21.15.22:0/208510587 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6bf015ff40 0x7f6bf006c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:47.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6bef7fe640 1 -- 172.21.15.22:0/208510587 >> v2:172.21.15.151:3300/0 conn(0x7f6bf013f140 msgr2=0x7f6bf01606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:47.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6bef7fe640 1 --2- 172.21.15.22:0/208510587 >> v2:172.21.15.151:3300/0 conn(0x7f6bf013f140 0x7f6bf01606d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:47.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6bef7fe640 1 -- 172.21.15.22:0/208510587 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf006d9f0 con 0x7f6bf006cbb0 2024-03-29T04:18:47.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6bef7fe640 1 --2- 172.21.15.22:0/208510587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bf006cbb0 0x7f6bf0141a30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00cfe30 tx=0x7f6be801c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed8937c7e162870 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:47.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6bee7fc640 1 -- 172.21.15.22:0/208510587 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be80a5580 con 0x7f6bf006cbb0 2024-03-29T04:18:47.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6bee7fc640 1 -- 172.21.15.22:0/208510587 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6be80a5720 con 0x7f6bf006cbb0 2024-03-29T04:18:47.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6bee7fc640 1 -- 172.21.15.22:0/208510587 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be80a5af0 con 0x7f6bf006cbb0 2024-03-29T04:18:47.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6bf574a640 1 -- 172.21.15.22:0/208510587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bf006cbb0 msgr2=0x7f6bf0141a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:47.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6bf574a640 1 --2- 172.21.15.22:0/208510587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bf006cbb0 0x7f6bf0141a30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00cfe30 tx=0x7f6be801c700 comp rx=0 tx=0).stop 2024-03-29T04:18:47.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6bf574a640 1 -- 172.21.15.22:0/208510587 shutdown_connections 2024-03-29T04:18:47.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6bf574a640 1 --2- 172.21.15.22:0/208510587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bf006cbb0 0x7f6bf0141a30 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:47.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6bf574a640 1 --2- 172.21.15.22:0/208510587 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6bf015ff40 0x7f6bf006c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:47.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6bf574a640 1 --2- 172.21.15.22:0/208510587 >> v2:172.21.15.151:3300/0 conn(0x7f6bf013f140 0x7f6bf01606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:47.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6bf574a640 1 -- 172.21.15.22:0/208510587 >> 172.21.15.22:0/208510587 conn(0x7f6bf00d00a0 msgr2=0x7f6bf00d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:47.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6bf574a640 1 -- 172.21.15.22:0/208510587 shutdown_connections 2024-03-29T04:18:47.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.552+0000 7f6bf574a640 1 -- 172.21.15.22:0/208510587 wait complete. 2024-03-29T04:18:47.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6bf574a640 1 Processor -- start 2024-03-29T04:18:47.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6bf574a640 1 -- start start 2024-03-29T04:18:47.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6bf574a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6bf006cbb0 0x7f6bf0141350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:47.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6bf574a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bf013f140 0x7f6bf01388e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:47.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6bf574a640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6bf015ff40 0x7f6bf015b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:47.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6bf574a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf0160d90 con 0x7f6bf013f140 2024-03-29T04:18:47.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6bf574a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf0160c10 con 0x7f6bf006cbb0 2024-03-29T04:18:47.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6bf574a640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f6bf00ce940 con 0x7f6bf015ff40 2024-03-29T04:18:47.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6be67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bf013f140 0x7f6bf01388e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:47.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6be67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bf013f140 0x7f6bf01388e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43786/0 (socket says 172.21.15.22:43786) 2024-03-29T04:18:47.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6be67fc640 1 -- 172.21.15.22:0/3110289312 learned_addr learned my addr 172.21.15.22:0/3110289312 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:47.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6beeffd640 1 --2- 172.21.15.22:0/3110289312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6bf006cbb0 0x7f6bf0141350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:47.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6bef7fe640 1 --2- 172.21.15.22:0/3110289312 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6bf015ff40 0x7f6bf015b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:47.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6be67fc640 1 -- 172.21.15.22:0/3110289312 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6bf015ff40 msgr2=0x7f6bf015b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:47.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6be67fc640 1 --2- 172.21.15.22:0/3110289312 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6bf015ff40 0x7f6bf015b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:47.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6be67fc640 1 -- 172.21.15.22:0/3110289312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6bf006cbb0 msgr2=0x7f6bf0141350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:47.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6be67fc640 1 --2- 172.21.15.22:0/3110289312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6bf006cbb0 0x7f6bf0141350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:47.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6be67fc640 1 -- 172.21.15.22:0/3110289312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf00b3ea0 con 0x7f6bf013f140 2024-03-29T04:18:47.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6be67fc640 1 --2- 172.21.15.22:0/3110289312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bf013f140 0x7f6bf01388e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800d5f0 tx=0x7f6bd800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:47.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6be7fff640 1 -- 172.21.15.22:0/3110289312 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd80125f0 con 0x7f6bf013f140 2024-03-29T04:18:47.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6be7fff640 1 -- 172.21.15.22:0/3110289312 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bd8019070 con 0x7f6bf013f140 2024-03-29T04:18:47.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6bf574a640 1 -- 172.21.15.22:0/3110289312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf00b4140 con 0x7f6bf013f140 2024-03-29T04:18:47.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6be7fff640 1 -- 172.21.15.22:0/3110289312 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd80129d0 con 0x7f6bf013f140 2024-03-29T04:18:47.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6bf574a640 1 -- 172.21.15.22:0/3110289312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf013a3e0 con 0x7f6bf013f140 2024-03-29T04:18:47.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6bf574a640 1 -- 172.21.15.22:0/3110289312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bf00b4660 con 0x7f6bf013f140 2024-03-29T04:18:47.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6be7fff640 1 -- 172.21.15.22:0/3110289312 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f6bd8012b70 con 0x7f6bf013f140 2024-03-29T04:18:47.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.556+0000 7f6be7fff640 1 --2- 172.21.15.22:0/3110289312 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6bd0067fc0 0x7f6bd006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:47.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.560+0000 7f6be7fff640 1 -- 172.21.15.22:0/3110289312 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6821+0+0 (secure 0 0 0) 0x7f6bd8086cb0 con 0x7f6bf013f140 2024-03-29T04:18:47.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.560+0000 7f6beeffd640 1 --2- 172.21.15.22:0/3110289312 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6bd0067fc0 0x7f6bd006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:47.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.560+0000 7f6beeffd640 1 --2- 172.21.15.22:0/3110289312 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6bd0067fc0 0x7f6bd006a480 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6be0009ae0 tx=0x7f6be0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:47.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.564+0000 7f6be7fff640 1 -- 172.21.15.22:0/3110289312 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f6bf00b4660 con 0x7f6bf013f140 2024-03-29T04:18:47.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:47.760+0000 7f6bf574a640 1 -- 172.21.15.22:0/3110289312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6bf0068900 con 0x7f6bf013f140 2024-03-29T04:18:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.420+0000 7f6be7fff640 1 -- 172.21.15.22:0/3110289312 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f6bd8057090 con 0x7f6bf013f140 2024-03-29T04:18:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.424+0000 7f6bf574a640 1 -- 172.21.15.22:0/3110289312 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6bd0067fc0 msgr2=0x7f6bd006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.424+0000 7f6bf574a640 1 --2- 172.21.15.22:0/3110289312 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6bd0067fc0 0x7f6bd006a480 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6be0009ae0 tx=0x7f6be0009190 comp rx=0 tx=0).stop 2024-03-29T04:18:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.424+0000 7f6bf574a640 1 -- 172.21.15.22:0/3110289312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bf013f140 msgr2=0x7f6bf01388e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.424+0000 7f6bf574a640 1 --2- 172.21.15.22:0/3110289312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bf013f140 0x7f6bf01388e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800d5f0 tx=0x7f6bd800dab0 comp rx=0 tx=0).stop 2024-03-29T04:18:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.424+0000 7f6bf574a640 1 -- 172.21.15.22:0/3110289312 shutdown_connections 2024-03-29T04:18:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.424+0000 7f6bf574a640 1 --2- 172.21.15.22:0/3110289312 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6bf015ff40 0x7f6bf015b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.424+0000 7f6bf574a640 1 --2- 172.21.15.22:0/3110289312 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6bd0067fc0 0x7f6bd006a480 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.424+0000 7f6bf574a640 1 --2- 172.21.15.22:0/3110289312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bf013f140 0x7f6bf01388e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.424+0000 7f6bf574a640 1 --2- 172.21.15.22:0/3110289312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6bf006cbb0 0x7f6bf0141350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.424+0000 7f6bf574a640 1 -- 172.21.15.22:0/3110289312 >> 172.21.15.22:0/3110289312 conn(0x7f6bf00d00a0 msgr2=0x7f6bf00d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:48.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.424+0000 7f6bf574a640 1 -- 172.21.15.22:0/3110289312 shutdown_connections 2024-03-29T04:18:48.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.424+0000 7f6bf574a640 1 -- 172.21.15.22:0/3110289312 wait complete. 2024-03-29T04:18:48.427 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-29T04:18:48.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.580+0000 7fd72ba53640 1 Processor -- start 2024-03-29T04:18:48.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.580+0000 7fd72ba53640 1 -- start start 2024-03-29T04:18:48.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd72ba53640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd7241449a0 0x7fd72415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:48.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd72ba53640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd72415f220 0x7fd7240ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:48.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd72ba53640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fd7240ce860 0x7fd724072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:48.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd72ba53640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fd72415fc80 con 0x7fd7241449a0 2024-03-29T04:18:48.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd72ba53640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fd724144d80 con 0x7fd7240ce860 2024-03-29T04:18:48.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd72ba53640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd72415f6c0 con 0x7fd72415f220 2024-03-29T04:18:48.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd7297ef640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd7241449a0 0x7fd72415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:48.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd7297ef640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd7241449a0 0x7fd72415ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43794/0 (socket says 172.21.15.22:43794) 2024-03-29T04:18:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd7297ef640 1 -- 172.21.15.22:0/2830344841 learned_addr learned my addr 172.21.15.22:0/2830344841 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd729ff0640 1 --2- 172.21.15.22:0/2830344841 >> v2:172.21.15.151:3300/0 conn(0x7fd7240ce860 0x7fd724072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd728fee640 1 --2- 172.21.15.22:0/2830344841 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd72415f220 0x7fd7240ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd7297ef640 1 -- 172.21.15.22:0/2830344841 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd72415f220 msgr2=0x7fd7240ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd7297ef640 1 --2- 172.21.15.22:0/2830344841 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd72415f220 0x7fd7240ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd7297ef640 1 -- 172.21.15.22:0/2830344841 >> v2:172.21.15.151:3300/0 conn(0x7fd7240ce860 msgr2=0x7fd724072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd7297ef640 1 --2- 172.21.15.22:0/2830344841 >> v2:172.21.15.151:3300/0 conn(0x7fd7240ce860 0x7fd724072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd7297ef640 1 -- 172.21.15.22:0/2830344841 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72415f9a0 con 0x7fd7241449a0 2024-03-29T04:18:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd729ff0640 1 --2- 172.21.15.22:0/2830344841 >> v2:172.21.15.151:3300/0 conn(0x7fd7240ce860 0x7fd724072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:48.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd728fee640 1 --2- 172.21.15.22:0/2830344841 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd72415f220 0x7fd7240ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:48.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd7297ef640 1 --2- 172.21.15.22:0/2830344841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7241449a0 0x7fd72415ece0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd7240cf5f0 tx=0x7fd72001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52753e7054d8f933 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:48.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd71bfff640 1 -- 172.21.15.22:0/2830344841 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7200b0070 con 0x7fd7241449a0 2024-03-29T04:18:48.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd71bfff640 1 -- 172.21.15.22:0/2830344841 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7200ab600 con 0x7fd7241449a0 2024-03-29T04:18:48.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd71bfff640 1 -- 172.21.15.22:0/2830344841 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7200ab9d0 con 0x7fd7241449a0 2024-03-29T04:18:48.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd72ba53640 1 -- 172.21.15.22:0/2830344841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7241449a0 msgr2=0x7fd72415ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:48.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.584+0000 7fd72ba53640 1 --2- 172.21.15.22:0/2830344841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7241449a0 0x7fd72415ece0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd7240cf5f0 tx=0x7fd72001c3d0 comp rx=0 tx=0).stop 2024-03-29T04:18:48.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd72ba53640 1 -- 172.21.15.22:0/2830344841 shutdown_connections 2024-03-29T04:18:48.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd72ba53640 1 --2- 172.21.15.22:0/2830344841 >> v2:172.21.15.151:3300/0 conn(0x7fd7240ce860 0x7fd724072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:48.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd72ba53640 1 --2- 172.21.15.22:0/2830344841 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd72415f220 0x7fd7240ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:48.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd72ba53640 1 --2- 172.21.15.22:0/2830344841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7241449a0 0x7fd72415ece0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:48.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd72ba53640 1 -- 172.21.15.22:0/2830344841 >> 172.21.15.22:0/2830344841 conn(0x7fd7240b1b80 msgr2=0x7fd7240b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:48.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd72ba53640 1 -- 172.21.15.22:0/2830344841 shutdown_connections 2024-03-29T04:18:48.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd72ba53640 1 -- 172.21.15.22:0/2830344841 wait complete. 2024-03-29T04:18:48.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd72ba53640 1 Processor -- start 2024-03-29T04:18:48.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd72ba53640 1 -- start start 2024-03-29T04:18:48.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd72ba53640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd7240ce860 0x7fd72415dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:48.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd72ba53640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7241449a0 0x7fd7241436d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:48.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd72ba53640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd72415f220 0x7fd72415c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:48.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd72ba53640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd7240ca5f0 con 0x7fd7241449a0 2024-03-29T04:18:48.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd72ba53640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd7240ca470 con 0x7fd72415f220 2024-03-29T04:18:48.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd72ba53640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fd7240ca770 con 0x7fd7240ce860 2024-03-29T04:18:48.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd728fee640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7241449a0 0x7fd7241436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:48.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd728fee640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7241449a0 0x7fd7241436d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43810/0 (socket says 172.21.15.22:43810) 2024-03-29T04:18:48.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd728fee640 1 -- 172.21.15.22:0/1993682127 learned_addr learned my addr 172.21.15.22:0/1993682127 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:48.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd7297ef640 1 --2- 172.21.15.22:0/1993682127 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd7240ce860 0x7fd72415dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:48.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd729ff0640 1 --2- 172.21.15.22:0/1993682127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd72415f220 0x7fd72415c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:48.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd728fee640 1 -- 172.21.15.22:0/1993682127 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd7240ce860 msgr2=0x7fd72415dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:48.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd728fee640 1 --2- 172.21.15.22:0/1993682127 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd7240ce860 0x7fd72415dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:48.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd728fee640 1 -- 172.21.15.22:0/1993682127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd72415f220 msgr2=0x7fd72415c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:48.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd728fee640 1 --2- 172.21.15.22:0/1993682127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd72415f220 0x7fd72415c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:48.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd728fee640 1 -- 172.21.15.22:0/1993682127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72413a6e0 con 0x7fd7241449a0 2024-03-29T04:18:48.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.588+0000 7fd7297ef640 1 --2- 172.21.15.22:0/1993682127 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd7240ce860 0x7fd72415dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:48.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.592+0000 7fd728fee640 1 --2- 172.21.15.22:0/1993682127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7241449a0 0x7fd7241436d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd71400c7f0 tx=0x7fd71400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:48.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.592+0000 7fd719ffb640 1 -- 172.21.15.22:0/1993682127 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714015070 con 0x7fd7241449a0 2024-03-29T04:18:48.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.592+0000 7fd719ffb640 1 -- 172.21.15.22:0/1993682127 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd714007820 con 0x7fd7241449a0 2024-03-29T04:18:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.592+0000 7fd72ba53640 1 -- 172.21.15.22:0/1993682127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72415e4e0 con 0x7fd7241449a0 2024-03-29T04:18:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.592+0000 7fd719ffb640 1 -- 172.21.15.22:0/1993682127 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714007bd0 con 0x7fd7241449a0 2024-03-29T04:18:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.592+0000 7fd72ba53640 1 -- 172.21.15.22:0/1993682127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd724142ed0 con 0x7fd7241449a0 2024-03-29T04:18:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.592+0000 7fd72ba53640 1 -- 172.21.15.22:0/1993682127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7240687f0 con 0x7fd7241449a0 2024-03-29T04:18:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.592+0000 7fd719ffb640 1 -- 172.21.15.22:0/1993682127 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fd714018070 con 0x7fd7241449a0 2024-03-29T04:18:48.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.592+0000 7fd719ffb640 1 --2- 172.21.15.22:0/1993682127 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd708067fc0 0x7fd70806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:48.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.592+0000 7fd719ffb640 1 -- 172.21.15.22:0/1993682127 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6821+0+0 (secure 0 0 0) 0x7fd714086d40 con 0x7fd7241449a0 2024-03-29T04:18:48.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.592+0000 7fd7297ef640 1 --2- 172.21.15.22:0/1993682127 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd708067fc0 0x7fd70806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:48.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.592+0000 7fd7297ef640 1 --2- 172.21.15.22:0/1993682127 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd708067fc0 0x7fd70806a480 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd7200ad000 tx=0x7fd720008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:48.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.596+0000 7fd719ffb640 1 -- 172.21.15.22:0/1993682127 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fd714057200 con 0x7fd7241449a0 2024-03-29T04:18:48.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:48.764+0000 7fd72ba53640 1 -- 172.21.15.22:0/1993682127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd7241408f0 con 0x7fd7241449a0 2024-03-29T04:18:49.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.420+0000 7fd719ffb640 1 -- 172.21.15.22:0/1993682127 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) ==== 181+0+0 (secure 0 0 0) 0x7fd714056b00 con 0x7fd7241449a0 2024-03-29T04:18:49.403 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' removed 2024-03-29T04:18:49.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.424+0000 7fd72ba53640 1 -- 172.21.15.22:0/1993682127 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd708067fc0 msgr2=0x7fd70806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:49.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.424+0000 7fd72ba53640 1 --2- 172.21.15.22:0/1993682127 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd708067fc0 0x7fd70806a480 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd7200ad000 tx=0x7fd720008040 comp rx=0 tx=0).stop 2024-03-29T04:18:49.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.424+0000 7fd72ba53640 1 -- 172.21.15.22:0/1993682127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7241449a0 msgr2=0x7fd7241436d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:49.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.424+0000 7fd72ba53640 1 --2- 172.21.15.22:0/1993682127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7241449a0 0x7fd7241436d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd71400c7f0 tx=0x7fd71400ccb0 comp rx=0 tx=0).stop 2024-03-29T04:18:49.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.424+0000 7fd72ba53640 1 -- 172.21.15.22:0/1993682127 shutdown_connections 2024-03-29T04:18:49.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.424+0000 7fd72ba53640 1 --2- 172.21.15.22:0/1993682127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd72415f220 0x7fd72415c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:49.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.424+0000 7fd72ba53640 1 --2- 172.21.15.22:0/1993682127 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fd708067fc0 0x7fd70806a480 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:49.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.424+0000 7fd72ba53640 1 --2- 172.21.15.22:0/1993682127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7241449a0 0x7fd7241436d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:49.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.424+0000 7fd72ba53640 1 --2- 172.21.15.22:0/1993682127 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fd7240ce860 0x7fd72415dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:49.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.424+0000 7fd72ba53640 1 -- 172.21.15.22:0/1993682127 >> 172.21.15.22:0/1993682127 conn(0x7fd7240b1b80 msgr2=0x7fd7240c9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:49.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.428+0000 7fd72ba53640 1 -- 172.21.15.22:0/1993682127 shutdown_connections 2024-03-29T04:18:49.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.428+0000 7fd72ba53640 1 -- 172.21.15.22:0/1993682127 wait complete. 2024-03-29T04:18:49.436 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-29T04:18:49.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.580+0000 7f03108d5640 1 Processor -- start 2024-03-29T04:18:49.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.580+0000 7f03108d5640 1 -- start start 2024-03-29T04:18:49.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f03108d5640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f030c0b5550 0x7f030c160610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:49.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f03108d5640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f030c0b5930 0x7f030c15c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:49.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f03108d5640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f030c15d090 con 0x7f030c073490 2024-03-29T04:18:49.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f03108d5640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f030c15d370 con 0x7f030c0b5930 2024-03-29T04:18:49.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f03108d5640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f030c15fe10 con 0x7f030c0b5550 2024-03-29T04:18:49.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030ad9d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f030c073490 0x7f030c06c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42650/0 (socket says 172.21.15.22:42650) 2024-03-29T04:18:49.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030ad9d640 1 -- 172.21.15.22:0/4060051719 learned_addr learned my addr 172.21.15.22:0/4060051719 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:49.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f0309d9b640 1 --2- 172.21.15.22:0/4060051719 >> v2:172.21.15.151:3300/0 conn(0x7f030c0b5930 0x7f030c15c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:49.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030959a640 1 -- 172.21.15.22:0/4060051719 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1379503986 0 0) 0x7f030c15d090 con 0x7f030c073490 2024-03-29T04:18:49.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030a59c640 1 --2- 172.21.15.22:0/4060051719 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f030c0b5550 0x7f030c160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:49.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030959a640 1 -- 172.21.15.22:0/4060051719 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02f8003530 con 0x7f030c073490 2024-03-29T04:18:49.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030959a640 1 -- 172.21.15.22:0/4060051719 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 973629646 0 0) 0x7f02f8003530 con 0x7f030c073490 2024-03-29T04:18:49.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030959a640 1 -- 172.21.15.22:0/4060051719 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f030c0b5550 msgr2=0x7f030c160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:49.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030959a640 1 --2- 172.21.15.22:0/4060051719 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f030c0b5550 0x7f030c160610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:49.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030959a640 1 -- 172.21.15.22:0/4060051719 >> v2:172.21.15.151:3300/0 conn(0x7f030c0b5930 msgr2=0x7f030c15c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:49.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030959a640 1 --2- 172.21.15.22:0/4060051719 >> v2:172.21.15.151:3300/0 conn(0x7f030c0b5930 0x7f030c15c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:49.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030959a640 1 -- 172.21.15.22:0/4060051719 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030c1600f0 con 0x7f030c073490 2024-03-29T04:18:49.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030959a640 1 -- 172.21.15.22:0/4060051719 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f0304001a00 con 0x7f030c073490 2024-03-29T04:18:49.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030959a640 1 -- 172.21.15.22:0/4060051719 >> v1:172.21.15.22:6789/0 conn(0x7f030c073490 legacy=0x7f030c06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:49.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030959a640 1 --2- 172.21.15.22:0/4060051719 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f030c0b5550 0x7f02f80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:49.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030959a640 1 --2- 172.21.15.22:0/4060051719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02f8003f40 0x7f02f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:49.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030959a640 1 --2- 172.21.15.22:0/4060051719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02f8004d50 0x7f02f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:49.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030959a640 1 -- 172.21.15.22:0/4060051719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f030c15d370 con 0x7f02f8003f40 2024-03-29T04:18:49.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030959a640 1 -- 172.21.15.22:0/4060051719 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f030c15fe10 con 0x7f02f8004d50 2024-03-29T04:18:49.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030959a640 1 -- 172.21.15.22:0/4060051719 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f02f800b9d0 con 0x7f030c0b5550 2024-03-29T04:18:49.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f0309d9b640 1 --2- 172.21.15.22:0/4060051719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02f8003f40 0x7f02f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:49.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030a59c640 1 --2- 172.21.15.22:0/4060051719 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f030c0b5550 0x7f02f80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:49.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030ad9d640 1 --2- 172.21.15.22:0/4060051719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02f8004d50 0x7f02f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:49.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f0309d9b640 1 -- 172.21.15.22:0/4060051719 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f030c0b5550 msgr2=0x7f02f80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:49.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f0309d9b640 1 --2- 172.21.15.22:0/4060051719 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f030c0b5550 0x7f02f80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:49.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f0309d9b640 1 -- 172.21.15.22:0/4060051719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02f8004d50 msgr2=0x7f02f80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:49.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f0309d9b640 1 --2- 172.21.15.22:0/4060051719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02f8004d50 0x7f02f80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:49.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f0309d9b640 1 -- 172.21.15.22:0/4060051719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f8003530 con 0x7f02f8003f40 2024-03-29T04:18:49.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.584+0000 7f030ad9d640 1 --2- 172.21.15.22:0/4060051719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02f8004d50 0x7f02f80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:49.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.588+0000 7f0309d9b640 1 --2- 172.21.15.22:0/4060051719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02f8003f40 0x7f02f8004810 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009ab0 tx=0x7f02f401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a60512cc9628f40 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:49.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.588+0000 7f030959a640 1 -- 172.21.15.22:0/4060051719 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f40b2070 con 0x7f02f8003f40 2024-03-29T04:18:49.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.588+0000 7f030959a640 1 -- 172.21.15.22:0/4060051719 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02f40a5680 con 0x7f02f8003f40 2024-03-29T04:18:49.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.588+0000 7f030959a640 1 -- 172.21.15.22:0/4060051719 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f40096b0 con 0x7f02f8003f40 2024-03-29T04:18:49.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.588+0000 7f03108d5640 1 -- 172.21.15.22:0/4060051719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02f8003f40 msgr2=0x7f02f8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:49.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.588+0000 7f03108d5640 1 --2- 172.21.15.22:0/4060051719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02f8003f40 0x7f02f8004810 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009ab0 tx=0x7f02f401ca30 comp rx=0 tx=0).stop 2024-03-29T04:18:49.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.588+0000 7f030ad9d640 1 -- 172.21.15.22:0/4060051719 reap_dead start 2024-03-29T04:18:49.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.588+0000 7f03108d5640 1 -- 172.21.15.22:0/4060051719 shutdown_connections 2024-03-29T04:18:49.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.588+0000 7f03108d5640 1 -- 172.21.15.22:0/4060051719 >> 172.21.15.22:0/4060051719 conn(0x7f030c0b1b80 msgr2=0x7f030c0b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:49.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.588+0000 7f03108d5640 1 -- 172.21.15.22:0/4060051719 shutdown_connections 2024-03-29T04:18:49.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.588+0000 7f03108d5640 1 -- 172.21.15.22:0/4060051719 wait complete. 2024-03-29T04:18:49.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.588+0000 7f03108d5640 1 Processor -- start 2024-03-29T04:18:49.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.588+0000 7f03108d5640 1 -- start start 2024-03-29T04:18:49.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f03108d5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02f8003f40 0x7f030c156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:49.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f03108d5640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f030c0c9200 0x7f030c0c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:49.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f03108d5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f030c1588f0 0x7f030c158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:49.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f03108d5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f030c15cde0 con 0x7f030c1588f0 2024-03-29T04:18:49.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f03108d5640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f030c15cc60 con 0x7f02f8003f40 2024-03-29T04:18:49.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f03108d5640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f030c06cd80 con 0x7f030c0c9200 2024-03-29T04:18:49.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f030ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f030c1588f0 0x7f030c158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:49.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f030ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f030c1588f0 0x7f030c158d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43828/0 (socket says 172.21.15.22:43828) 2024-03-29T04:18:49.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f030ad9d640 1 -- 172.21.15.22:0/4143384321 learned_addr learned my addr 172.21.15.22:0/4143384321 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:49.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f030a59c640 1 --2- 172.21.15.22:0/4143384321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02f8003f40 0x7f030c156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:49.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f0309d9b640 1 --2- 172.21.15.22:0/4143384321 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f030c0c9200 0x7f030c0c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:49.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f030ad9d640 1 -- 172.21.15.22:0/4143384321 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f030c0c9200 msgr2=0x7f030c0c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:49.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f030ad9d640 1 --2- 172.21.15.22:0/4143384321 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f030c0c9200 0x7f030c0c9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:49.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f030ad9d640 1 -- 172.21.15.22:0/4143384321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02f8003f40 msgr2=0x7f030c156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:49.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f030ad9d640 1 --2- 172.21.15.22:0/4143384321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02f8003f40 0x7f030c156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:49.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f030ad9d640 1 -- 172.21.15.22:0/4143384321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030c1f8060 con 0x7f030c1588f0 2024-03-29T04:18:49.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f0309d9b640 1 --2- 172.21.15.22:0/4143384321 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f030c0c9200 0x7f030c0c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:49.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f030ad9d640 1 --2- 172.21.15.22:0/4143384321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f030c1588f0 0x7f030c158d50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f030c072b80 tx=0x7f0304010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:49.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f0302ffd640 1 -- 172.21.15.22:0/4143384321 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030401a070 con 0x7f030c1588f0 2024-03-29T04:18:49.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f0302ffd640 1 -- 172.21.15.22:0/4143384321 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0304011040 con 0x7f030c1588f0 2024-03-29T04:18:49.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f03108d5640 1 -- 172.21.15.22:0/4143384321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030c1f8340 con 0x7f030c1588f0 2024-03-29T04:18:49.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f0302ffd640 1 -- 172.21.15.22:0/4143384321 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030400ed50 con 0x7f030c1588f0 2024-03-29T04:18:49.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f03108d5640 1 -- 172.21.15.22:0/4143384321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f030c1f8900 con 0x7f030c1588f0 2024-03-29T04:18:49.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f03108d5640 1 -- 172.21.15.22:0/4143384321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f030c0687f0 con 0x7f030c1588f0 2024-03-29T04:18:49.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f0302ffd640 1 -- 172.21.15.22:0/4143384321 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f030401d070 con 0x7f030c1588f0 2024-03-29T04:18:49.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f0302ffd640 1 --2- 172.21.15.22:0/4143384321 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f02ec067fc0 0x7f02ec06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:49.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f0302ffd640 1 -- 172.21.15.22:0/4143384321 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6383+0+0 (secure 0 0 0) 0x7f0304088fc0 con 0x7f030c1588f0 2024-03-29T04:18:49.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.592+0000 7f030a59c640 1 --2- 172.21.15.22:0/4143384321 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f02ec067fc0 0x7f02ec06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:49.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.596+0000 7f030a59c640 1 --2- 172.21.15.22:0/4143384321 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f02ec067fc0 0x7f02ec06a480 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009950 tx=0x7f02fc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:49.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.600+0000 7f0302ffd640 1 -- 172.21.15.22:0/4143384321 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f03040594d0 con 0x7f030c1588f0 2024-03-29T04:18:49.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:49.760+0000 7f03108d5640 1 -- 172.21.15.22:0/4143384321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f030c1f8ad0 con 0x7f030c1588f0 2024-03-29T04:18:50.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.428+0000 7f0302ffd640 1 -- 172.21.15.22:0/4143384321 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) ==== 169+0+0 (secure 0 0 0) 0x7f0304058dd0 con 0x7f030c1588f0 2024-03-29T04:18:50.410 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' removed 2024-03-29T04:18:50.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.440+0000 7f03108d5640 1 -- 172.21.15.22:0/4143384321 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f02ec067fc0 msgr2=0x7f02ec06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:50.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.440+0000 7f03108d5640 1 --2- 172.21.15.22:0/4143384321 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f02ec067fc0 0x7f02ec06a480 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009950 tx=0x7f02fc008040 comp rx=0 tx=0).stop 2024-03-29T04:18:50.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.440+0000 7f03108d5640 1 -- 172.21.15.22:0/4143384321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f030c1588f0 msgr2=0x7f030c158d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:50.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.440+0000 7f03108d5640 1 --2- 172.21.15.22:0/4143384321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f030c1588f0 0x7f030c158d50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f030c072b80 tx=0x7f0304010d70 comp rx=0 tx=0).stop 2024-03-29T04:18:50.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.444+0000 7f03108d5640 1 -- 172.21.15.22:0/4143384321 shutdown_connections 2024-03-29T04:18:50.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.444+0000 7f03108d5640 1 --2- 172.21.15.22:0/4143384321 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f02ec067fc0 0x7f02ec06a480 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:50.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.444+0000 7f03108d5640 1 --2- 172.21.15.22:0/4143384321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f030c1588f0 0x7f030c158d50 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:50.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.444+0000 7f03108d5640 1 --2- 172.21.15.22:0/4143384321 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f030c0c9200 0x7f030c0c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:50.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.444+0000 7f03108d5640 1 --2- 172.21.15.22:0/4143384321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02f8003f40 0x7f030c156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:50.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.444+0000 7f03108d5640 1 -- 172.21.15.22:0/4143384321 >> 172.21.15.22:0/4143384321 conn(0x7f030c0b1b80 msgr2=0x7f030c15ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:50.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.444+0000 7f03108d5640 1 -- 172.21.15.22:0/4143384321 shutdown_connections 2024-03-29T04:18:50.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.444+0000 7f03108d5640 1 -- 172.21.15.22:0/4143384321 wait complete. 2024-03-29T04:18:50.442 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-29T04:18:50.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.584+0000 7f5d9d437640 1 Processor -- start 2024-03-29T04:18:50.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.584+0000 7f5d9d437640 1 -- start start 2024-03-29T04:18:50.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.588+0000 7f5d9d437640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d98159fb0 0x7f5d9815c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:50.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.588+0000 7f5d9d437640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5d980b1fc0 0x7f5d980ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:50.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.588+0000 7f5d9d437640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d980d0020 con 0x7f5d980b1b60 2024-03-29T04:18:50.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.588+0000 7f5d9d437640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f5d980ccfa0 con 0x7f5d980b1fc0 2024-03-29T04:18:50.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.588+0000 7f5d9d437640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5d9815ce70 con 0x7f5d98159fb0 2024-03-29T04:18:50.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.588+0000 7f5d977fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5d980b1fc0 0x7f5d980ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:50.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.588+0000 7f5d977fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5d980b1fc0 0x7f5d980ceb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43838/0 (socket says 172.21.15.22:43838) 2024-03-29T04:18:50.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.588+0000 7f5d977fe640 1 -- 172.21.15.22:0/2025881202 learned_addr learned my addr 172.21.15.22:0/2025881202 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:50.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.588+0000 7f5d96ffd640 1 --2- 172.21.15.22:0/2025881202 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d98159fb0 0x7f5d9815c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:50.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.588+0000 7f5d977fe640 1 -- 172.21.15.22:0/2025881202 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d98159fb0 msgr2=0x7f5d9815c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:50.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.588+0000 7f5d977fe640 1 --2- 172.21.15.22:0/2025881202 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d98159fb0 0x7f5d9815c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:50.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.588+0000 7f5d977fe640 1 -- 172.21.15.22:0/2025881202 >> v1:172.21.15.151:6789/0 conn(0x7f5d980b1b60 legacy=0x7f5d980cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:50.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.588+0000 7f5d977fe640 1 -- 172.21.15.22:0/2025881202 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9815d150 con 0x7f5d980b1fc0 2024-03-29T04:18:50.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.588+0000 7f5d96ffd640 1 --2- 172.21.15.22:0/2025881202 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d98159fb0 0x7f5d9815c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:50.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.588+0000 7f5d977fe640 1 --2- 172.21.15.22:0/2025881202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d980b1fc0 0x7f5d980ceb30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5d980d01f0 tx=0x7f5d9001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b6488bcdd2e0d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:50.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.588+0000 7f5d95ffb640 1 -- 172.21.15.22:0/2025881202 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d900a57f0 con 0x7f5d980b1fc0 2024-03-29T04:18:50.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.588+0000 7f5d95ffb640 1 -- 172.21.15.22:0/2025881202 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d900a5990 con 0x7f5d980b1fc0 2024-03-29T04:18:50.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.588+0000 7f5d95ffb640 1 -- 172.21.15.22:0/2025881202 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d900a5d60 con 0x7f5d980b1fc0 2024-03-29T04:18:50.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.588+0000 7f5d9d437640 1 -- 172.21.15.22:0/2025881202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d980b1fc0 msgr2=0x7f5d980ceb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:50.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.588+0000 7f5d9d437640 1 --2- 172.21.15.22:0/2025881202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d980b1fc0 0x7f5d980ceb30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5d980d01f0 tx=0x7f5d9001ca10 comp rx=0 tx=0).stop 2024-03-29T04:18:50.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d9d437640 1 -- 172.21.15.22:0/2025881202 shutdown_connections 2024-03-29T04:18:50.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d9d437640 1 --2- 172.21.15.22:0/2025881202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d980b1fc0 0x7f5d980ceb30 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:50.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d9d437640 1 --2- 172.21.15.22:0/2025881202 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d98159fb0 0x7f5d9815c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:50.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d9d437640 1 -- 172.21.15.22:0/2025881202 >> 172.21.15.22:0/2025881202 conn(0x7f5d980b3cf0 msgr2=0x7f5d980b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:50.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d9d437640 1 -- 172.21.15.22:0/2025881202 shutdown_connections 2024-03-29T04:18:50.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d9d437640 1 -- 172.21.15.22:0/2025881202 wait complete. 2024-03-29T04:18:50.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d9d437640 1 Processor -- start 2024-03-29T04:18:50.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d9d437640 1 -- start start 2024-03-29T04:18:50.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d9d437640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d980b1b60 0x7f5d98147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:50.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d9d437640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d980b1fc0 0x7f5d98157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:50.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d9d437640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d98159fb0 0x7f5d981484a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:50.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d9d437640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5d98169010 con 0x7f5d980b1b60 2024-03-29T04:18:50.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d9d437640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5d980d0f20 con 0x7f5d980b1fc0 2024-03-29T04:18:50.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d9d437640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5d98169190 con 0x7f5d98159fb0 2024-03-29T04:18:50.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d96ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d980b1b60 0x7f5d98147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:50.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d96ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d980b1b60 0x7f5d98147e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43854/0 (socket says 172.21.15.22:43854) 2024-03-29T04:18:50.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d96ffd640 1 -- 172.21.15.22:0/2216184576 learned_addr learned my addr 172.21.15.22:0/2216184576 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:50.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d967fc640 1 --2- 172.21.15.22:0/2216184576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d980b1fc0 0x7f5d98157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:50.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d977fe640 1 --2- 172.21.15.22:0/2216184576 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d98159fb0 0x7f5d981484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:50.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d96ffd640 1 -- 172.21.15.22:0/2216184576 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d98159fb0 msgr2=0x7f5d981484a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:50.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d96ffd640 1 --2- 172.21.15.22:0/2216184576 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d98159fb0 0x7f5d981484a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:50.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d96ffd640 1 -- 172.21.15.22:0/2216184576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d980b1fc0 msgr2=0x7f5d98157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:50.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d96ffd640 1 --2- 172.21.15.22:0/2216184576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d980b1fc0 0x7f5d98157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:50.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d96ffd640 1 -- 172.21.15.22:0/2216184576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d98149790 con 0x7f5d980b1b60 2024-03-29T04:18:50.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d96ffd640 1 --2- 172.21.15.22:0/2216184576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d980b1b60 0x7f5d98147e90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800c7f0 tx=0x7f5d8800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:50.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d7f7fe640 1 -- 172.21.15.22:0/2216184576 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d88015070 con 0x7f5d980b1b60 2024-03-29T04:18:50.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d7f7fe640 1 -- 172.21.15.22:0/2216184576 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d880077c0 con 0x7f5d980b1b60 2024-03-29T04:18:50.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d9d437640 1 -- 172.21.15.22:0/2216184576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d98148c10 con 0x7f5d980b1b60 2024-03-29T04:18:50.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d7f7fe640 1 -- 172.21.15.22:0/2216184576 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d88007bb0 con 0x7f5d980b1b60 2024-03-29T04:18:50.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d9d437640 1 -- 172.21.15.22:0/2216184576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d981491d0 con 0x7f5d980b1b60 2024-03-29T04:18:50.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.592+0000 7f5d9d437640 1 -- 172.21.15.22:0/2216184576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d980687f0 con 0x7f5d980b1b60 2024-03-29T04:18:50.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.596+0000 7f5d7f7fe640 1 -- 172.21.15.22:0/2216184576 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f5d88018070 con 0x7f5d980b1b60 2024-03-29T04:18:50.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.596+0000 7f5d7f7fe640 1 --2- 172.21.15.22:0/2216184576 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5d84067fc0 0x7f5d8406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:50.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.596+0000 7f5d7f7fe640 1 -- 172.21.15.22:0/2216184576 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5853+0+0 (secure 0 0 0) 0x7f5d880866b0 con 0x7f5d980b1b60 2024-03-29T04:18:50.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.596+0000 7f5d967fc640 1 --2- 172.21.15.22:0/2216184576 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5d84067fc0 0x7f5d8406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:50.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.596+0000 7f5d967fc640 1 --2- 172.21.15.22:0/2216184576 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5d84067fc0 0x7f5d8406a480 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5d98153c80 tx=0x7f5d8c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:50.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.600+0000 7f5d7f7fe640 1 -- 172.21.15.22:0/2216184576 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f5d88056dd0 con 0x7f5d980b1b60 2024-03-29T04:18:50.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:50.764+0000 7f5d9d437640 1 -- 172.21.15.22:0/2216184576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5d980b1940 con 0x7f5d980b1b60 2024-03-29T04:18:51.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.436+0000 7f5d7f7fe640 1 -- 172.21.15.22:0/2216184576 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) ==== 178+0+0 (secure 0 0 0) 0x7f5d880566d0 con 0x7f5d980b1b60 2024-03-29T04:18:51.419 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data_ec' removed 2024-03-29T04:18:51.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.448+0000 7f5d9d437640 1 -- 172.21.15.22:0/2216184576 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5d84067fc0 msgr2=0x7f5d8406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:51.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.448+0000 7f5d9d437640 1 --2- 172.21.15.22:0/2216184576 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5d84067fc0 0x7f5d8406a480 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5d98153c80 tx=0x7f5d8c009040 comp rx=0 tx=0).stop 2024-03-29T04:18:51.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.448+0000 7f5d9d437640 1 -- 172.21.15.22:0/2216184576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d980b1b60 msgr2=0x7f5d98147e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:51.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.448+0000 7f5d9d437640 1 --2- 172.21.15.22:0/2216184576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d980b1b60 0x7f5d98147e90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800c7f0 tx=0x7f5d8800ccb0 comp rx=0 tx=0).stop 2024-03-29T04:18:51.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.448+0000 7f5d9d437640 1 -- 172.21.15.22:0/2216184576 shutdown_connections 2024-03-29T04:18:51.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.448+0000 7f5d9d437640 1 --2- 172.21.15.22:0/2216184576 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d98159fb0 0x7f5d981484a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:51.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.448+0000 7f5d9d437640 1 --2- 172.21.15.22:0/2216184576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d980b1fc0 0x7f5d98157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:51.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.448+0000 7f5d9d437640 1 --2- 172.21.15.22:0/2216184576 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5d84067fc0 0x7f5d8406a480 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:51.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.448+0000 7f5d9d437640 1 --2- 172.21.15.22:0/2216184576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d980b1b60 0x7f5d98147e90 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:51.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.448+0000 7f5d9d437640 1 -- 172.21.15.22:0/2216184576 >> 172.21.15.22:0/2216184576 conn(0x7f5d980b3cf0 msgr2=0x7f5d980d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:51.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.448+0000 7f5d9d437640 1 -- 172.21.15.22:0/2216184576 shutdown_connections 2024-03-29T04:18:51.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.448+0000 7f5d9d437640 1 -- 172.21.15.22:0/2216184576 wait complete. 2024-03-29T04:18:51.464 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-29T04:18:51.464 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-29T04:18:51.464 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-29T04:18:51.464 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:18:51.464 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-29T04:18:51.464 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-29T04:18:51.465 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-29T04:18:51.465 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:18:51.465 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-29T04:18:51.465 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-29T04:18:51.465 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-29T04:18:51.465 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:18:51.465 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-29T04:18:51.465 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-29T04:18:51.466 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-29T04:18:51.466 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:18:51.475 INFO:tasks.ceph.mds.a:Stopped 2024-03-29T04:18:51.475 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-29T04:18:51.477 INFO:tasks.ceph.mds.c:Stopped 2024-03-29T04:18:51.478 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-29T04:18:51.480 INFO:tasks.ceph.mds.b:Stopped 2024-03-29T04:18:51.480 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-29T04:18:51.482 INFO:tasks.ceph.mds.d:Stopped 2024-03-29T04:18:51.482 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-29T04:18:51.520 INFO:tasks.ceph.mds.a:Started 2024-03-29T04:18:51.520 DEBUG:teuthology.parallel:result is None 2024-03-29T04:18:51.521 INFO:tasks.ceph.mds.c:Started 2024-03-29T04:18:51.521 DEBUG:teuthology.parallel:result is None 2024-03-29T04:18:51.526 INFO:tasks.ceph.mds.b:Started 2024-03-29T04:18:51.526 DEBUG:teuthology.parallel:result is None 2024-03-29T04:18:51.527 INFO:tasks.ceph.mds.d:Started 2024-03-29T04:18:51.527 DEBUG:teuthology.parallel:result is None 2024-03-29T04:18:51.528 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-29T04:18:51.586 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2024-03-29T04:18:51.591 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2024-03-29T04:18:51.596 INFO:tasks.ceph.mds.b.smithi151.stdout:starting mds.b at 2024-03-29T04:18:51.599 INFO:tasks.ceph.mds.d.smithi151.stdout:starting mds.d at 2024-03-29T04:18:51.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.676+0000 7f60c4023640 1 Processor -- start 2024-03-29T04:18:51.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.676+0000 7f60c4023640 1 -- start start 2024-03-29T04:18:51.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.676+0000 7f60c4023640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f60bc15e400 0x7f60bc0b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:51.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.676+0000 7f60c4023640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f60bc0ad8e0 0x7f60bc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:51.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.676+0000 7f60c4023640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f60bc06dcc0 0x7f60bc06b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:51.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.676+0000 7f60c4023640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f60bc06f190 con 0x7f60bc06dcc0 2024-03-29T04:18:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.676+0000 7f60c4023640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f60bc0b08a0 con 0x7f60bc0ad8e0 2024-03-29T04:18:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.676+0000 7f60c4023640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f60bc06f3e0 con 0x7f60bc15e400 2024-03-29T04:18:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.676+0000 7f60c25c0640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f60bc06dcc0 0x7f60bc06b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.676+0000 7f60c25c0640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f60bc06dcc0 0x7f60bc06b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43894/0 (socket says 172.21.15.22:43894) 2024-03-29T04:18:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.676+0000 7f60c25c0640 1 -- 172.21.15.22:0/1019575919 learned_addr learned my addr 172.21.15.22:0/1019575919 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.676+0000 7f60c1dbf640 1 --2- 172.21.15.22:0/1019575919 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f60bc15e400 0x7f60bc0b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.676+0000 7f60c15be640 1 --2- 172.21.15.22:0/1019575919 >> v2:172.21.15.151:3300/0 conn(0x7f60bc0ad8e0 0x7f60bc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.676+0000 7f60c25c0640 1 -- 172.21.15.22:0/1019575919 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f60bc15e400 msgr2=0x7f60bc0b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.676+0000 7f60c25c0640 1 --2- 172.21.15.22:0/1019575919 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f60bc15e400 0x7f60bc0b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:51.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.676+0000 7f60c25c0640 1 -- 172.21.15.22:0/1019575919 >> v2:172.21.15.151:3300/0 conn(0x7f60bc0ad8e0 msgr2=0x7f60bc06d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:51.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.676+0000 7f60c25c0640 1 --2- 172.21.15.22:0/1019575919 >> v2:172.21.15.151:3300/0 conn(0x7f60bc0ad8e0 0x7f60bc06d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:51.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.676+0000 7f60c25c0640 1 -- 172.21.15.22:0/1019575919 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60bc06f6c0 con 0x7f60bc06dcc0 2024-03-29T04:18:51.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c25c0640 1 --2- 172.21.15.22:0/1019575919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60bc06dcc0 0x7f60bc06b660 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f60bc06c6f0 tx=0x7f60b401c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d141d2cc2be8ba1a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:51.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c0dbd640 1 -- 172.21.15.22:0/1019575919 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b40ac070 con 0x7f60bc06dcc0 2024-03-29T04:18:51.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c0dbd640 1 -- 172.21.15.22:0/1019575919 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60b40a8420 con 0x7f60bc06dcc0 2024-03-29T04:18:51.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c0dbd640 1 -- 172.21.15.22:0/1019575919 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b40a87f0 con 0x7f60bc06dcc0 2024-03-29T04:18:51.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c4023640 1 -- 172.21.15.22:0/1019575919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60bc06dcc0 msgr2=0x7f60bc06b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:51.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c4023640 1 --2- 172.21.15.22:0/1019575919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60bc06dcc0 0x7f60bc06b660 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f60bc06c6f0 tx=0x7f60b401c730 comp rx=0 tx=0).stop 2024-03-29T04:18:51.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c4023640 1 -- 172.21.15.22:0/1019575919 shutdown_connections 2024-03-29T04:18:51.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c4023640 1 --2- 172.21.15.22:0/1019575919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60bc06dcc0 0x7f60bc06b660 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:51.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c4023640 1 --2- 172.21.15.22:0/1019575919 >> v2:172.21.15.151:3300/0 conn(0x7f60bc0ad8e0 0x7f60bc06d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:51.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c4023640 1 --2- 172.21.15.22:0/1019575919 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f60bc15e400 0x7f60bc0b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:51.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c4023640 1 -- 172.21.15.22:0/1019575919 >> 172.21.15.22:0/1019575919 conn(0x7f60bc072500 msgr2=0x7f60bc072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:51.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c4023640 1 -- 172.21.15.22:0/1019575919 shutdown_connections 2024-03-29T04:18:51.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c4023640 1 -- 172.21.15.22:0/1019575919 wait complete. 2024-03-29T04:18:51.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c4023640 1 Processor -- start 2024-03-29T04:18:51.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c4023640 1 -- start start 2024-03-29T04:18:51.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c4023640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60bc06dcc0 0x7f60bc14dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:51.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c4023640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60bc0ad8e0 0x7f60bc158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:51.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c4023640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f60bc15e400 0x7f60bc13e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:51.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c4023640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc0affe0 con 0x7f60bc06dcc0 2024-03-29T04:18:51.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c4023640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc0afe60 con 0x7f60bc0ad8e0 2024-03-29T04:18:51.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c4023640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f60bc0b54e0 con 0x7f60bc15e400 2024-03-29T04:18:51.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c1dbf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60bc06dcc0 0x7f60bc14dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:51.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c1dbf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60bc06dcc0 0x7f60bc14dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43906/0 (socket says 172.21.15.22:43906) 2024-03-29T04:18:51.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.680+0000 7f60c1dbf640 1 -- 172.21.15.22:0/2576588663 learned_addr learned my addr 172.21.15.22:0/2576588663 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:51.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.684+0000 7f60c25c0640 1 --2- 172.21.15.22:0/2576588663 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f60bc15e400 0x7f60bc13e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:51.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.684+0000 7f60c15be640 1 --2- 172.21.15.22:0/2576588663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60bc0ad8e0 0x7f60bc158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:51.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.684+0000 7f60c1dbf640 1 -- 172.21.15.22:0/2576588663 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f60bc15e400 msgr2=0x7f60bc13e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:51.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.684+0000 7f60c1dbf640 1 --2- 172.21.15.22:0/2576588663 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f60bc15e400 0x7f60bc13e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:51.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.684+0000 7f60c1dbf640 1 -- 172.21.15.22:0/2576588663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60bc0ad8e0 msgr2=0x7f60bc158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:51.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.684+0000 7f60c1dbf640 1 --2- 172.21.15.22:0/2576588663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60bc0ad8e0 0x7f60bc158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:51.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.684+0000 7f60c1dbf640 1 -- 172.21.15.22:0/2576588663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60bc1585b0 con 0x7f60bc06dcc0 2024-03-29T04:18:51.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.684+0000 7f60c25c0640 1 --2- 172.21.15.22:0/2576588663 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f60bc15e400 0x7f60bc13e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:51.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.684+0000 7f60c1dbf640 1 --2- 172.21.15.22:0/2576588663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60bc06dcc0 0x7f60bc14dba0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00e8a0 tx=0x7f60ac00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:51.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.684+0000 7f60ba7fc640 1 -- 172.21.15.22:0/2576588663 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac017070 con 0x7f60bc06dcc0 2024-03-29T04:18:51.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.684+0000 7f60ba7fc640 1 -- 172.21.15.22:0/2576588663 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ac00c940 con 0x7f60bc06dcc0 2024-03-29T04:18:51.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.684+0000 7f60c4023640 1 -- 172.21.15.22:0/2576588663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60bc158880 con 0x7f60bc06dcc0 2024-03-29T04:18:51.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.684+0000 7f60ba7fc640 1 -- 172.21.15.22:0/2576588663 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac00ccf0 con 0x7f60bc06dcc0 2024-03-29T04:18:51.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.684+0000 7f60c4023640 1 -- 172.21.15.22:0/2576588663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60bc15f9e0 con 0x7f60bc06dcc0 2024-03-29T04:18:51.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.684+0000 7f60c4023640 1 -- 172.21.15.22:0/2576588663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60bc068890 con 0x7f60bc06dcc0 2024-03-29T04:18:51.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.684+0000 7f60ba7fc640 1 -- 172.21.15.22:0/2576588663 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f60ac01a070 con 0x7f60bc06dcc0 2024-03-29T04:18:51.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.684+0000 7f60ba7fc640 1 --2- 172.21.15.22:0/2576588663 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f60a8067fc0 0x7f60a806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:51.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.684+0000 7f60ba7fc640 1 -- 172.21.15.22:0/2576588663 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5335+0+0 (secure 0 0 0) 0x7f60ac086220 con 0x7f60bc06dcc0 2024-03-29T04:18:51.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.684+0000 7f60c15be640 1 --2- 172.21.15.22:0/2576588663 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f60a8067fc0 0x7f60a806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:51.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.684+0000 7f60c15be640 1 --2- 172.21.15.22:0/2576588663 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f60a8067fc0 0x7f60a806a480 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f60bc14cb70 tx=0x7f60b0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:51.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.688+0000 7f60ba7fc640 1 -- 172.21.15.22:0/2576588663 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f60ac056b40 con 0x7f60bc06dcc0 2024-03-29T04:18:51.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:51.860+0000 7f60c4023640 1 -- 172.21.15.22:0/2576588663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f60bc06b980 con 0x7f60bc06dcc0 2024-03-29T04:18:52.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.456+0000 7f60ba7fc640 1 -- 172.21.15.22:0/2576588663 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) ==== 97+0+0 (secure 0 0 0) 0x7f60ac056440 con 0x7f60bc06dcc0 2024-03-29T04:18:52.436 INFO:teuthology.orchestra.run.smithi022.stderr: removed all blocklist entries 2024-03-29T04:18:52.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.460+0000 7f60c4023640 1 -- 172.21.15.22:0/2576588663 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f60a8067fc0 msgr2=0x7f60a806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:52.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.460+0000 7f60c4023640 1 --2- 172.21.15.22:0/2576588663 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f60a8067fc0 0x7f60a806a480 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f60bc14cb70 tx=0x7f60b0008040 comp rx=0 tx=0).stop 2024-03-29T04:18:52.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.460+0000 7f60c4023640 1 -- 172.21.15.22:0/2576588663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60bc06dcc0 msgr2=0x7f60bc14dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:52.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.460+0000 7f60c4023640 1 --2- 172.21.15.22:0/2576588663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60bc06dcc0 0x7f60bc14dba0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00e8a0 tx=0x7f60ac00eca0 comp rx=0 tx=0).stop 2024-03-29T04:18:52.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.460+0000 7f60c4023640 1 -- 172.21.15.22:0/2576588663 shutdown_connections 2024-03-29T04:18:52.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.460+0000 7f60c4023640 1 --2- 172.21.15.22:0/2576588663 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f60bc15e400 0x7f60bc13e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:52.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.460+0000 7f60c4023640 1 --2- 172.21.15.22:0/2576588663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60bc0ad8e0 0x7f60bc158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:52.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.460+0000 7f60c4023640 1 --2- 172.21.15.22:0/2576588663 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f60a8067fc0 0x7f60a806a480 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:52.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.460+0000 7f60c4023640 1 --2- 172.21.15.22:0/2576588663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60bc06dcc0 0x7f60bc14dba0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:52.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.460+0000 7f60c4023640 1 -- 172.21.15.22:0/2576588663 >> 172.21.15.22:0/2576588663 conn(0x7f60bc072500 msgr2=0x7f60bc0ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:52.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.460+0000 7f60c4023640 1 -- 172.21.15.22:0/2576588663 shutdown_connections 2024-03-29T04:18:52.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.460+0000 7f60c4023640 1 -- 172.21.15.22:0/2576588663 wait complete. 2024-03-29T04:18:52.463 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-29T04:18:52.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.600+0000 7f5526d34640 1 Processor -- start 2024-03-29T04:18:52.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.600+0000 7f5526d34640 1 -- start start 2024-03-29T04:18:52.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.600+0000 7f5526d34640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f55200b0090 0x7f5520158530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:52.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.600+0000 7f5526d34640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f55200b0470 0x7f5520158a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:52.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.600+0000 7f5526d34640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f552006c350 con 0x7f552006b660 2024-03-29T04:18:52.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.600+0000 7f5526d34640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f552006e780 con 0x7f55200b0470 2024-03-29T04:18:52.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.600+0000 7f5526d34640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f552006c630 con 0x7f55200b0090 2024-03-29T04:18:52.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f551ffff640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f55200b0470 0x7f5520158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:52.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f551ffff640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f55200b0470 0x7f5520158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:35148/0 (socket says 172.21.15.22:35148) 2024-03-29T04:18:52.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f55252d1640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f552006b660 0x7f552006bad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.22:38554/0 (socket says 172.21.15.22:38554) 2024-03-29T04:18:52.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f551ffff640 1 -- 172.21.15.22:0/1524713874 learned_addr learned my addr 172.21.15.22:0/1524713874 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:52.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f5524ad0640 1 --2- 172.21.15.22:0/1524713874 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f55200b0090 0x7f5520158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:52.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f5524ad0640 1 -- 172.21.15.22:0/1524713874 >> v2:172.21.15.151:3300/0 conn(0x7f55200b0470 msgr2=0x7f5520158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:52.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f5524ad0640 1 --2- 172.21.15.22:0/1524713874 >> v2:172.21.15.151:3300/0 conn(0x7f55200b0470 0x7f5520158a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:52.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f5524ad0640 1 -- 172.21.15.22:0/1524713874 >> v1:172.21.15.151:6789/0 conn(0x7f552006b660 legacy=0x7f552006bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:52.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f5524ad0640 1 -- 172.21.15.22:0/1524713874 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552006c910 con 0x7f55200b0090 2024-03-29T04:18:52.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f551ffff640 1 --2- 172.21.15.22:0/1524713874 >> v2:172.21.15.151:3300/0 conn(0x7f55200b0470 0x7f5520158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:18:52.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f5524ad0640 1 --2- 172.21.15.22:0/1524713874 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f55200b0090 0x7f5520158530 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f55100097f0 tx=0x7f551001c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36dbbdca4da6868 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:52.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f551f7fe640 1 -- 172.21.15.22:0/1524713874 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55100b0070 con 0x7f55200b0090 2024-03-29T04:18:52.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f551f7fe640 1 -- 172.21.15.22:0/1524713874 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55100ab600 con 0x7f55200b0090 2024-03-29T04:18:52.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f551f7fe640 1 -- 172.21.15.22:0/1524713874 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55100ab9d0 con 0x7f55200b0090 2024-03-29T04:18:52.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f5526d34640 1 -- 172.21.15.22:0/1524713874 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f55200b0090 msgr2=0x7f5520158530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:52.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f5526d34640 1 --2- 172.21.15.22:0/1524713874 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f55200b0090 0x7f5520158530 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f55100097f0 tx=0x7f551001c380 comp rx=0 tx=0).stop 2024-03-29T04:18:52.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f5526d34640 1 -- 172.21.15.22:0/1524713874 shutdown_connections 2024-03-29T04:18:52.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f5526d34640 1 --2- 172.21.15.22:0/1524713874 >> v2:172.21.15.151:3300/0 conn(0x7f55200b0470 0x7f5520158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:52.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f5526d34640 1 --2- 172.21.15.22:0/1524713874 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f55200b0090 0x7f5520158530 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:52.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f5526d34640 1 -- 172.21.15.22:0/1524713874 >> 172.21.15.22:0/1524713874 conn(0x7f55200b2c70 msgr2=0x7f55200b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:52.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f5526d34640 1 -- 172.21.15.22:0/1524713874 shutdown_connections 2024-03-29T04:18:52.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.604+0000 7f5526d34640 1 -- 172.21.15.22:0/1524713874 wait complete. 2024-03-29T04:18:52.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f5526d34640 1 Processor -- start 2024-03-29T04:18:52.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f5526d34640 1 -- start start 2024-03-29T04:18:52.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f5526d34640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f552006b660 0x7f55200ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:52.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f5526d34640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55200b0090 0x7f5520157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:52.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f5526d34640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55200b0470 0x7f552014e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:52.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f5526d34640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f55200b5f00 con 0x7f55200b0470 2024-03-29T04:18:52.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f5526d34640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f55200b5d80 con 0x7f55200b0090 2024-03-29T04:18:52.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f5526d34640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f55200b6080 con 0x7f552006b660 2024-03-29T04:18:52.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f55252d1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55200b0470 0x7f552014e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:52.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f55252d1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55200b0470 0x7f552014e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43922/0 (socket says 172.21.15.22:43922) 2024-03-29T04:18:52.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f55252d1640 1 -- 172.21.15.22:0/3791630650 learned_addr learned my addr 172.21.15.22:0/3791630650 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:52.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f551ffff640 1 --2- 172.21.15.22:0/3791630650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55200b0090 0x7f5520157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:52.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f5524ad0640 1 --2- 172.21.15.22:0/3791630650 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f552006b660 0x7f55200ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:52.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f55252d1640 1 -- 172.21.15.22:0/3791630650 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f552006b660 msgr2=0x7f55200ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:52.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f55252d1640 1 --2- 172.21.15.22:0/3791630650 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f552006b660 0x7f55200ae0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:52.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f55252d1640 1 -- 172.21.15.22:0/3791630650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55200b0090 msgr2=0x7f5520157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:52.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f55252d1640 1 --2- 172.21.15.22:0/3791630650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55200b0090 0x7f5520157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:52.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f55252d1640 1 -- 172.21.15.22:0/3791630650 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5520157870 con 0x7f55200b0470 2024-03-29T04:18:52.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f55252d1640 1 --2- 172.21.15.22:0/3791630650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55200b0470 0x7f552014e2a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f551800c730 tx=0x7f551800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:52.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f551d7fa640 1 -- 172.21.15.22:0/3791630650 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518018070 con 0x7f55200b0470 2024-03-29T04:18:52.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f5526d34640 1 -- 172.21.15.22:0/3791630650 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f552014e7e0 con 0x7f55200b0470 2024-03-29T04:18:52.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f551d7fa640 1 -- 172.21.15.22:0/3791630650 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5518004d70 con 0x7f55200b0470 2024-03-29T04:18:52.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f551d7fa640 1 -- 172.21.15.22:0/3791630650 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551800a880 con 0x7f55200b0470 2024-03-29T04:18:52.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f5526d34640 1 -- 172.21.15.22:0/3791630650 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55200ae980 con 0x7f55200b0470 2024-03-29T04:18:52.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.608+0000 7f5526d34640 1 -- 172.21.15.22:0/3791630650 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5520068890 con 0x7f55200b0470 2024-03-29T04:18:52.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.612+0000 7f551d7fa640 1 -- 172.21.15.22:0/3791630650 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f551800aa20 con 0x7f55200b0470 2024-03-29T04:18:52.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.612+0000 7f551d7fa640 1 --2- 172.21.15.22:0/3791630650 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f550c067fc0 0x7f550c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:52.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.612+0000 7f551d7fa640 1 -- 172.21.15.22:0/3791630650 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f55180856c0 con 0x7f55200b0470 2024-03-29T04:18:52.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.612+0000 7f5524ad0640 1 --2- 172.21.15.22:0/3791630650 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f550c067fc0 0x7f550c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:52.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.612+0000 7f5524ad0640 1 --2- 172.21.15.22:0/3791630650 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f550c067fc0 0x7f550c06a480 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f55100ad000 tx=0x7f55100a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:52.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.616+0000 7f551d7fa640 1 -- 172.21.15.22:0/3791630650 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f5520157870 con 0x7f55200b0470 2024-03-29T04:18:52.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.836+0000 7f5526d34640 1 -- 172.21.15.22:0/3791630650 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f55200b4810 con 0x7f55200b0470 2024-03-29T04:18:52.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.836+0000 7f551d7fa640 1 -- 172.21.15.22:0/3791630650 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7f5518055960 con 0x7f55200b0470 2024-03-29T04:18:52.818 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:52.818 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-29T04:18:52.819 INFO:teuthology.orchestra.run.smithi022.stdout: "auth_dump": [ 2024-03-29T04:18:52.819 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.819 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.a", 2024-03-29T04:18:52.819 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQChQAZm8SiSFBAAaJWqaxBx6kDLU7I0kFLktA==", 2024-03-29T04:18:52.819 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.819 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-03-29T04:18:52.819 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-03-29T04:18:52.819 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-03-29T04:18:52.819 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-03-29T04:18:52.819 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.819 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.819 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.820 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.b", 2024-03-29T04:18:52.820 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQChQAZmiqqdIBAAzeqQSv7ISWKkAKFtOvkVAw==", 2024-03-29T04:18:52.820 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.820 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-03-29T04:18:52.820 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-03-29T04:18:52.820 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-03-29T04:18:52.820 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-03-29T04:18:52.820 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.820 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.820 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.c", 2024-03-29T04:18:52.820 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQChQAZmKb50GhAAnYbR1X7mjwGXOIBuUyKbyg==", 2024-03-29T04:18:52.820 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.821 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-03-29T04:18:52.821 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-03-29T04:18:52.821 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-03-29T04:18:52.821 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-03-29T04:18:52.821 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.821 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.821 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.821 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.d", 2024-03-29T04:18:52.821 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQChQAZmUioOKRAADH572VL0PgMZtPBAkSo5Zg==", 2024-03-29T04:18:52.821 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.821 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-03-29T04:18:52.821 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-03-29T04:18:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-03-29T04:18:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-03-29T04:18:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.0", 2024-03-29T04:18:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQCjQAZmPvDTBRAATcIvO00st0mEUJajJ74c3g==", 2024-03-29T04:18:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-03-29T04:18:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-03-29T04:18:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-03-29T04:18:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.823 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.823 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.1", 2024-03-29T04:18:52.823 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQCmQAZmghcIChAAJtWKve3N64fgrFbwqAu5fA==", 2024-03-29T04:18:52.823 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.823 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-03-29T04:18:52.823 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-03-29T04:18:52.823 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-03-29T04:18:52.823 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.823 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.823 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.2", 2024-03-29T04:18:52.823 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQCpQAZm6cmbEBAAX8HSs37nD+h4M5NxnaSCvQ==", 2024-03-29T04:18:52.824 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.824 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-03-29T04:18:52.824 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-03-29T04:18:52.824 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-03-29T04:18:52.824 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.824 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.824 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.824 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.3", 2024-03-29T04:18:52.824 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQCsQAZmuoEQGBAAHej5NEVOUGzYe7X0uQ16Fg==", 2024-03-29T04:18:52.824 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.824 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-03-29T04:18:52.824 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-03-29T04:18:52.824 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-03-29T04:18:52.825 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.825 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.825 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.825 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.4", 2024-03-29T04:18:52.825 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQCyQAZmqVNfExAAOqWGiHZ5tauRUw5vYLn0dw==", 2024-03-29T04:18:52.825 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.825 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-03-29T04:18:52.825 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-03-29T04:18:52.825 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-03-29T04:18:52.825 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.825 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.825 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.825 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.5", 2024-03-29T04:18:52.826 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQC1QAZmMuxLGBAAml0kEA4UJBm6TlDEC2wvjA==", 2024-03-29T04:18:52.826 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.826 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-03-29T04:18:52.826 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-03-29T04:18:52.826 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-03-29T04:18:52.826 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.826 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.826 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.6", 2024-03-29T04:18:52.826 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQC4QAZmROE5HxAAE7wEvV6p+bHMWB3tca+PxQ==", 2024-03-29T04:18:52.826 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.826 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-03-29T04:18:52.827 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-03-29T04:18:52.827 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-03-29T04:18:52.827 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.827 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.827 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.7", 2024-03-29T04:18:52.827 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQC7QAZmPyGqJRAAX2sQ5dQpmAFwMltdIP5ivg==", 2024-03-29T04:18:52.827 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.827 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-03-29T04:18:52.827 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-03-29T04:18:52.827 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-03-29T04:18:52.827 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.828 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.828 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.0", 2024-03-29T04:18:52.828 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQChQAZmViasLhAA6Vlzyow8ABjacgHj4nOI/A==", 2024-03-29T04:18:52.828 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.828 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-03-29T04:18:52.828 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-03-29T04:18:52.828 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-03-29T04:18:52.828 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-03-29T04:18:52.828 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.828 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.828 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.828 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.1", 2024-03-29T04:18:52.829 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQChQAZmPjHrNhAAF2KuoExf7NtGgDgQAORiKA==", 2024-03-29T04:18:52.829 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.829 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-03-29T04:18:52.829 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-03-29T04:18:52.829 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-03-29T04:18:52.829 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-03-29T04:18:52.829 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.829 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.829 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.829 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.2", 2024-03-29T04:18:52.829 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQChQAZmyA39MxAAyXqwQAI7pdhqMfjtFNZnQg==", 2024-03-29T04:18:52.829 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.829 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-03-29T04:18:52.830 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-03-29T04:18:52.830 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-03-29T04:18:52.830 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-03-29T04:18:52.830 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.830 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.830 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.830 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.3", 2024-03-29T04:18:52.830 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQCiQAZmx/GrABAAQG0VVWy/qu6lpNKiombqQw==", 2024-03-29T04:18:52.830 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.830 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-03-29T04:18:52.830 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-03-29T04:18:52.830 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-03-29T04:18:52.831 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-03-29T04:18:52.831 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.831 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.831 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.831 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.admin", 2024-03-29T04:18:52.831 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQCgQAZmb4RpOBAAP6XtH09rEis7nWUy2KM2BQ==", 2024-03-29T04:18:52.831 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.831 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-03-29T04:18:52.831 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-03-29T04:18:52.831 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-03-29T04:18:52.831 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-03-29T04:18:52.831 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.831 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.832 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.832 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-mds", 2024-03-29T04:18:52.832 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDGQAZm2HnEMxAA/YqpRfIBRrwBO4JVCKcg5A==", 2024-03-29T04:18:52.832 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.832 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-mds" 2024-03-29T04:18:52.832 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.832 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.832 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-mgr", 2024-03-29T04:18:52.832 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDGQAZmApbEMxAAd67TORaBaIkKls32E5cv7g==", 2024-03-29T04:18:52.832 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.832 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-29T04:18:52.832 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.833 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.833 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.833 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-osd", 2024-03-29T04:18:52.833 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDGQAZmDqzEMxAAc7adI2POsGAEuifh80tPOw==", 2024-03-29T04:18:52.833 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.833 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-osd" 2024-03-29T04:18:52.833 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.833 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.833 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.833 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-rbd", 2024-03-29T04:18:52.833 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDGQAZmI8fEMxAA8nccKTv+k/AVCGDKSvOnfg==", 2024-03-29T04:18:52.833 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.834 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-29T04:18:52.834 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.834 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.834 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.834 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-29T04:18:52.834 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDGQAZm9t3EMxAAQR23CeRUeCtnVjwiODgkQA==", 2024-03-29T04:18:52.834 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.834 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-29T04:18:52.834 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.834 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.834 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.834 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-rgw", 2024-03-29T04:18:52.835 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDGQAZmM/TEMxAAG6FSazngtkTKXin3ewRg5g==", 2024-03-29T04:18:52.835 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.835 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-29T04:18:52.835 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.835 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.835 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.835 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.x", 2024-03-29T04:18:52.835 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQChQAZmttHXEBAAtEZLIP57GMS6Hl5oD0g7cg==", 2024-03-29T04:18:52.835 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.835 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-03-29T04:18:52.835 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2024-03-29T04:18:52.835 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-03-29T04:18:52.835 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.836 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:18:52.836 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:52.836 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.y", 2024-03-29T04:18:52.836 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQChQAZm9VsMDhAAyPeg5VhuJCh+ckoeg2oyLA==", 2024-03-29T04:18:52.836 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-03-29T04:18:52.836 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-03-29T04:18:52.836 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2024-03-29T04:18:52.836 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-03-29T04:18:52.836 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.836 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:52.836 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-03-29T04:18:52.836 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-29T04:18:52.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.840+0000 7f5526d34640 1 -- 172.21.15.22:0/3791630650 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f550c067fc0 msgr2=0x7f550c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:52.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.840+0000 7f5526d34640 1 --2- 172.21.15.22:0/3791630650 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f550c067fc0 0x7f550c06a480 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f55100ad000 tx=0x7f55100a6000 comp rx=0 tx=0).stop 2024-03-29T04:18:52.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.840+0000 7f5526d34640 1 -- 172.21.15.22:0/3791630650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55200b0470 msgr2=0x7f552014e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:52.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.840+0000 7f5526d34640 1 --2- 172.21.15.22:0/3791630650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55200b0470 0x7f552014e2a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f551800c730 tx=0x7f551800cb30 comp rx=0 tx=0).stop 2024-03-29T04:18:52.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.840+0000 7f5526d34640 1 -- 172.21.15.22:0/3791630650 shutdown_connections 2024-03-29T04:18:52.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.840+0000 7f5526d34640 1 --2- 172.21.15.22:0/3791630650 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f550c067fc0 0x7f550c06a480 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:52.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.840+0000 7f5526d34640 1 --2- 172.21.15.22:0/3791630650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55200b0470 0x7f552014e2a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:52.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.840+0000 7f5526d34640 1 --2- 172.21.15.22:0/3791630650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55200b0090 0x7f5520157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:52.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.840+0000 7f5526d34640 1 --2- 172.21.15.22:0/3791630650 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f552006b660 0x7f55200ae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:52.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.840+0000 7f5526d34640 1 -- 172.21.15.22:0/3791630650 >> 172.21.15.22:0/3791630650 conn(0x7f55200b2c70 msgr2=0x7f5520154b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:52.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.840+0000 7f5526d34640 1 -- 172.21.15.22:0/3791630650 shutdown_connections 2024-03-29T04:18:52.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.840+0000 7f5526d34640 1 -- 172.21.15.22:0/3791630650 wait complete. 2024-03-29T04:18:52.839 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-29T04:18:52.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94ca83f640 1 Processor -- start 2024-03-29T04:18:52.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94ca83f640 1 -- start start 2024-03-29T04:18:52.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94ca83f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94c406cd80 0x7f94c4144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:52.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94ca83f640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c40b4ba0 con 0x7f94c40b2ef0 2024-03-29T04:18:52.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94ca83f640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c40b33f0 con 0x7f94c4144270 2024-03-29T04:18:52.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94ca83f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f94c4144fa0 con 0x7f94c406cd80 2024-03-29T04:18:52.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94bbfff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f94c40b2ef0 0x7f94c406c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42678/0 (socket says 172.21.15.22:42678) 2024-03-29T04:18:52.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94bbfff640 1 -- 172.21.15.22:0/2606899775 learned_addr learned my addr 172.21.15.22:0/2606899775 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:52.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94c8ddc640 1 --2- 172.21.15.22:0/2606899775 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94c406cd80 0x7f94c4144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:52.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94c37fe640 1 -- 172.21.15.22:0/2606899775 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 541789570 0 0) 0x7f94c40b4ba0 con 0x7f94c40b2ef0 2024-03-29T04:18:52.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94c37fe640 1 -- 172.21.15.22:0/2606899775 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94b0005740 con 0x7f94c40b2ef0 2024-03-29T04:18:52.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94c37fe640 1 -- 172.21.15.22:0/2606899775 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1573585720 0 0) 0x7f94b0005740 con 0x7f94c40b2ef0 2024-03-29T04:18:52.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94c37fe640 1 -- 172.21.15.22:0/2606899775 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94c406cd80 msgr2=0x7f94c4144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:52.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94c37fe640 1 --2- 172.21.15.22:0/2606899775 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94c406cd80 0x7f94c4144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:52.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94c37fe640 1 -- 172.21.15.22:0/2606899775 >> v1:172.21.15.151:6789/0 conn(0x7f94c4144270 legacy=0x7f94c40b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:52.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94c37fe640 1 -- 172.21.15.22:0/2606899775 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c40b36d0 con 0x7f94c40b2ef0 2024-03-29T04:18:52.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94c37fe640 1 -- 172.21.15.22:0/2606899775 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f94ac001680 con 0x7f94c40b2ef0 2024-03-29T04:18:52.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94c37fe640 1 -- 172.21.15.22:0/2606899775 >> v1:172.21.15.22:6789/0 conn(0x7f94c40b2ef0 legacy=0x7f94c406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:52.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94c37fe640 1 --2- 172.21.15.22:0/2606899775 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94c406cd80 0x7f94b0005c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:52.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94c37fe640 1 --2- 172.21.15.22:0/2606899775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b0006140 0x7f94b0006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:52.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94c37fe640 1 --2- 172.21.15.22:0/2606899775 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94b0006f60 0x7f94b00073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:52.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94c37fe640 1 -- 172.21.15.22:0/2606899775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f94b000dc10 con 0x7f94b0006140 2024-03-29T04:18:52.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94c37fe640 1 -- 172.21.15.22:0/2606899775 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f94b000ddc0 con 0x7f94b0006f60 2024-03-29T04:18:52.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94c37fe640 1 -- 172.21.15.22:0/2606899775 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f94b000dfc0 con 0x7f94c406cd80 2024-03-29T04:18:52.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94bbfff640 1 --2- 172.21.15.22:0/2606899775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b0006140 0x7f94b0006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:52.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94c8ddc640 1 --2- 172.21.15.22:0/2606899775 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94b0006f60 0x7f94b00073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:52.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94c3fff640 1 --2- 172.21.15.22:0/2606899775 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94c406cd80 0x7f94b0005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:52.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94bbfff640 1 -- 172.21.15.22:0/2606899775 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94c406cd80 msgr2=0x7f94b0005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:52.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94bbfff640 1 --2- 172.21.15.22:0/2606899775 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94c406cd80 0x7f94b0005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:52.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94bbfff640 1 -- 172.21.15.22:0/2606899775 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94b0006f60 msgr2=0x7f94b00073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:52.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94bbfff640 1 --2- 172.21.15.22:0/2606899775 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94b0006f60 0x7f94b00073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:52.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94bbfff640 1 -- 172.21.15.22:0/2606899775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b0005740 con 0x7f94b0006140 2024-03-29T04:18:52.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.988+0000 7f94c3fff640 1 --2- 172.21.15.22:0/2606899775 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94c406cd80 0x7f94b0005c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:52.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.992+0000 7f94bbfff640 1 --2- 172.21.15.22:0/2606899775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b0006140 0x7f94b0006a20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00c210 tx=0x7f94ac01cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889aecae43e7cbf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:52.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.992+0000 7f94c37fe640 1 -- 172.21.15.22:0/2606899775 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac007d20 con 0x7f94b0006140 2024-03-29T04:18:52.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.992+0000 7f94c37fe640 1 -- 172.21.15.22:0/2606899775 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94ac0a9020 con 0x7f94b0006140 2024-03-29T04:18:52.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.992+0000 7f94c37fe640 1 -- 172.21.15.22:0/2606899775 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac005ae0 con 0x7f94b0006140 2024-03-29T04:18:52.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.992+0000 7f94ca83f640 1 -- 172.21.15.22:0/2606899775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b0006140 msgr2=0x7f94b0006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:52.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.992+0000 7f94ca83f640 1 --2- 172.21.15.22:0/2606899775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b0006140 0x7f94b0006a20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00c210 tx=0x7f94ac01cb80 comp rx=0 tx=0).stop 2024-03-29T04:18:52.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.992+0000 7f94c8ddc640 1 -- 172.21.15.22:0/2606899775 reap_dead start 2024-03-29T04:18:52.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.992+0000 7f94ca83f640 1 -- 172.21.15.22:0/2606899775 shutdown_connections 2024-03-29T04:18:52.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.992+0000 7f94ca83f640 1 -- 172.21.15.22:0/2606899775 >> 172.21.15.22:0/2606899775 conn(0x7f94c40cfee0 msgr2=0x7f94c40d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:52.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.992+0000 7f94ca83f640 1 -- 172.21.15.22:0/2606899775 shutdown_connections 2024-03-29T04:18:52.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.992+0000 7f94ca83f640 1 -- 172.21.15.22:0/2606899775 wait complete. 2024-03-29T04:18:52.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.992+0000 7f94ca83f640 1 Processor -- start 2024-03-29T04:18:52.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.992+0000 7f94ca83f640 1 -- start start 2024-03-29T04:18:52.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.992+0000 7f94ca83f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b0006140 0x7f94c40c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:52.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.992+0000 7f94ca83f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94c4140bc0 0x7f94c4133810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:52.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.992+0000 7f94ca83f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94c40c7670 0x7f94c40c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:52.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.992+0000 7f94ca83f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f94c4144650 con 0x7f94b0006140 2024-03-29T04:18:52.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.992+0000 7f94ca83f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f94c40b3f70 con 0x7f94c4140bc0 2024-03-29T04:18:52.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.992+0000 7f94ca83f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f94c41447d0 con 0x7f94c40c7670 2024-03-29T04:18:52.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94c3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b0006140 0x7f94c40c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:52.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94c3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b0006140 0x7f94c40c6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43940/0 (socket says 172.21.15.22:43940) 2024-03-29T04:18:52.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94c3fff640 1 -- 172.21.15.22:0/182439038 learned_addr learned my addr 172.21.15.22:0/182439038 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:52.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94bbfff640 1 --2- 172.21.15.22:0/182439038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94c4140bc0 0x7f94c4133810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:52.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94c8ddc640 1 --2- 172.21.15.22:0/182439038 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94c40c7670 0x7f94c40c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:52.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94c3fff640 1 -- 172.21.15.22:0/182439038 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94c40c7670 msgr2=0x7f94c40c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:52.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94c3fff640 1 --2- 172.21.15.22:0/182439038 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94c40c7670 0x7f94c40c7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:52.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94c3fff640 1 -- 172.21.15.22:0/182439038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94c4140bc0 msgr2=0x7f94c4133810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:52.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94c3fff640 1 --2- 172.21.15.22:0/182439038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94c4140bc0 0x7f94c4133810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:52.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94c3fff640 1 -- 172.21.15.22:0/182439038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c41413c0 con 0x7f94b0006140 2024-03-29T04:18:52.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94c8ddc640 1 --2- 172.21.15.22:0/182439038 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94c40c7670 0x7f94c40c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:52.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94c3fff640 1 --2- 172.21.15.22:0/182439038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b0006140 0x7f94c40c6910 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f94b400fb70 tx=0x7f94b400d4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:52.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94c17fa640 1 -- 172.21.15.22:0/182439038 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b400fe70 con 0x7f94b0006140 2024-03-29T04:18:52.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94c17fa640 1 -- 172.21.15.22:0/182439038 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b4019070 con 0x7f94b0006140 2024-03-29T04:18:52.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94ca83f640 1 -- 172.21.15.22:0/182439038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c40c5870 con 0x7f94b0006140 2024-03-29T04:18:52.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94c17fa640 1 -- 172.21.15.22:0/182439038 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b400dd80 con 0x7f94b0006140 2024-03-29T04:18:52.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94ca83f640 1 -- 172.21.15.22:0/182439038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c40c5e30 con 0x7f94b0006140 2024-03-29T04:18:52.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94ca83f640 1 -- 172.21.15.22:0/182439038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c4068890 con 0x7f94b0006140 2024-03-29T04:18:52.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94c17fa640 1 -- 172.21.15.22:0/182439038 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f94b4005070 con 0x7f94b0006140 2024-03-29T04:18:52.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94c17fa640 1 --2- 172.21.15.22:0/182439038 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f94a4067ff0 0x7f94a406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:52.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94c17fa640 1 -- 172.21.15.22:0/182439038 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f94b4089350 con 0x7f94b0006140 2024-03-29T04:18:52.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94bbfff640 1 --2- 172.21.15.22:0/182439038 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f94a4067ff0 0x7f94a406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:52.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:52.996+0000 7f94bbfff640 1 --2- 172.21.15.22:0/182439038 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f94a4067ff0 0x7f94a406a4b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0aa000 tx=0x7f94c40b35c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:52.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.000+0000 7f94c17fa640 1 -- 172.21.15.22:0/182439038 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f94b4059cc0 con 0x7f94b0006140 2024-03-29T04:18:53.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.220+0000 7f94ca83f640 1 -- 172.21.15.22:0/182439038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f94c40722d0 con 0x7f94b0006140 2024-03-29T04:18:53.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.224+0000 7f94c17fa640 1 -- 172.21.15.22:0/182439038 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f94b40595c0 con 0x7f94b0006140 2024-03-29T04:18:53.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.228+0000 7f94ca83f640 1 -- 172.21.15.22:0/182439038 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f94a4067ff0 msgr2=0x7f94a406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:53.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.228+0000 7f94ca83f640 1 --2- 172.21.15.22:0/182439038 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f94a4067ff0 0x7f94a406a4b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0aa000 tx=0x7f94c40b35c0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.228+0000 7f94ca83f640 1 -- 172.21.15.22:0/182439038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b0006140 msgr2=0x7f94c40c6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:53.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.228+0000 7f94ca83f640 1 --2- 172.21.15.22:0/182439038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b0006140 0x7f94c40c6910 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f94b400fb70 tx=0x7f94b400d4b0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.228+0000 7f94ca83f640 1 -- 172.21.15.22:0/182439038 shutdown_connections 2024-03-29T04:18:53.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.228+0000 7f94ca83f640 1 --2- 172.21.15.22:0/182439038 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94c40c7670 0x7f94c40c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.228+0000 7f94ca83f640 1 --2- 172.21.15.22:0/182439038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94c4140bc0 0x7f94c4133810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.228+0000 7f94ca83f640 1 --2- 172.21.15.22:0/182439038 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f94a4067ff0 0x7f94a406a4b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.228+0000 7f94ca83f640 1 --2- 172.21.15.22:0/182439038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b0006140 0x7f94c40c6910 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.228+0000 7f94ca83f640 1 -- 172.21.15.22:0/182439038 >> 172.21.15.22:0/182439038 conn(0x7f94c40cfee0 msgr2=0x7f94c4141670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:53.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.228+0000 7f94ca83f640 1 -- 172.21.15.22:0/182439038 shutdown_connections 2024-03-29T04:18:53.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.228+0000 7f94ca83f640 1 -- 172.21.15.22:0/182439038 wait complete. 2024-03-29T04:18:53.223 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-29T04:18:53.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.356+0000 7fb48d84f640 1 Processor -- start 2024-03-29T04:18:53.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.356+0000 7fb48d84f640 1 -- start start 2024-03-29T04:18:53.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.360+0000 7fb48d84f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb4880b41f0 0x7fb4880cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:53.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.360+0000 7fb48d84f640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fb48806ca50 0x7fb4880ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:53.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.360+0000 7fb48d84f640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4880ccef0 con 0x7fb48806c670 2024-03-29T04:18:53.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.360+0000 7fb48d84f640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fb4880b34c0 con 0x7fb48806ca50 2024-03-29T04:18:53.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.360+0000 7fb48d84f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb4880b3710 con 0x7fb4880b41f0 2024-03-29T04:18:53.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.360+0000 7fb4877fe640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fb48806ca50 0x7fb4880ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:53.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.360+0000 7fb4867fc640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb4880b41f0 0x7fb4880cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:53.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.360+0000 7fb4877fe640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fb48806ca50 0x7fb4880ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:35178/0 (socket says 172.21.15.22:35178) 2024-03-29T04:18:53.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.360+0000 7fb4877fe640 1 -- 172.21.15.22:0/1939501636 learned_addr learned my addr 172.21.15.22:0/1939501636 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:53.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.360+0000 7fb4877fe640 1 -- 172.21.15.22:0/1939501636 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb4880b41f0 msgr2=0x7fb4880cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:53.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.360+0000 7fb4877fe640 1 --2- 172.21.15.22:0/1939501636 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb4880b41f0 0x7fb4880cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.360+0000 7fb4877fe640 1 -- 172.21.15.22:0/1939501636 >> v1:172.21.15.151:6789/0 conn(0x7fb48806c670 legacy=0x7fb48806e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:53.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.360+0000 7fb4877fe640 1 -- 172.21.15.22:0/1939501636 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4880b39f0 con 0x7fb48806ca50 2024-03-29T04:18:53.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.364+0000 7fb4877fe640 1 --2- 172.21.15.22:0/1939501636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb48806ca50 0x7fb4880ce5b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb480009e50 tx=0x7fb48001ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1ecd42aff0a08b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:53.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.364+0000 7fb485ffb640 1 -- 172.21.15.22:0/1939501636 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4800a5720 con 0x7fb48806ca50 2024-03-29T04:18:53.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.364+0000 7fb485ffb640 1 -- 172.21.15.22:0/1939501636 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4800a58c0 con 0x7fb48806ca50 2024-03-29T04:18:53.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.364+0000 7fb485ffb640 1 -- 172.21.15.22:0/1939501636 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4800a5c90 con 0x7fb48806ca50 2024-03-29T04:18:53.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.364+0000 7fb48d84f640 1 -- 172.21.15.22:0/1939501636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb48806ca50 msgr2=0x7fb4880ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:53.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.364+0000 7fb48d84f640 1 --2- 172.21.15.22:0/1939501636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb48806ca50 0x7fb4880ce5b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb480009e50 tx=0x7fb48001ca30 comp rx=0 tx=0).stop 2024-03-29T04:18:53.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.364+0000 7fb48d84f640 1 -- 172.21.15.22:0/1939501636 shutdown_connections 2024-03-29T04:18:53.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.364+0000 7fb48d84f640 1 --2- 172.21.15.22:0/1939501636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb48806ca50 0x7fb4880ce5b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.364+0000 7fb48d84f640 1 --2- 172.21.15.22:0/1939501636 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb4880b41f0 0x7fb4880cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.364+0000 7fb48d84f640 1 -- 172.21.15.22:0/1939501636 >> 172.21.15.22:0/1939501636 conn(0x7fb4880cfee0 msgr2=0x7fb4880d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:53.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.364+0000 7fb48d84f640 1 -- 172.21.15.22:0/1939501636 shutdown_connections 2024-03-29T04:18:53.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.364+0000 7fb48d84f640 1 -- 172.21.15.22:0/1939501636 wait complete. 2024-03-29T04:18:53.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.364+0000 7fb48d84f640 1 Processor -- start 2024-03-29T04:18:53.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.364+0000 7fb48d84f640 1 -- start start 2024-03-29T04:18:53.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.364+0000 7fb48d84f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb48806c670 0x7fb48815bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:53.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.364+0000 7fb48d84f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb48806ca50 0x7fb4881f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:53.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.364+0000 7fb48d84f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4880b41f0 0x7fb48813a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:53.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.364+0000 7fb48d84f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb488169070 con 0x7fb4880b41f0 2024-03-29T04:18:53.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.364+0000 7fb48d84f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb488168ef0 con 0x7fb48806ca50 2024-03-29T04:18:53.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.364+0000 7fb48d84f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb4881691f0 con 0x7fb48806c670 2024-03-29T04:18:53.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb4877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4880b41f0 0x7fb48813a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:53.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb4877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4880b41f0 0x7fb48813a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43946/0 (socket says 172.21.15.22:43946) 2024-03-29T04:18:53.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb4877fe640 1 -- 172.21.15.22:0/72890157 learned_addr learned my addr 172.21.15.22:0/72890157 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:53.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb4867fc640 1 --2- 172.21.15.22:0/72890157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb48806ca50 0x7fb4881f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:53.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb486ffd640 1 --2- 172.21.15.22:0/72890157 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb48806c670 0x7fb48815bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:53.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb4877fe640 1 -- 172.21.15.22:0/72890157 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb48806c670 msgr2=0x7fb48815bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:53.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb4877fe640 1 --2- 172.21.15.22:0/72890157 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb48806c670 0x7fb48815bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb4877fe640 1 -- 172.21.15.22:0/72890157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb48806ca50 msgr2=0x7fb4881f7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:53.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb4877fe640 1 --2- 172.21.15.22:0/72890157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb48806ca50 0x7fb4881f7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb4877fe640 1 -- 172.21.15.22:0/72890157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4881f8730 con 0x7fb4880b41f0 2024-03-29T04:18:53.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb4877fe640 1 --2- 172.21.15.22:0/72890157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4880b41f0 0x7fb48813a5d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb48001a660 tx=0x7fb4800197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:53.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb46ffff640 1 -- 172.21.15.22:0/72890157 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4800bc020 con 0x7fb4880b41f0 2024-03-29T04:18:53.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb46ffff640 1 -- 172.21.15.22:0/72890157 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4800b8670 con 0x7fb4880b41f0 2024-03-29T04:18:53.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb48d84f640 1 -- 172.21.15.22:0/72890157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4881f8a10 con 0x7fb4880b41f0 2024-03-29T04:18:53.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb46ffff640 1 -- 172.21.15.22:0/72890157 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4800b8a60 con 0x7fb4880b41f0 2024-03-29T04:18:53.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb48d84f640 1 -- 172.21.15.22:0/72890157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb488132530 con 0x7fb4880b41f0 2024-03-29T04:18:53.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb48d84f640 1 -- 172.21.15.22:0/72890157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb488068890 con 0x7fb4880b41f0 2024-03-29T04:18:53.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb46ffff640 1 -- 172.21.15.22:0/72890157 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb4800bf020 con 0x7fb4880b41f0 2024-03-29T04:18:53.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb46ffff640 1 --2- 172.21.15.22:0/72890157 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb474067fc0 0x7fb47406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:53.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb46ffff640 1 -- 172.21.15.22:0/72890157 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fb480121200 con 0x7fb4880b41f0 2024-03-29T04:18:53.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb486ffd640 1 --2- 172.21.15.22:0/72890157 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb474067fc0 0x7fb47406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:53.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.368+0000 7fb486ffd640 1 --2- 172.21.15.22:0/72890157 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb474067fc0 0x7fb47406a480 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb478007fd0 tx=0x7fb478009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:53.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.372+0000 7fb46ffff640 1 -- 172.21.15.22:0/72890157 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fb4800c6030 con 0x7fb4880b41f0 2024-03-29T04:18:53.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.588+0000 7fb48d84f640 1 -- 172.21.15.22:0/72890157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fb488132700 con 0x7fb4880b41f0 2024-03-29T04:18:53.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.592+0000 7fb46ffff640 1 -- 172.21.15.22:0/72890157 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fb480016620 con 0x7fb4880b41f0 2024-03-29T04:18:53.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.596+0000 7fb48d84f640 1 -- 172.21.15.22:0/72890157 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb474067fc0 msgr2=0x7fb47406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:53.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.596+0000 7fb48d84f640 1 --2- 172.21.15.22:0/72890157 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb474067fc0 0x7fb47406a480 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb478007fd0 tx=0x7fb478009040 comp rx=0 tx=0).stop 2024-03-29T04:18:53.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.596+0000 7fb48d84f640 1 -- 172.21.15.22:0/72890157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4880b41f0 msgr2=0x7fb48813a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:53.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.596+0000 7fb48d84f640 1 --2- 172.21.15.22:0/72890157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4880b41f0 0x7fb48813a5d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb48001a660 tx=0x7fb4800197d0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.596+0000 7fb48d84f640 1 -- 172.21.15.22:0/72890157 shutdown_connections 2024-03-29T04:18:53.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.596+0000 7fb48d84f640 1 --2- 172.21.15.22:0/72890157 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb474067fc0 0x7fb47406a480 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.596+0000 7fb48d84f640 1 --2- 172.21.15.22:0/72890157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4880b41f0 0x7fb48813a5d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.596+0000 7fb48d84f640 1 --2- 172.21.15.22:0/72890157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb48806ca50 0x7fb4881f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.596+0000 7fb48d84f640 1 --2- 172.21.15.22:0/72890157 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb48806c670 0x7fb48815bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.596+0000 7fb48d84f640 1 -- 172.21.15.22:0/72890157 >> 172.21.15.22:0/72890157 conn(0x7fb4880cfee0 msgr2=0x7fb4880b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:53.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.596+0000 7fb48d84f640 1 -- 172.21.15.22:0/72890157 shutdown_connections 2024-03-29T04:18:53.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.596+0000 7fb48d84f640 1 -- 172.21.15.22:0/72890157 wait complete. 2024-03-29T04:18:53.591 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-29T04:18:53.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.724+0000 7fec083aa640 1 Processor -- start 2024-03-29T04:18:53.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.724+0000 7fec083aa640 1 -- start start 2024-03-29T04:18:53.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.724+0000 7fec083aa640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec000b5540 0x7fec0006c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:53.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.724+0000 7fec083aa640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fec0006ccb0 0x7fec00164bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:53.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.724+0000 7fec083aa640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec0006d430 con 0x7fec0006f9e0 2024-03-29T04:18:53.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.724+0000 7fec083aa640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fec00152550 con 0x7fec0006ccb0 2024-03-29T04:18:53.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.724+0000 7fec083aa640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fec0006d710 con 0x7fec000b5540 2024-03-29T04:18:53.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.724+0000 7fec05945640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec000b5540 0x7fec0006c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:53.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.724+0000 7fec06947640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fec0006ccb0 0x7fec00164bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:53.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.724+0000 7fec05945640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec000b5540 0x7fec0006c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3301/0 says I am v2:172.21.15.22:56222/0 (socket says 172.21.15.22:56222) 2024-03-29T04:18:53.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.724+0000 7fec05945640 1 -- 172.21.15.22:0/1690762387 learned_addr learned my addr 172.21.15.22:0/1690762387 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:53.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.724+0000 7fec05945640 1 -- 172.21.15.22:0/1690762387 >> v2:172.21.15.151:3300/0 conn(0x7fec0006ccb0 msgr2=0x7fec00164bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:53.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.724+0000 7fec05945640 1 --2- 172.21.15.22:0/1690762387 >> v2:172.21.15.151:3300/0 conn(0x7fec0006ccb0 0x7fec00164bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.724+0000 7fec05945640 1 -- 172.21.15.22:0/1690762387 >> v1:172.21.15.151:6789/0 conn(0x7fec0006f9e0 legacy=0x7fec0006fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:53.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.724+0000 7fec05945640 1 -- 172.21.15.22:0/1690762387 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0006d9f0 con 0x7fec000b5540 2024-03-29T04:18:53.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.724+0000 7fec05144640 1 -- 172.21.15.22:0/1690762387 <== mon.1 v1:172.21.15.151:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 753089694 0 0) 0x7fec0006d430 con 0x7fec0006f9e0 2024-03-29T04:18:53.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.724+0000 7fec06947640 1 --2- 172.21.15.22:0/1690762387 >> v2:172.21.15.151:3300/0 conn(0x7fec0006ccb0 0x7fec00164bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:18:53.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.728+0000 7fec05945640 1 --2- 172.21.15.22:0/1690762387 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec000b5540 0x7fec0006c770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7febf0009870 tx=0x7febf001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef696dfaa7c4c19f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:53.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.728+0000 7fec05144640 1 -- 172.21.15.22:0/1690762387 <== mon.2 v2:172.21.15.151:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf00a95f0 con 0x7fec000b5540 2024-03-29T04:18:53.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.728+0000 7fec05144640 1 -- 172.21.15.22:0/1690762387 <== mon.2 v2:172.21.15.151:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf00a9790 con 0x7fec000b5540 2024-03-29T04:18:53.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.728+0000 7fec05144640 1 -- 172.21.15.22:0/1690762387 <== mon.2 v2:172.21.15.151:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf00a9b60 con 0x7fec000b5540 2024-03-29T04:18:53.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.728+0000 7fec083aa640 1 -- 172.21.15.22:0/1690762387 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec000b5540 msgr2=0x7fec0006c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:53.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.728+0000 7fec083aa640 1 --2- 172.21.15.22:0/1690762387 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec000b5540 0x7fec0006c770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7febf0009870 tx=0x7febf001c530 comp rx=0 tx=0).stop 2024-03-29T04:18:53.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.728+0000 7fec083aa640 1 -- 172.21.15.22:0/1690762387 shutdown_connections 2024-03-29T04:18:53.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.728+0000 7fec083aa640 1 --2- 172.21.15.22:0/1690762387 >> v2:172.21.15.151:3300/0 conn(0x7fec0006ccb0 0x7fec00164bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.728+0000 7fec083aa640 1 --2- 172.21.15.22:0/1690762387 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec000b5540 0x7fec0006c770 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.728+0000 7fec083aa640 1 -- 172.21.15.22:0/1690762387 >> 172.21.15.22:0/1690762387 conn(0x7fec0006e410 msgr2=0x7fec0006e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:53.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.728+0000 7fec083aa640 1 -- 172.21.15.22:0/1690762387 shutdown_connections 2024-03-29T04:18:53.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.728+0000 7fec083aa640 1 -- 172.21.15.22:0/1690762387 wait complete. 2024-03-29T04:18:53.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.728+0000 7fec083aa640 1 Processor -- start 2024-03-29T04:18:53.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.728+0000 7fec083aa640 1 -- start start 2024-03-29T04:18:53.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec083aa640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec0006ccb0 0x7fec000cf760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:53.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec083aa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec0006f9e0 0x7fec0014bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:53.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec083aa640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec000b5540 0x7fec00151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:53.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec083aa640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fec0015c460 con 0x7fec0006f9e0 2024-03-29T04:18:53.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec083aa640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fec00167950 con 0x7fec000b5540 2024-03-29T04:18:53.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec083aa640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fec000cfca0 con 0x7fec0006ccb0 2024-03-29T04:18:53.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec05945640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec0006f9e0 0x7fec0014bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:53.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec05945640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec0006f9e0 0x7fec0014bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43948/0 (socket says 172.21.15.22:43948) 2024-03-29T04:18:53.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec05945640 1 -- 172.21.15.22:0/2819698160 learned_addr learned my addr 172.21.15.22:0/2819698160 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:53.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec06146640 1 --2- 172.21.15.22:0/2819698160 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec0006ccb0 0x7fec000cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:53.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec06947640 1 --2- 172.21.15.22:0/2819698160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec000b5540 0x7fec00151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:53.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec05945640 1 -- 172.21.15.22:0/2819698160 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec0006ccb0 msgr2=0x7fec000cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:53.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec05945640 1 --2- 172.21.15.22:0/2819698160 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec0006ccb0 0x7fec000cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec05945640 1 -- 172.21.15.22:0/2819698160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec000b5540 msgr2=0x7fec00151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:53.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec05945640 1 --2- 172.21.15.22:0/2819698160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec000b5540 0x7fec00151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec05945640 1 -- 172.21.15.22:0/2819698160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec001f4bb0 con 0x7fec0006f9e0 2024-03-29T04:18:53.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec06146640 1 --2- 172.21.15.22:0/2819698160 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec0006ccb0 0x7fec000cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:53.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec05945640 1 --2- 172.21.15.22:0/2819698160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec0006f9e0 0x7fec0014bba0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7febf001a160 tx=0x7febf00192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:53.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7febf6ffd640 1 -- 172.21.15.22:0/2819698160 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0004220 con 0x7fec0006f9e0 2024-03-29T04:18:53.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7febf6ffd640 1 -- 172.21.15.22:0/2819698160 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf00043c0 con 0x7fec0006f9e0 2024-03-29T04:18:53.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec083aa640 1 -- 172.21.15.22:0/2819698160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec000cfed0 con 0x7fec0006f9e0 2024-03-29T04:18:53.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7febf6ffd640 1 -- 172.21.15.22:0/2819698160 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0003820 con 0x7fec0006f9e0 2024-03-29T04:18:53.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec083aa640 1 -- 172.21.15.22:0/2819698160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec000d0490 con 0x7fec0006f9e0 2024-03-29T04:18:53.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec083aa640 1 -- 172.21.15.22:0/2819698160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec001f4d80 con 0x7fec0006f9e0 2024-03-29T04:18:53.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7febf6ffd640 1 -- 172.21.15.22:0/2819698160 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7febf00039c0 con 0x7fec0006f9e0 2024-03-29T04:18:53.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7febf6ffd640 1 --2- 172.21.15.22:0/2819698160 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7febe4067fc0 0x7febe406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:53.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7febf6ffd640 1 -- 172.21.15.22:0/2819698160 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7febf0120600 con 0x7fec0006f9e0 2024-03-29T04:18:53.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.732+0000 7fec06146640 1 --2- 172.21.15.22:0/2819698160 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7febe4067fc0 0x7febe406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:53.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.736+0000 7fec06146640 1 --2- 172.21.15.22:0/2819698160 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7febe4067fc0 0x7febe406a480 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7febfc007fd0 tx=0x7febfc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:53.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.736+0000 7febf6ffd640 1 -- 172.21.15.22:0/2819698160 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7febf00c0060 con 0x7fec0006f9e0 2024-03-29T04:18:53.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.960+0000 7fec083aa640 1 -- 172.21.15.22:0/2819698160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fec00068890 con 0x7fec0006f9e0 2024-03-29T04:18:53.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.964+0000 7febf6ffd640 1 -- 172.21.15.22:0/2819698160 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7febf00b7070 con 0x7fec0006f9e0 2024-03-29T04:18:53.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.964+0000 7fec083aa640 1 -- 172.21.15.22:0/2819698160 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7febe4067fc0 msgr2=0x7febe406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:53.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.964+0000 7fec083aa640 1 --2- 172.21.15.22:0/2819698160 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7febe4067fc0 0x7febe406a480 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7febfc007fd0 tx=0x7febfc009040 comp rx=0 tx=0).stop 2024-03-29T04:18:53.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.964+0000 7fec083aa640 1 -- 172.21.15.22:0/2819698160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec0006f9e0 msgr2=0x7fec0014bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:53.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.964+0000 7fec083aa640 1 --2- 172.21.15.22:0/2819698160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec0006f9e0 0x7fec0014bba0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7febf001a160 tx=0x7febf00192d0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.964+0000 7fec083aa640 1 -- 172.21.15.22:0/2819698160 shutdown_connections 2024-03-29T04:18:53.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.964+0000 7fec083aa640 1 --2- 172.21.15.22:0/2819698160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec000b5540 0x7fec00151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.964+0000 7fec083aa640 1 --2- 172.21.15.22:0/2819698160 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7febe4067fc0 0x7febe406a480 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.964+0000 7fec083aa640 1 --2- 172.21.15.22:0/2819698160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec0006f9e0 0x7fec0014bba0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.964+0000 7fec083aa640 1 --2- 172.21.15.22:0/2819698160 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fec0006ccb0 0x7fec000cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:53.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.964+0000 7fec083aa640 1 -- 172.21.15.22:0/2819698160 >> 172.21.15.22:0/2819698160 conn(0x7fec0006e410 msgr2=0x7fec000b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:53.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.964+0000 7fec083aa640 1 -- 172.21.15.22:0/2819698160 shutdown_connections 2024-03-29T04:18:53.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:53.964+0000 7fec083aa640 1 -- 172.21.15.22:0/2819698160 wait complete. 2024-03-29T04:18:53.963 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-29T04:18:54.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.104+0000 7fc24a099640 1 Processor -- start 2024-03-29T04:18:54.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.104+0000 7fc24a099640 1 -- start start 2024-03-29T04:18:54.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.104+0000 7fc24a099640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc24415e600 0x7fc2440b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:54.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.104+0000 7fc24a099640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc2440d1100 0x7fc2440d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:54.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.104+0000 7fc24a099640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc2440b8160 0x7fc2440b85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:54.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.104+0000 7fc24a099640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fc2440b7450 con 0x7fc2440b8160 2024-03-29T04:18:54.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.104+0000 7fc24a099640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fc2440d1aa0 con 0x7fc2440d1100 2024-03-29T04:18:54.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.104+0000 7fc24a099640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc2440d4140 con 0x7fc24415e600 2024-03-29T04:18:54.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.104+0000 7fc243fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc2440b8160 0x7fc2440b85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.104+0000 7fc243fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc2440b8160 0x7fc2440b85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43958/0 (socket says 172.21.15.22:43958) 2024-03-29T04:18:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.104+0000 7fc243fff640 1 -- 172.21.15.22:0/1035388743 learned_addr learned my addr 172.21.15.22:0/1035388743 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.104+0000 7fc2437fe640 1 --2- 172.21.15.22:0/1035388743 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc24415e600 0x7fc2440b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.104+0000 7fc242ffd640 1 --2- 172.21.15.22:0/1035388743 >> v2:172.21.15.151:3300/0 conn(0x7fc2440d1100 0x7fc2440d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.104+0000 7fc243fff640 1 -- 172.21.15.22:0/1035388743 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc24415e600 msgr2=0x7fc2440b9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.104+0000 7fc243fff640 1 --2- 172.21.15.22:0/1035388743 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc24415e600 0x7fc2440b9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.104+0000 7fc243fff640 1 -- 172.21.15.22:0/1035388743 >> v2:172.21.15.151:3300/0 conn(0x7fc2440d1100 msgr2=0x7fc2440d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.104+0000 7fc243fff640 1 --2- 172.21.15.22:0/1035388743 >> v2:172.21.15.151:3300/0 conn(0x7fc2440d1100 0x7fc2440d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.104+0000 7fc243fff640 1 -- 172.21.15.22:0/1035388743 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2440d3e60 con 0x7fc2440b8160 2024-03-29T04:18:54.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.108+0000 7fc243fff640 1 --2- 172.21.15.22:0/1035388743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2440b8160 0x7fc2440b85f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc2440d1c60 tx=0x7fc23c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=786fc76003578588 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:54.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.108+0000 7fc2427fc640 1 -- 172.21.15.22:0/1035388743 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0a5840 con 0x7fc2440b8160 2024-03-29T04:18:54.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.108+0000 7fc2427fc640 1 -- 172.21.15.22:0/1035388743 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc23c0a59e0 con 0x7fc2440b8160 2024-03-29T04:18:54.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.108+0000 7fc2427fc640 1 -- 172.21.15.22:0/1035388743 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0a5db0 con 0x7fc2440b8160 2024-03-29T04:18:54.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.108+0000 7fc24a099640 1 -- 172.21.15.22:0/1035388743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2440b8160 msgr2=0x7fc2440b85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:54.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.108+0000 7fc24a099640 1 --2- 172.21.15.22:0/1035388743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2440b8160 0x7fc2440b85f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc2440d1c60 tx=0x7fc23c01c9c0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.108+0000 7fc24a099640 1 -- 172.21.15.22:0/1035388743 shutdown_connections 2024-03-29T04:18:54.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.108+0000 7fc24a099640 1 --2- 172.21.15.22:0/1035388743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2440b8160 0x7fc2440b85f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.108+0000 7fc24a099640 1 --2- 172.21.15.22:0/1035388743 >> v2:172.21.15.151:3300/0 conn(0x7fc2440d1100 0x7fc2440d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.108+0000 7fc24a099640 1 --2- 172.21.15.22:0/1035388743 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc24415e600 0x7fc2440b9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.108+0000 7fc24a099640 1 -- 172.21.15.22:0/1035388743 >> 172.21.15.22:0/1035388743 conn(0x7fc2440b6230 msgr2=0x7fc2440b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:54.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.108+0000 7fc24a099640 1 -- 172.21.15.22:0/1035388743 shutdown_connections 2024-03-29T04:18:54.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.108+0000 7fc24a099640 1 -- 172.21.15.22:0/1035388743 wait complete. 2024-03-29T04:18:54.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.108+0000 7fc24a099640 1 Processor -- start 2024-03-29T04:18:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.108+0000 7fc24a099640 1 -- start start 2024-03-29T04:18:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.108+0000 7fc24a099640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc2440b8160 0x7fc24414bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.108+0000 7fc24a099640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2440d1100 0x7fc24414c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.108+0000 7fc24a099640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc24415e600 0x7fc2440ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.108+0000 7fc24a099640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc2440d2780 con 0x7fc24415e600 2024-03-29T04:18:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.108+0000 7fc24a099640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc2440d2600 con 0x7fc2440d1100 2024-03-29T04:18:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.108+0000 7fc24a099640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc2440d2900 con 0x7fc2440b8160 2024-03-29T04:18:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc243fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc24415e600 0x7fc2440ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc243fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc24415e600 0x7fc2440ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43970/0 (socket says 172.21.15.22:43970) 2024-03-29T04:18:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc243fff640 1 -- 172.21.15.22:0/1411899109 learned_addr learned my addr 172.21.15.22:0/1411899109 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc242ffd640 1 --2- 172.21.15.22:0/1411899109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2440d1100 0x7fc24414c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:54.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc2437fe640 1 --2- 172.21.15.22:0/1411899109 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc2440b8160 0x7fc24414bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:54.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc243fff640 1 -- 172.21.15.22:0/1411899109 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc2440b8160 msgr2=0x7fc24414bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:54.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc243fff640 1 --2- 172.21.15.22:0/1411899109 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc2440b8160 0x7fc24414bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc243fff640 1 -- 172.21.15.22:0/1411899109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2440d1100 msgr2=0x7fc24414c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:54.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc243fff640 1 --2- 172.21.15.22:0/1411899109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2440d1100 0x7fc24414c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc243fff640 1 -- 172.21.15.22:0/1411899109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc24415aab0 con 0x7fc24415e600 2024-03-29T04:18:54.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc242ffd640 1 --2- 172.21.15.22:0/1411899109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2440d1100 0x7fc24414c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:54.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc243fff640 1 --2- 172.21.15.22:0/1411899109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc24415e600 0x7fc2440ccd30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc23c01b070 tx=0x7fc23c019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:54.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc22bfff640 1 -- 172.21.15.22:0/1411899109 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0bc020 con 0x7fc24415e600 2024-03-29T04:18:54.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc22bfff640 1 -- 172.21.15.22:0/1411899109 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc23c0b8670 con 0x7fc24415e600 2024-03-29T04:18:54.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc24a099640 1 -- 172.21.15.22:0/1411899109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc24415ad90 con 0x7fc24415e600 2024-03-29T04:18:54.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc22bfff640 1 -- 172.21.15.22:0/1411899109 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0b8a40 con 0x7fc24415e600 2024-03-29T04:18:54.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc24a099640 1 -- 172.21.15.22:0/1411899109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc244152aa0 con 0x7fc24415e600 2024-03-29T04:18:54.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc24a099640 1 -- 172.21.15.22:0/1411899109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc244068890 con 0x7fc24415e600 2024-03-29T04:18:54.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc22bfff640 1 -- 172.21.15.22:0/1411899109 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fc23c0bf020 con 0x7fc24415e600 2024-03-29T04:18:54.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc22bfff640 1 --2- 172.21.15.22:0/1411899109 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc230067fc0 0x7fc23006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:54.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc22bfff640 1 -- 172.21.15.22:0/1411899109 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fc23c1219c0 con 0x7fc24415e600 2024-03-29T04:18:54.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.112+0000 7fc2437fe640 1 --2- 172.21.15.22:0/1411899109 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc230067fc0 0x7fc23006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:54.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.116+0000 7fc2437fe640 1 --2- 172.21.15.22:0/1411899109 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc230067fc0 0x7fc23006a480 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc2340096f0 tx=0x7fc234008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:54.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.116+0000 7fc22bfff640 1 -- 172.21.15.22:0/1411899109 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fc23c0c6030 con 0x7fc24415e600 2024-03-29T04:18:54.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.336+0000 7fc24a099640 1 -- 172.21.15.22:0/1411899109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fc24414cdb0 con 0x7fc24415e600 2024-03-29T04:18:54.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.340+0000 7fc22bfff640 1 -- 172.21.15.22:0/1411899109 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fc23c0165e0 con 0x7fc24415e600 2024-03-29T04:18:54.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.344+0000 7fc24a099640 1 -- 172.21.15.22:0/1411899109 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc230067fc0 msgr2=0x7fc23006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:54.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.344+0000 7fc24a099640 1 --2- 172.21.15.22:0/1411899109 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc230067fc0 0x7fc23006a480 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc2340096f0 tx=0x7fc234008040 comp rx=0 tx=0).stop 2024-03-29T04:18:54.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.344+0000 7fc24a099640 1 -- 172.21.15.22:0/1411899109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc24415e600 msgr2=0x7fc2440ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:54.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.344+0000 7fc24a099640 1 --2- 172.21.15.22:0/1411899109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc24415e600 0x7fc2440ccd30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc23c01b070 tx=0x7fc23c019760 comp rx=0 tx=0).stop 2024-03-29T04:18:54.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.344+0000 7fc24a099640 1 -- 172.21.15.22:0/1411899109 shutdown_connections 2024-03-29T04:18:54.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.344+0000 7fc24a099640 1 --2- 172.21.15.22:0/1411899109 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc230067fc0 0x7fc23006a480 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.344+0000 7fc24a099640 1 --2- 172.21.15.22:0/1411899109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc24415e600 0x7fc2440ccd30 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.344+0000 7fc24a099640 1 --2- 172.21.15.22:0/1411899109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2440d1100 0x7fc24414c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.344+0000 7fc24a099640 1 --2- 172.21.15.22:0/1411899109 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc2440b8160 0x7fc24414bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.344+0000 7fc24a099640 1 -- 172.21.15.22:0/1411899109 >> 172.21.15.22:0/1411899109 conn(0x7fc2440b6230 msgr2=0x7fc2440cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:54.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.344+0000 7fc24a099640 1 -- 172.21.15.22:0/1411899109 shutdown_connections 2024-03-29T04:18:54.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.344+0000 7fc24a099640 1 -- 172.21.15.22:0/1411899109 wait complete. 2024-03-29T04:18:54.341 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-29T04:18:54.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.488+0000 7f587b870640 1 Processor -- start 2024-03-29T04:18:54.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.488+0000 7f587b870640 1 -- start start 2024-03-29T04:18:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.488+0000 7f587b870640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f587415e600 0x7f58740b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.488+0000 7f587b870640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f58740d1560 0x7f58740d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.488+0000 7f587b870640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58740d3e60 con 0x7f58740d1100 2024-03-29T04:18:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.488+0000 7f587b870640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f58740b7450 con 0x7f587415e600 2024-03-29T04:18:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.488+0000 7f587b870640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f58740d4140 con 0x7f58740d1560 2024-03-29T04:18:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.488+0000 7f5878e0b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f58740d1100 0x7f58740b8160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42684/0 (socket says 172.21.15.22:42684) 2024-03-29T04:18:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.488+0000 7f587960c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f587415e600 0x7f58740b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:54.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.488+0000 7f5878e0b640 1 -- 172.21.15.22:0/3640636315 learned_addr learned my addr 172.21.15.22:0/3640636315 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:54.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.488+0000 7f5879e0d640 1 --2- 172.21.15.22:0/3640636315 >> v2:172.21.15.151:3300/0 conn(0x7f58740d1560 0x7f58740d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:54.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.488+0000 7f586bfff640 1 -- 172.21.15.22:0/3640636315 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3379896898 0 0) 0x7f58740d3e60 con 0x7f58740d1100 2024-03-29T04:18:54.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.488+0000 7f587960c640 1 -- 172.21.15.22:0/3640636315 >> v2:172.21.15.151:3300/0 conn(0x7f58740d1560 msgr2=0x7f58740d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:54.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.488+0000 7f587960c640 1 --2- 172.21.15.22:0/3640636315 >> v2:172.21.15.151:3300/0 conn(0x7f58740d1560 0x7f58740d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.488+0000 7f587960c640 1 -- 172.21.15.22:0/3640636315 >> v1:172.21.15.22:6789/0 conn(0x7f58740d1100 legacy=0x7f58740b8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:54.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.488+0000 7f587960c640 1 -- 172.21.15.22:0/3640636315 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58740b7680 con 0x7f587415e600 2024-03-29T04:18:54.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.492+0000 7f587960c640 1 --2- 172.21.15.22:0/3640636315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f587415e600 0x7f58740b9f00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f58740d1c90 tx=0x7f587001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b2e11462e2e763 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:54.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.492+0000 7f586bfff640 1 -- 172.21.15.22:0/3640636315 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58700b2070 con 0x7f587415e600 2024-03-29T04:18:54.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.492+0000 7f586bfff640 1 -- 172.21.15.22:0/3640636315 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58700a59c0 con 0x7f587415e600 2024-03-29T04:18:54.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.492+0000 7f586bfff640 1 -- 172.21.15.22:0/3640636315 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58700a5d90 con 0x7f587415e600 2024-03-29T04:18:54.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.492+0000 7f587b870640 1 -- 172.21.15.22:0/3640636315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f587415e600 msgr2=0x7f58740b9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:54.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.492+0000 7f587b870640 1 --2- 172.21.15.22:0/3640636315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f587415e600 0x7f58740b9f00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f58740d1c90 tx=0x7f587001c900 comp rx=0 tx=0).stop 2024-03-29T04:18:54.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.492+0000 7f587b870640 1 -- 172.21.15.22:0/3640636315 shutdown_connections 2024-03-29T04:18:54.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.492+0000 7f587b870640 1 --2- 172.21.15.22:0/3640636315 >> v2:172.21.15.151:3300/0 conn(0x7f58740d1560 0x7f58740d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.492+0000 7f587b870640 1 --2- 172.21.15.22:0/3640636315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f587415e600 0x7f58740b9f00 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.492+0000 7f587b870640 1 -- 172.21.15.22:0/3640636315 >> 172.21.15.22:0/3640636315 conn(0x7f58740b6230 msgr2=0x7f58740b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:54.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.492+0000 7f587b870640 1 -- 172.21.15.22:0/3640636315 shutdown_connections 2024-03-29T04:18:54.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.492+0000 7f587b870640 1 -- 172.21.15.22:0/3640636315 wait complete. 2024-03-29T04:18:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.492+0000 7f587b870640 1 Processor -- start 2024-03-29T04:18:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.492+0000 7f587b870640 1 -- start start 2024-03-29T04:18:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f587b870640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58740d1100 0x7f58740c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f587b870640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58740d1560 0x7f58740c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f587b870640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f587415e600 0x7f58741548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f587b870640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f58740b9080 con 0x7f58740d1560 2024-03-29T04:18:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f587b870640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f58740d1970 con 0x7f58740d1100 2024-03-29T04:18:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f587b870640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f58740b9200 con 0x7f587415e600 2024-03-29T04:18:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f5878e0b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58740d1560 0x7f58740c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f5878e0b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58740d1560 0x7f58740c4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43984/0 (socket says 172.21.15.22:43984) 2024-03-29T04:18:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f5878e0b640 1 -- 172.21.15.22:0/3175816800 learned_addr learned my addr 172.21.15.22:0/3175816800 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f587960c640 1 --2- 172.21.15.22:0/3175816800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58740d1100 0x7f58740c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f5878e0b640 1 -- 172.21.15.22:0/3175816800 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f587415e600 msgr2=0x7f58741548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f5878e0b640 1 --2- 172.21.15.22:0/3175816800 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f587415e600 0x7f58741548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f5878e0b640 1 -- 172.21.15.22:0/3175816800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58740d1100 msgr2=0x7f58740c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f5878e0b640 1 --2- 172.21.15.22:0/3175816800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58740d1100 0x7f58740c71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f5878e0b640 1 -- 172.21.15.22:0/3175816800 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5874153840 con 0x7f58740d1560 2024-03-29T04:18:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f5878e0b640 1 --2- 172.21.15.22:0/3175816800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58740d1560 0x7f58740c4900 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f586400c4a0 tx=0x7f586400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f5869ffb640 1 -- 172.21.15.22:0/3175816800 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864018070 con 0x7f58740d1560 2024-03-29T04:18:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f5869ffb640 1 -- 172.21.15.22:0/3175816800 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58640074e0 con 0x7f58740d1560 2024-03-29T04:18:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f587b870640 1 -- 172.21.15.22:0/3175816800 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5874153b20 con 0x7f58740d1560 2024-03-29T04:18:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f5869ffb640 1 -- 172.21.15.22:0/3175816800 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864004a30 con 0x7f58740d1560 2024-03-29T04:18:54.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f587b870640 1 -- 172.21.15.22:0/3175816800 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58741540e0 con 0x7f58740d1560 2024-03-29T04:18:54.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f587b870640 1 -- 172.21.15.22:0/3175816800 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5874068890 con 0x7f58740d1560 2024-03-29T04:18:54.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f5869ffb640 1 -- 172.21.15.22:0/3175816800 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f586401b070 con 0x7f58740d1560 2024-03-29T04:18:54.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f5869ffb640 1 --2- 172.21.15.22:0/3175816800 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5858067fc0 0x7f585806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:54.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f5869ffb640 1 -- 172.21.15.22:0/3175816800 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f5864085670 con 0x7f58740d1560 2024-03-29T04:18:54.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.496+0000 7f587960c640 1 --2- 172.21.15.22:0/3175816800 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5858067fc0 0x7f585806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:54.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.500+0000 7f587960c640 1 --2- 172.21.15.22:0/3175816800 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5858067fc0 0x7f585806a480 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f58700af000 tx=0x7f58700a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:54.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.500+0000 7f5869ffb640 1 -- 172.21.15.22:0/3175816800 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f5864055fe0 con 0x7f58740d1560 2024-03-29T04:18:54.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.728+0000 7f587b870640 1 -- 172.21.15.22:0/3175816800 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f5874152fa0 con 0x7f58740d1560 2024-03-29T04:18:54.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.732+0000 7f5869ffb640 1 -- 172.21.15.22:0/3175816800 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7f586405ae20 con 0x7f58740d1560 2024-03-29T04:18:54.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.732+0000 7f587b870640 1 -- 172.21.15.22:0/3175816800 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5858067fc0 msgr2=0x7f585806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:54.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.732+0000 7f587b870640 1 --2- 172.21.15.22:0/3175816800 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5858067fc0 0x7f585806a480 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f58700af000 tx=0x7f58700a7070 comp rx=0 tx=0).stop 2024-03-29T04:18:54.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.732+0000 7f587b870640 1 -- 172.21.15.22:0/3175816800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58740d1560 msgr2=0x7f58740c4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:54.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.732+0000 7f587b870640 1 --2- 172.21.15.22:0/3175816800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58740d1560 0x7f58740c4900 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f586400c4a0 tx=0x7f586400c960 comp rx=0 tx=0).stop 2024-03-29T04:18:54.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.732+0000 7f587b870640 1 -- 172.21.15.22:0/3175816800 shutdown_connections 2024-03-29T04:18:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.732+0000 7f587b870640 1 --2- 172.21.15.22:0/3175816800 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f587415e600 0x7f58741548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.732+0000 7f587b870640 1 --2- 172.21.15.22:0/3175816800 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5858067fc0 0x7f585806a480 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.732+0000 7f587b870640 1 --2- 172.21.15.22:0/3175816800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58740d1560 0x7f58740c4900 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.732+0000 7f587b870640 1 --2- 172.21.15.22:0/3175816800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58740d1100 0x7f58740c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.732+0000 7f587b870640 1 -- 172.21.15.22:0/3175816800 >> 172.21.15.22:0/3175816800 conn(0x7f58740b6230 msgr2=0x7f58740cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.732+0000 7f587b870640 1 -- 172.21.15.22:0/3175816800 shutdown_connections 2024-03-29T04:18:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.732+0000 7f587b870640 1 -- 172.21.15.22:0/3175816800 wait complete. 2024-03-29T04:18:54.731 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-29T04:18:54.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.872+0000 7f9dfd8f1640 1 Processor -- start 2024-03-29T04:18:54.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.872+0000 7f9dfd8f1640 1 -- start start 2024-03-29T04:18:54.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.872+0000 7f9dfd8f1640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9df815a840 0x7f9df815e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:54.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.872+0000 7f9dfd8f1640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9df80b3f00 0x7f9df806e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:54.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.872+0000 7f9dfd8f1640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9df8072cb0 con 0x7f9df815e940 2024-03-29T04:18:54.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.872+0000 7f9dfd8f1640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f9df815eda0 con 0x7f9df80b3f00 2024-03-29T04:18:54.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.872+0000 7f9dfd8f1640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9df8072f90 con 0x7f9df815a840 2024-03-29T04:18:54.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.872+0000 7f9df77fe640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9df80b3f00 0x7f9df806e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:54.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.872+0000 7f9df6ffd640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9df815a840 0x7f9df815e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:54.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.872+0000 7f9df77fe640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9df80b3f00 0x7f9df806e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:35248/0 (socket says 172.21.15.22:35248) 2024-03-29T04:18:54.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.872+0000 7f9df77fe640 1 -- 172.21.15.22:0/1689641067 learned_addr learned my addr 172.21.15.22:0/1689641067 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:54.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.872+0000 7f9deffff640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f9df815e940 0x7f9df806da50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.22:38584/0 (socket says 172.21.15.22:38584) 2024-03-29T04:18:54.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.872+0000 7f9df77fe640 1 -- 172.21.15.22:0/1689641067 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9df815a840 msgr2=0x7f9df815e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:54.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.872+0000 7f9df77fe640 1 --2- 172.21.15.22:0/1689641067 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9df815a840 0x7f9df815e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.872+0000 7f9df77fe640 1 -- 172.21.15.22:0/1689641067 >> v1:172.21.15.151:6789/0 conn(0x7f9df815e940 legacy=0x7f9df806da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:54.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.872+0000 7f9df77fe640 1 -- 172.21.15.22:0/1689641067 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df8073270 con 0x7f9df80b3f00 2024-03-29T04:18:54.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.872+0000 7f9df6ffd640 1 --2- 172.21.15.22:0/1689641067 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9df815a840 0x7f9df815e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:18:54.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.876+0000 7f9df77fe640 1 --2- 172.21.15.22:0/1689641067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9df80b3f00 0x7f9df806e160 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009e50 tx=0x7f9df001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=761b577121a7db0e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:54.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.876+0000 7f9df67fc640 1 -- 172.21.15.22:0/1689641067 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df00a5740 con 0x7f9df80b3f00 2024-03-29T04:18:54.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.876+0000 7f9df67fc640 1 -- 172.21.15.22:0/1689641067 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9df00a58e0 con 0x7f9df80b3f00 2024-03-29T04:18:54.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.876+0000 7f9df67fc640 1 -- 172.21.15.22:0/1689641067 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df00a5cb0 con 0x7f9df80b3f00 2024-03-29T04:18:54.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.876+0000 7f9dfd8f1640 1 -- 172.21.15.22:0/1689641067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9df80b3f00 msgr2=0x7f9df806e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:54.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.876+0000 7f9dfd8f1640 1 --2- 172.21.15.22:0/1689641067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9df80b3f00 0x7f9df806e160 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009e50 tx=0x7f9df001c970 comp rx=0 tx=0).stop 2024-03-29T04:18:54.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.876+0000 7f9dfd8f1640 1 -- 172.21.15.22:0/1689641067 shutdown_connections 2024-03-29T04:18:54.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.876+0000 7f9dfd8f1640 1 --2- 172.21.15.22:0/1689641067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9df80b3f00 0x7f9df806e160 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.876+0000 7f9dfd8f1640 1 --2- 172.21.15.22:0/1689641067 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9df815a840 0x7f9df815e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.876+0000 7f9dfd8f1640 1 -- 172.21.15.22:0/1689641067 >> 172.21.15.22:0/1689641067 conn(0x7f9df80cfee0 msgr2=0x7f9df80d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:54.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.876+0000 7f9dfd8f1640 1 -- 172.21.15.22:0/1689641067 shutdown_connections 2024-03-29T04:18:54.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.876+0000 7f9dfd8f1640 1 -- 172.21.15.22:0/1689641067 wait complete. 2024-03-29T04:18:54.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9dfd8f1640 1 Processor -- start 2024-03-29T04:18:54.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9dfd8f1640 1 -- start start 2024-03-29T04:18:54.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9dfd8f1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9df80b3f00 0x7f9df8159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:54.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9dfd8f1640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9df815a840 0x7f9df815b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:54.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9dfd8f1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9df815e940 0x7f9df815b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:54.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9dfd8f1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9df815f270 con 0x7f9df80b3f00 2024-03-29T04:18:54.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9dfd8f1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9df815f0f0 con 0x7f9df815e940 2024-03-29T04:18:54.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9dfd8f1640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9df80728d0 con 0x7f9df815a840 2024-03-29T04:18:54.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9df6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9df80b3f00 0x7f9df8159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:54.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9df6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9df80b3f00 0x7f9df8159550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43994/0 (socket says 172.21.15.22:43994) 2024-03-29T04:18:54.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9df6ffd640 1 -- 172.21.15.22:0/616620940 learned_addr learned my addr 172.21.15.22:0/616620940 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:54.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9df77fe640 1 --2- 172.21.15.22:0/616620940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9df815e940 0x7f9df815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:54.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9deffff640 1 --2- 172.21.15.22:0/616620940 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9df815a840 0x7f9df815b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:54.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9df6ffd640 1 -- 172.21.15.22:0/616620940 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9df815a840 msgr2=0x7f9df815b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:54.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9df6ffd640 1 --2- 172.21.15.22:0/616620940 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9df815a840 0x7f9df815b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9df6ffd640 1 -- 172.21.15.22:0/616620940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9df815e940 msgr2=0x7f9df815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:54.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9df6ffd640 1 --2- 172.21.15.22:0/616620940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9df815e940 0x7f9df815b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:54.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9df6ffd640 1 -- 172.21.15.22:0/616620940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df815bac0 con 0x7f9df80b3f00 2024-03-29T04:18:54.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9df6ffd640 1 --2- 172.21.15.22:0/616620940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9df80b3f00 0x7f9df8159550 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9de800ebe0 tx=0x7f9de800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:54.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9df4ff9640 1 -- 172.21.15.22:0/616620940 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de8017070 con 0x7f9df80b3f00 2024-03-29T04:18:54.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9df4ff9640 1 -- 172.21.15.22:0/616620940 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9de800cc30 con 0x7f9df80b3f00 2024-03-29T04:18:54.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9dfd8f1640 1 -- 172.21.15.22:0/616620940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df815bd90 con 0x7f9df80b3f00 2024-03-29T04:18:54.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9df4ff9640 1 -- 172.21.15.22:0/616620940 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de8013470 con 0x7f9df80b3f00 2024-03-29T04:18:54.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9dfd8f1640 1 -- 172.21.15.22:0/616620940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df815c350 con 0x7f9df80b3f00 2024-03-29T04:18:54.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.880+0000 7f9dfd8f1640 1 -- 172.21.15.22:0/616620940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9df8068890 con 0x7f9df80b3f00 2024-03-29T04:18:54.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.884+0000 7f9df4ff9640 1 -- 172.21.15.22:0/616620940 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9de801a070 con 0x7f9df80b3f00 2024-03-29T04:18:54.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.884+0000 7f9df4ff9640 1 --2- 172.21.15.22:0/616620940 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9dd8067fc0 0x7f9dd806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:54.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.884+0000 7f9df4ff9640 1 -- 172.21.15.22:0/616620940 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f9de8086250 con 0x7f9df80b3f00 2024-03-29T04:18:54.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.884+0000 7f9deffff640 1 --2- 172.21.15.22:0/616620940 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9dd8067fc0 0x7f9dd806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:54.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.884+0000 7f9deffff640 1 --2- 172.21.15.22:0/616620940 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9dd8067fc0 0x7f9dd806a480 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9df8157b50 tx=0x7f9de0007cd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:54.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:54.888+0000 7f9df4ff9640 1 -- 172.21.15.22:0/616620940 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9de8056c50 con 0x7f9df80b3f00 2024-03-29T04:18:55.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.096+0000 7f9dfd8f1640 1 -- 172.21.15.22:0/616620940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f9df80719d0 con 0x7f9df80b3f00 2024-03-29T04:18:55.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.100+0000 7f9df4ff9640 1 -- 172.21.15.22:0/616620940 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f9de8056550 con 0x7f9df80b3f00 2024-03-29T04:18:55.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.104+0000 7f9dfd8f1640 1 -- 172.21.15.22:0/616620940 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9dd8067fc0 msgr2=0x7f9dd806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:55.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.104+0000 7f9dfd8f1640 1 --2- 172.21.15.22:0/616620940 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9dd8067fc0 0x7f9dd806a480 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9df8157b50 tx=0x7f9de0007cd0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.104+0000 7f9dfd8f1640 1 -- 172.21.15.22:0/616620940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9df80b3f00 msgr2=0x7f9df8159550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:55.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.104+0000 7f9dfd8f1640 1 --2- 172.21.15.22:0/616620940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9df80b3f00 0x7f9df8159550 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9de800ebe0 tx=0x7f9de800c520 comp rx=0 tx=0).stop 2024-03-29T04:18:55.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.104+0000 7f9dfd8f1640 1 -- 172.21.15.22:0/616620940 shutdown_connections 2024-03-29T04:18:55.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.104+0000 7f9dfd8f1640 1 --2- 172.21.15.22:0/616620940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9df815e940 0x7f9df815b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.104+0000 7f9dfd8f1640 1 --2- 172.21.15.22:0/616620940 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9df815a840 0x7f9df815b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.104+0000 7f9dfd8f1640 1 --2- 172.21.15.22:0/616620940 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9dd8067fc0 0x7f9dd806a480 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.104+0000 7f9dfd8f1640 1 --2- 172.21.15.22:0/616620940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9df80b3f00 0x7f9df8159550 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.104+0000 7f9dfd8f1640 1 -- 172.21.15.22:0/616620940 >> 172.21.15.22:0/616620940 conn(0x7f9df80cfee0 msgr2=0x7f9df806bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:55.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.104+0000 7f9dfd8f1640 1 -- 172.21.15.22:0/616620940 shutdown_connections 2024-03-29T04:18:55.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.104+0000 7f9dfd8f1640 1 -- 172.21.15.22:0/616620940 wait complete. 2024-03-29T04:18:55.099 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:55.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.228+0000 7f9d3559e640 1 Processor -- start 2024-03-29T04:18:55.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.228+0000 7f9d3559e640 1 -- start start 2024-03-29T04:18:55.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.228+0000 7f9d3559e640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9d3015e600 0x7f9d300b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:55.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.228+0000 7f9d3559e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9d300d1560 0x7f9d300d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:55.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.228+0000 7f9d3559e640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d300d3e60 con 0x7f9d300d1100 2024-03-29T04:18:55.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.228+0000 7f9d3559e640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f9d300b7450 con 0x7f9d300d1560 2024-03-29T04:18:55.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.228+0000 7f9d3559e640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9d300d4140 con 0x7f9d3015e600 2024-03-29T04:18:55.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.228+0000 7f9d2f7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9d300d1560 0x7f9d300d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:55.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.228+0000 7f9d2f7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9d300d1560 0x7f9d300d3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44150/0 (socket says 172.21.15.22:44150) 2024-03-29T04:18:55.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.228+0000 7f9d2f7fe640 1 -- 172.21.15.22:0/2416919093 learned_addr learned my addr 172.21.15.22:0/2416919093 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:55.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.228+0000 7f9d2effd640 1 --2- 172.21.15.22:0/2416919093 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9d3015e600 0x7f9d300b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:55.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.228+0000 7f9d2f7fe640 1 -- 172.21.15.22:0/2416919093 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9d3015e600 msgr2=0x7f9d300b9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:55.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.228+0000 7f9d2f7fe640 1 --2- 172.21.15.22:0/2416919093 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9d3015e600 0x7f9d300b9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.228+0000 7f9d2f7fe640 1 -- 172.21.15.22:0/2416919093 >> v1:172.21.15.151:6789/0 conn(0x7f9d300d1100 legacy=0x7f9d300b8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:55.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.228+0000 7f9d2f7fe640 1 -- 172.21.15.22:0/2416919093 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d300b7680 con 0x7f9d300d1560 2024-03-29T04:18:55.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.228+0000 7f9d2f7fe640 1 --2- 172.21.15.22:0/2416919093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d300d1560 0x7f9d300d3610 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9d28009e50 tx=0x7f9d2801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a04b99f5091f6a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:55.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.228+0000 7f9d2dffb640 1 -- 172.21.15.22:0/2416919093 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d280a58d0 con 0x7f9d300d1560 2024-03-29T04:18:55.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.228+0000 7f9d2dffb640 1 -- 172.21.15.22:0/2416919093 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d280a5a70 con 0x7f9d300d1560 2024-03-29T04:18:55.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.228+0000 7f9d2dffb640 1 -- 172.21.15.22:0/2416919093 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d280a5e40 con 0x7f9d300d1560 2024-03-29T04:18:55.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d3559e640 1 -- 172.21.15.22:0/2416919093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d300d1560 msgr2=0x7f9d300d3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:55.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d3559e640 1 --2- 172.21.15.22:0/2416919093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d300d1560 0x7f9d300d3610 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9d28009e50 tx=0x7f9d2801ca00 comp rx=0 tx=0).stop 2024-03-29T04:18:55.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d3559e640 1 -- 172.21.15.22:0/2416919093 shutdown_connections 2024-03-29T04:18:55.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d3559e640 1 --2- 172.21.15.22:0/2416919093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d300d1560 0x7f9d300d3610 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d3559e640 1 --2- 172.21.15.22:0/2416919093 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9d3015e600 0x7f9d300b9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d3559e640 1 -- 172.21.15.22:0/2416919093 >> 172.21.15.22:0/2416919093 conn(0x7f9d300b6230 msgr2=0x7f9d300b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:55.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d3559e640 1 -- 172.21.15.22:0/2416919093 shutdown_connections 2024-03-29T04:18:55.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d3559e640 1 -- 172.21.15.22:0/2416919093 wait complete. 2024-03-29T04:18:55.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d3559e640 1 Processor -- start 2024-03-29T04:18:55.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d3559e640 1 -- start start 2024-03-29T04:18:55.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d3559e640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9d300d1100 0x7f9d3015b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:55.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d3559e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d300d1560 0x7f9d3014f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:55.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d3559e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d3015e600 0x7f9d3006c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:55.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d3559e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9d300b9080 con 0x7f9d3015e600 2024-03-29T04:18:55.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d3559e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9d300d3b50 con 0x7f9d300d1560 2024-03-29T04:18:55.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d3559e640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9d300b9200 con 0x7f9d300d1100 2024-03-29T04:18:55.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d2f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d3015e600 0x7f9d3006c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:55.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d2f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d3015e600 0x7f9d3006c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44154/0 (socket says 172.21.15.22:44154) 2024-03-29T04:18:55.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d2f7fe640 1 -- 172.21.15.22:0/3691044451 learned_addr learned my addr 172.21.15.22:0/3691044451 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:55.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d2effd640 1 --2- 172.21.15.22:0/3691044451 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9d300d1100 0x7f9d3015b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:55.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d2e7fc640 1 --2- 172.21.15.22:0/3691044451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d300d1560 0x7f9d3014f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:55.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d2f7fe640 1 -- 172.21.15.22:0/3691044451 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9d300d1100 msgr2=0x7f9d3015b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:55.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d2f7fe640 1 --2- 172.21.15.22:0/3691044451 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9d300d1100 0x7f9d3015b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d2f7fe640 1 -- 172.21.15.22:0/3691044451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d300d1560 msgr2=0x7f9d3014f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:55.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d2f7fe640 1 --2- 172.21.15.22:0/3691044451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d300d1560 0x7f9d3014f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d2f7fe640 1 -- 172.21.15.22:0/3691044451 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d300cd7b0 con 0x7f9d3015e600 2024-03-29T04:18:55.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d2effd640 1 --2- 172.21.15.22:0/3691044451 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9d300d1100 0x7f9d3015b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:55.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d2e7fc640 1 --2- 172.21.15.22:0/3691044451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d300d1560 0x7f9d3014f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:55.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d2f7fe640 1 --2- 172.21.15.22:0/3691044451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d3015e600 0x7f9d3006c850 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9d28009e50 tx=0x7f9d280197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:55.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d177fe640 1 -- 172.21.15.22:0/3691044451 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d280a58d0 con 0x7f9d3015e600 2024-03-29T04:18:55.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d177fe640 1 -- 172.21.15.22:0/3691044451 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d280be070 con 0x7f9d3015e600 2024-03-29T04:18:55.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d3559e640 1 -- 172.21.15.22:0/3691044451 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d300cda90 con 0x7f9d3015e600 2024-03-29T04:18:55.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d177fe640 1 -- 172.21.15.22:0/3691044451 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d28009510 con 0x7f9d3015e600 2024-03-29T04:18:55.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.232+0000 7f9d3559e640 1 -- 172.21.15.22:0/3691044451 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d300ce050 con 0x7f9d3015e600 2024-03-29T04:18:55.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.236+0000 7f9d3559e640 1 -- 172.21.15.22:0/3691044451 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d30068890 con 0x7f9d3015e600 2024-03-29T04:18:55.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.236+0000 7f9d177fe640 1 -- 172.21.15.22:0/3691044451 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9d280b86b0 con 0x7f9d3015e600 2024-03-29T04:18:55.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.236+0000 7f9d177fe640 1 --2- 172.21.15.22:0/3691044451 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9d1c067fc0 0x7f9d1c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:55.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.236+0000 7f9d177fe640 1 -- 172.21.15.22:0/3691044451 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f9d281224b0 con 0x7f9d3015e600 2024-03-29T04:18:55.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.236+0000 7f9d2effd640 1 --2- 172.21.15.22:0/3691044451 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9d1c067fc0 0x7f9d1c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:55.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.236+0000 7f9d2effd640 1 --2- 172.21.15.22:0/3691044451 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9d1c067fc0 0x7f9d1c06a480 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9d301fa610 tx=0x7f9d20008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:55.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.240+0000 7f9d177fe640 1 -- 172.21.15.22:0/3691044451 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9d28016620 con 0x7f9d3015e600 2024-03-29T04:18:55.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.396+0000 7f9d3559e640 1 -- 172.21.15.22:0/3691044451 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d300d3590 con 0x7f9d3015e600 2024-03-29T04:18:55.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.396+0000 7f9d177fe640 1 -- 172.21.15.22:0/3691044451 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+13164 (secure 0 0 0) 0x7f9d280f2f00 con 0x7f9d3015e600 2024-03-29T04:18:55.377 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:55.377 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":36,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:18:52.470399+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:55.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.400+0000 7f9d3559e640 1 -- 172.21.15.22:0/3691044451 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9d1c067fc0 msgr2=0x7f9d1c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:55.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.400+0000 7f9d3559e640 1 --2- 172.21.15.22:0/3691044451 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9d1c067fc0 0x7f9d1c06a480 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9d301fa610 tx=0x7f9d20008040 comp rx=0 tx=0).stop 2024-03-29T04:18:55.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.400+0000 7f9d3559e640 1 -- 172.21.15.22:0/3691044451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d3015e600 msgr2=0x7f9d3006c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:55.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.400+0000 7f9d3559e640 1 --2- 172.21.15.22:0/3691044451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d3015e600 0x7f9d3006c850 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9d28009e50 tx=0x7f9d280197a0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.400+0000 7f9d3559e640 1 -- 172.21.15.22:0/3691044451 shutdown_connections 2024-03-29T04:18:55.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.400+0000 7f9d3559e640 1 --2- 172.21.15.22:0/3691044451 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9d1c067fc0 0x7f9d1c06a480 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.400+0000 7f9d3559e640 1 --2- 172.21.15.22:0/3691044451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d3015e600 0x7f9d3006c850 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.400+0000 7f9d3559e640 1 --2- 172.21.15.22:0/3691044451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d300d1560 0x7f9d3014f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.400+0000 7f9d3559e640 1 --2- 172.21.15.22:0/3691044451 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9d300d1100 0x7f9d3015b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.400+0000 7f9d3559e640 1 -- 172.21.15.22:0/3691044451 >> 172.21.15.22:0/3691044451 conn(0x7f9d300b6230 msgr2=0x7f9d3015cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:55.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.400+0000 7f9d3559e640 1 -- 172.21.15.22:0/3691044451 shutdown_connections 2024-03-29T04:18:55.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.400+0000 7f9d3559e640 1 -- 172.21.15.22:0/3691044451 wait complete. 2024-03-29T04:18:55.395 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:18:55.395 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:18:55.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.532+0000 7fa5e9819640 1 Processor -- start 2024-03-29T04:18:55.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.532+0000 7fa5e9819640 1 -- start start 2024-03-29T04:18:55.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.532+0000 7fa5e9819640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa5e413f140 0x7fa5e41606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:55.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.532+0000 7fa5e9819640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e415ff40 0x7fa5e406c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:55.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.532+0000 7fa5e9819640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fa5e406cbb0 0x7fa5e4141a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:55.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.532+0000 7fa5e9819640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fa5e406d000 con 0x7fa5e413f140 2024-03-29T04:18:55.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.532+0000 7fa5e9819640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fa5e406e320 con 0x7fa5e406cbb0 2024-03-29T04:18:55.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.532+0000 7fa5e9819640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e406d710 con 0x7fa5e415ff40 2024-03-29T04:18:55.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.532+0000 7fa5e2ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa5e413f140 0x7fa5e41606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:55.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.532+0000 7fa5e2ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa5e413f140 0x7fa5e41606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44160/0 (socket says 172.21.15.22:44160) 2024-03-29T04:18:55.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.532+0000 7fa5e2ffd640 1 -- 172.21.15.22:0/4144183799 learned_addr learned my addr 172.21.15.22:0/4144183799 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:55.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.532+0000 7fa5e37fe640 1 --2- 172.21.15.22:0/4144183799 >> v2:172.21.15.151:3300/0 conn(0x7fa5e406cbb0 0x7fa5e4141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:55.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.532+0000 7fa5e27fc640 1 --2- 172.21.15.22:0/4144183799 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e415ff40 0x7fa5e406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:55.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.532+0000 7fa5e2ffd640 1 -- 172.21.15.22:0/4144183799 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e415ff40 msgr2=0x7fa5e406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:55.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.532+0000 7fa5e2ffd640 1 --2- 172.21.15.22:0/4144183799 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e415ff40 0x7fa5e406c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.532+0000 7fa5e2ffd640 1 -- 172.21.15.22:0/4144183799 >> v2:172.21.15.151:3300/0 conn(0x7fa5e406cbb0 msgr2=0x7fa5e4141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:55.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.532+0000 7fa5e2ffd640 1 --2- 172.21.15.22:0/4144183799 >> v2:172.21.15.151:3300/0 conn(0x7fa5e406cbb0 0x7fa5e4141a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.532+0000 7fa5e2ffd640 1 -- 172.21.15.22:0/4144183799 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e406d9f0 con 0x7fa5e413f140 2024-03-29T04:18:55.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e2ffd640 1 --2- 172.21.15.22:0/4144183799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5e413f140 0x7fa5e41606d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40b5470 tx=0x7fa5d401c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e0a9c711d5aa8f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:55.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e1ffb640 1 -- 172.21.15.22:0/4144183799 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d40a5580 con 0x7fa5e413f140 2024-03-29T04:18:55.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e1ffb640 1 -- 172.21.15.22:0/4144183799 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5d40a5720 con 0x7fa5e413f140 2024-03-29T04:18:55.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e1ffb640 1 -- 172.21.15.22:0/4144183799 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d40a5af0 con 0x7fa5e413f140 2024-03-29T04:18:55.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e9819640 1 -- 172.21.15.22:0/4144183799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5e413f140 msgr2=0x7fa5e41606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:55.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e9819640 1 --2- 172.21.15.22:0/4144183799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5e413f140 0x7fa5e41606d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40b5470 tx=0x7fa5d401c700 comp rx=0 tx=0).stop 2024-03-29T04:18:55.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e9819640 1 -- 172.21.15.22:0/4144183799 shutdown_connections 2024-03-29T04:18:55.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e9819640 1 --2- 172.21.15.22:0/4144183799 >> v2:172.21.15.151:3300/0 conn(0x7fa5e406cbb0 0x7fa5e4141a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e9819640 1 --2- 172.21.15.22:0/4144183799 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e415ff40 0x7fa5e406c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e9819640 1 --2- 172.21.15.22:0/4144183799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5e413f140 0x7fa5e41606d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e9819640 1 -- 172.21.15.22:0/4144183799 >> 172.21.15.22:0/4144183799 conn(0x7fa5e40d00a0 msgr2=0x7fa5e40d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:55.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e9819640 1 -- 172.21.15.22:0/4144183799 shutdown_connections 2024-03-29T04:18:55.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e9819640 1 -- 172.21.15.22:0/4144183799 wait complete. 2024-03-29T04:18:55.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e9819640 1 Processor -- start 2024-03-29T04:18:55.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e9819640 1 -- start start 2024-03-29T04:18:55.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e9819640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5e406cbb0 0x7fa5e41390e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e9819640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e413f140 0x7fa5e406bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e9819640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5e415ff40 0x7fa5e4072570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e9819640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e4160d90 con 0x7fa5e406cbb0 2024-03-29T04:18:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e9819640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e4160c10 con 0x7fa5e415ff40 2024-03-29T04:18:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e9819640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e40ce940 con 0x7fa5e413f140 2024-03-29T04:18:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5e406cbb0 0x7fa5e41390e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5e406cbb0 0x7fa5e41390e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44164/0 (socket says 172.21.15.22:44164) 2024-03-29T04:18:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e2ffd640 1 -- 172.21.15.22:0/131122685 learned_addr learned my addr 172.21.15.22:0/131122685 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e27fc640 1 --2- 172.21.15.22:0/131122685 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e413f140 0x7fa5e406bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e37fe640 1 --2- 172.21.15.22:0/131122685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5e415ff40 0x7fa5e4072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e2ffd640 1 -- 172.21.15.22:0/131122685 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e413f140 msgr2=0x7fa5e406bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e2ffd640 1 --2- 172.21.15.22:0/131122685 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e413f140 0x7fa5e406bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e2ffd640 1 -- 172.21.15.22:0/131122685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5e415ff40 msgr2=0x7fa5e4072570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:55.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e2ffd640 1 --2- 172.21.15.22:0/131122685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5e415ff40 0x7fa5e4072570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e2ffd640 1 -- 172.21.15.22:0/131122685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e41399e0 con 0x7fa5e406cbb0 2024-03-29T04:18:55.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e27fc640 1 --2- 172.21.15.22:0/131122685 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e413f140 0x7fa5e406bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:55.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e2ffd640 1 --2- 172.21.15.22:0/131122685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5e406cbb0 0x7fa5e41390e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40af040 tx=0x7fa5d40194a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:55.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5cbfff640 1 -- 172.21.15.22:0/131122685 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d40bc020 con 0x7fa5e406cbb0 2024-03-29T04:18:55.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5cbfff640 1 -- 172.21.15.22:0/131122685 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5d40b8670 con 0x7fa5e406cbb0 2024-03-29T04:18:55.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e9819640 1 -- 172.21.15.22:0/131122685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e4139cc0 con 0x7fa5e406cbb0 2024-03-29T04:18:55.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5cbfff640 1 -- 172.21.15.22:0/131122685 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4009510 con 0x7fa5e406cbb0 2024-03-29T04:18:55.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.536+0000 7fa5e9819640 1 -- 172.21.15.22:0/131122685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e413a280 con 0x7fa5e406cbb0 2024-03-29T04:18:55.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.540+0000 7fa5e9819640 1 -- 172.21.15.22:0/131122685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e4068900 con 0x7fa5e406cbb0 2024-03-29T04:18:55.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.540+0000 7fa5cbfff640 1 -- 172.21.15.22:0/131122685 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fa5d40bf020 con 0x7fa5e406cbb0 2024-03-29T04:18:55.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.540+0000 7fa5cbfff640 1 --2- 172.21.15.22:0/131122685 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa5d0067fc0 0x7fa5d006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:55.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.540+0000 7fa5cbfff640 1 -- 172.21.15.22:0/131122685 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fa5d4121320 con 0x7fa5e406cbb0 2024-03-29T04:18:55.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.540+0000 7fa5e27fc640 1 --2- 172.21.15.22:0/131122685 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa5d0067fc0 0x7fa5d006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:55.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.540+0000 7fa5e27fc640 1 --2- 172.21.15.22:0/131122685 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa5d0067fc0 0x7fa5d006a480 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40b1a10 tx=0x7fa5d8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:55.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.544+0000 7fa5cbfff640 1 -- 172.21.15.22:0/131122685 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fa5d40c6030 con 0x7fa5e406cbb0 2024-03-29T04:18:55.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.704+0000 7fa5e9819640 1 -- 172.21.15.22:0/131122685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa5e415ec80 con 0x7fa5e406cbb0 2024-03-29T04:18:55.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.704+0000 7fa5cbfff640 1 -- 172.21.15.22:0/131122685 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7fa5d40165e0 con 0x7fa5e406cbb0 2024-03-29T04:18:55.687 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:18:55.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.708+0000 7fa5e9819640 1 -- 172.21.15.22:0/131122685 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa5d0067fc0 msgr2=0x7fa5d006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:55.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.708+0000 7fa5e9819640 1 --2- 172.21.15.22:0/131122685 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa5d0067fc0 0x7fa5d006a480 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40b1a10 tx=0x7fa5d8008040 comp rx=0 tx=0).stop 2024-03-29T04:18:55.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.708+0000 7fa5e9819640 1 -- 172.21.15.22:0/131122685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5e406cbb0 msgr2=0x7fa5e41390e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:55.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.708+0000 7fa5e9819640 1 --2- 172.21.15.22:0/131122685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5e406cbb0 0x7fa5e41390e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40af040 tx=0x7fa5d40194a0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.708+0000 7fa5e9819640 1 -- 172.21.15.22:0/131122685 shutdown_connections 2024-03-29T04:18:55.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.708+0000 7fa5e9819640 1 --2- 172.21.15.22:0/131122685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5e415ff40 0x7fa5e4072570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.708+0000 7fa5e9819640 1 --2- 172.21.15.22:0/131122685 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa5e413f140 0x7fa5e406bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.708+0000 7fa5e9819640 1 --2- 172.21.15.22:0/131122685 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa5d0067fc0 0x7fa5d006a480 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.708+0000 7fa5e9819640 1 --2- 172.21.15.22:0/131122685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5e406cbb0 0x7fa5e41390e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.708+0000 7fa5e9819640 1 -- 172.21.15.22:0/131122685 >> 172.21.15.22:0/131122685 conn(0x7fa5e40d00a0 msgr2=0x7fa5e40b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:55.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.708+0000 7fa5e9819640 1 -- 172.21.15.22:0/131122685 shutdown_connections 2024-03-29T04:18:55.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.708+0000 7fa5e9819640 1 -- 172.21.15.22:0/131122685 wait complete. 2024-03-29T04:18:55.705 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-29T04:18:55.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.844+0000 7f94e3ec1640 1 Processor -- start 2024-03-29T04:18:55.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.844+0000 7f94e3ec1640 1 -- start start 2024-03-29T04:18:55.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.848+0000 7f94e3ec1640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f94dc13f0a0 0x7f94dc160470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:55.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.848+0000 7f94e3ec1640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f94dc1609b0 0x7f94dc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:55.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.848+0000 7f94e3ec1640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94dc06dcc0 0x7f94dc06b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:55.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.848+0000 7f94e3ec1640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f94dc06e0e0 con 0x7f94dc1609b0 2024-03-29T04:18:55.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.848+0000 7f94e3ec1640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f94dc141970 con 0x7f94dc13f0a0 2024-03-29T04:18:55.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.848+0000 7f94e3ec1640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f94dc141c50 con 0x7f94dc06dcc0 2024-03-29T04:18:55.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.848+0000 7f94e145c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f94dc1609b0 0x7f94dc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:55.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.848+0000 7f94e145c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f94dc1609b0 0x7f94dc06d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44168/0 (socket says 172.21.15.22:44168) 2024-03-29T04:18:55.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.848+0000 7f94e145c640 1 -- 172.21.15.22:0/1087524906 learned_addr learned my addr 172.21.15.22:0/1087524906 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:55.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.848+0000 7f94e1c5d640 1 --2- 172.21.15.22:0/1087524906 >> v2:172.21.15.151:3300/0 conn(0x7f94dc13f0a0 0x7f94dc160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:55.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.848+0000 7f94e145c640 1 -- 172.21.15.22:0/1087524906 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94dc06dcc0 msgr2=0x7f94dc06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:55.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.848+0000 7f94e145c640 1 --2- 172.21.15.22:0/1087524906 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94dc06dcc0 0x7f94dc06b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.848+0000 7f94e145c640 1 -- 172.21.15.22:0/1087524906 >> v2:172.21.15.151:3300/0 conn(0x7f94dc13f0a0 msgr2=0x7f94dc160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:55.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.848+0000 7f94e145c640 1 --2- 172.21.15.22:0/1087524906 >> v2:172.21.15.151:3300/0 conn(0x7f94dc13f0a0 0x7f94dc160470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.848+0000 7f94e145c640 1 -- 172.21.15.22:0/1087524906 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94dc141f30 con 0x7f94dc1609b0 2024-03-29T04:18:55.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.848+0000 7f94e1c5d640 1 --2- 172.21.15.22:0/1087524906 >> v2:172.21.15.151:3300/0 conn(0x7f94dc13f0a0 0x7f94dc160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:55.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e145c640 1 --2- 172.21.15.22:0/1087524906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94dc1609b0 0x7f94dc06d780 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f94dc073e60 tx=0x7f94cc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b9afba4299f4cb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:55.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e0c5b640 1 -- 172.21.15.22:0/1087524906 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0aa610 con 0x7f94dc1609b0 2024-03-29T04:18:55.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e0c5b640 1 -- 172.21.15.22:0/1087524906 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94cc0af070 con 0x7f94dc1609b0 2024-03-29T04:18:55.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e0c5b640 1 -- 172.21.15.22:0/1087524906 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0aa9d0 con 0x7f94dc1609b0 2024-03-29T04:18:55.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e3ec1640 1 -- 172.21.15.22:0/1087524906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94dc1609b0 msgr2=0x7f94dc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:55.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e3ec1640 1 --2- 172.21.15.22:0/1087524906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94dc1609b0 0x7f94dc06d780 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f94dc073e60 tx=0x7f94cc01c4f0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e3ec1640 1 -- 172.21.15.22:0/1087524906 shutdown_connections 2024-03-29T04:18:55.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e3ec1640 1 --2- 172.21.15.22:0/1087524906 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94dc06dcc0 0x7f94dc06b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e3ec1640 1 --2- 172.21.15.22:0/1087524906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94dc1609b0 0x7f94dc06d780 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e3ec1640 1 --2- 172.21.15.22:0/1087524906 >> v2:172.21.15.151:3300/0 conn(0x7f94dc13f0a0 0x7f94dc160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e3ec1640 1 -- 172.21.15.22:0/1087524906 >> 172.21.15.22:0/1087524906 conn(0x7f94dc072500 msgr2=0x7f94dc072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:55.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e3ec1640 1 -- 172.21.15.22:0/1087524906 shutdown_connections 2024-03-29T04:18:55.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e3ec1640 1 -- 172.21.15.22:0/1087524906 wait complete. 2024-03-29T04:18:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e3ec1640 1 Processor -- start 2024-03-29T04:18:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e3ec1640 1 -- start start 2024-03-29T04:18:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e3ec1640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94dc06dcc0 0x7f94dc13fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e3ec1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94dc13f0a0 0x7f94dc152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e3ec1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94dc1609b0 0x7f94dc13c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e3ec1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1600a0 con 0x7f94dc13f0a0 2024-03-29T04:18:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e3ec1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc15ff20 con 0x7f94dc1609b0 2024-03-29T04:18:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e3ec1640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f94dc0cfa20 con 0x7f94dc06dcc0 2024-03-29T04:18:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e145c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94dc13f0a0 0x7f94dc152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e145c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94dc13f0a0 0x7f94dc152ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44182/0 (socket says 172.21.15.22:44182) 2024-03-29T04:18:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e145c640 1 -- 172.21.15.22:0/2561529939 learned_addr learned my addr 172.21.15.22:0/2561529939 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e1c5d640 1 --2- 172.21.15.22:0/2561529939 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94dc06dcc0 0x7f94dc13fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e245e640 1 --2- 172.21.15.22:0/2561529939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94dc1609b0 0x7f94dc13c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e145c640 1 -- 172.21.15.22:0/2561529939 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94dc06dcc0 msgr2=0x7f94dc13fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e145c640 1 --2- 172.21.15.22:0/2561529939 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94dc06dcc0 0x7f94dc13fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e145c640 1 -- 172.21.15.22:0/2561529939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94dc1609b0 msgr2=0x7f94dc13c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e145c640 1 --2- 172.21.15.22:0/2561529939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94dc1609b0 0x7f94dc13c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e145c640 1 -- 172.21.15.22:0/2561529939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94dc151a20 con 0x7f94dc13f0a0 2024-03-29T04:18:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e245e640 1 --2- 172.21.15.22:0/2561529939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94dc1609b0 0x7f94dc13c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e145c640 1 --2- 172.21.15.22:0/2561529939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94dc13f0a0 0x7f94dc152ac0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f94cc01aba0 tx=0x7f94cc01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94d27fc640 1 -- 172.21.15.22:0/2561529939 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0b3800 con 0x7f94dc13f0a0 2024-03-29T04:18:55.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94d27fc640 1 -- 172.21.15.22:0/2561529939 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94cc0b39a0 con 0x7f94dc13f0a0 2024-03-29T04:18:55.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e3ec1640 1 -- 172.21.15.22:0/2561529939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94dc151d00 con 0x7f94dc13f0a0 2024-03-29T04:18:55.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94d27fc640 1 -- 172.21.15.22:0/2561529939 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0b3d90 con 0x7f94dc13f0a0 2024-03-29T04:18:55.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.852+0000 7f94e3ec1640 1 -- 172.21.15.22:0/2561529939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94dc1522c0 con 0x7f94dc13f0a0 2024-03-29T04:18:55.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.856+0000 7f94e3ec1640 1 -- 172.21.15.22:0/2561529939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94dc068890 con 0x7f94dc13f0a0 2024-03-29T04:18:55.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.856+0000 7f94d27fc640 1 -- 172.21.15.22:0/2561529939 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f94dc06c3b0 con 0x7f94dc13f0a0 2024-03-29T04:18:55.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.856+0000 7f94d27fc640 1 --2- 172.21.15.22:0/2561529939 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f94c0067fc0 0x7f94c006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:55.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.856+0000 7f94d27fc640 1 -- 172.21.15.22:0/2561529939 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f94cc120a40 con 0x7f94dc13f0a0 2024-03-29T04:18:55.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.856+0000 7f94e1c5d640 1 --2- 172.21.15.22:0/2561529939 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f94c0067fc0 0x7f94c006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:55.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.856+0000 7f94e1c5d640 1 --2- 172.21.15.22:0/2561529939 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f94c0067fc0 0x7f94c006a480 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f94d800a9a0 tx=0x7f94d8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:55.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:55.860+0000 7f94d27fc640 1 -- 172.21.15.22:0/2561529939 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f94cc0c3060 con 0x7f94dc13f0a0 2024-03-29T04:18:56.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.024+0000 7f94e3ec1640 1 -- 172.21.15.22:0/2561529939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f94dc074e60 con 0x7f94dc13f0a0 2024-03-29T04:18:56.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.024+0000 7f94d27fc640 1 -- 172.21.15.22:0/2561529939 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) ==== 84+0+65 (secure 0 0 0) 0x7f94cc0a7070 con 0x7f94dc13f0a0 2024-03-29T04:18:56.005 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:56.005 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2024-03-29T04:18:56.005 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-29T04:18:56.005 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 1, 2024-03-29T04:18:56.005 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": ".mgr" 2024-03-29T04:18:56.005 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:18:56.005 INFO:teuthology.orchestra.run.smithi022.stdout:] 2024-03-29T04:18:56.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.024+0000 7f94e3ec1640 1 -- 172.21.15.22:0/2561529939 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f94c0067fc0 msgr2=0x7f94c006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:56.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.024+0000 7f94e3ec1640 1 --2- 172.21.15.22:0/2561529939 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f94c0067fc0 0x7f94c006a480 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f94d800a9a0 tx=0x7f94d8008040 comp rx=0 tx=0).stop 2024-03-29T04:18:56.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.024+0000 7f94e3ec1640 1 -- 172.21.15.22:0/2561529939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94dc13f0a0 msgr2=0x7f94dc152ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:56.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.024+0000 7f94e3ec1640 1 --2- 172.21.15.22:0/2561529939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94dc13f0a0 0x7f94dc152ac0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f94cc01aba0 tx=0x7f94cc01ad30 comp rx=0 tx=0).stop 2024-03-29T04:18:56.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.024+0000 7f94e3ec1640 1 -- 172.21.15.22:0/2561529939 shutdown_connections 2024-03-29T04:18:56.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.024+0000 7f94e3ec1640 1 --2- 172.21.15.22:0/2561529939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94dc1609b0 0x7f94dc13c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.024+0000 7f94e3ec1640 1 --2- 172.21.15.22:0/2561529939 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f94c0067fc0 0x7f94c006a480 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.024+0000 7f94e3ec1640 1 --2- 172.21.15.22:0/2561529939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94dc13f0a0 0x7f94dc152ac0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.024+0000 7f94e3ec1640 1 --2- 172.21.15.22:0/2561529939 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94dc06dcc0 0x7f94dc13fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.028+0000 7f94e3ec1640 1 -- 172.21.15.22:0/2561529939 >> 172.21.15.22:0/2561529939 conn(0x7f94dc072500 msgr2=0x7f94dc15e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:56.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.028+0000 7f94e3ec1640 1 -- 172.21.15.22:0/2561529939 shutdown_connections 2024-03-29T04:18:56.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.028+0000 7f94e3ec1640 1 -- 172.21.15.22:0/2561529939 wait complete. 2024-03-29T04:18:56.022 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-29T04:18:56.023 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-29T04:18:56.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f6719eef640 1 Processor -- start 2024-03-29T04:18:56.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f6719eef640 1 -- start start 2024-03-29T04:18:56.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f6719eef640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f6714073490 0x7f671406c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:56.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f6719eef640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f671415d090 con 0x7f67140b5550 2024-03-29T04:18:56.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f6719eef640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f671415d370 con 0x7f67140b5930 2024-03-29T04:18:56.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f6719eef640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f6714160d20 con 0x7f6714073490 2024-03-29T04:18:56.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f67137fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f67140b5550 0x7f6714160610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:52154/0 (socket says 172.21.15.22:52154) 2024-03-29T04:18:56.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f67137fe640 1 -- 172.21.15.22:0/2210990656 learned_addr learned my addr 172.21.15.22:0/2210990656 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:56.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f6713fff640 1 --2- 172.21.15.22:0/2210990656 >> v2:172.21.15.151:3300/0 conn(0x7f6714073490 0x7f671406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:56.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f67127fc640 1 -- 172.21.15.22:0/2210990656 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 891423360 0 0) 0x7f671415d090 con 0x7f67140b5550 2024-03-29T04:18:56.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f67127fc640 1 -- 172.21.15.22:0/2210990656 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6700003530 con 0x7f67140b5550 2024-03-29T04:18:56.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f67127fc640 1 -- 172.21.15.22:0/2210990656 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 760038452 0 0) 0x7f6700003530 con 0x7f67140b5550 2024-03-29T04:18:56.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f67127fc640 1 -- 172.21.15.22:0/2210990656 >> v2:172.21.15.151:3300/0 conn(0x7f6714073490 msgr2=0x7f671406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:56.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f67127fc640 1 --2- 172.21.15.22:0/2210990656 >> v2:172.21.15.151:3300/0 conn(0x7f6714073490 0x7f671406c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f67127fc640 1 -- 172.21.15.22:0/2210990656 >> v1:172.21.15.151:6789/0 conn(0x7f67140b5930 legacy=0x7f671415c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:56.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f67127fc640 1 -- 172.21.15.22:0/2210990656 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671415fe10 con 0x7f67140b5550 2024-03-29T04:18:56.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f67127fc640 1 -- 172.21.15.22:0/2210990656 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f66fc001680 con 0x7f67140b5550 2024-03-29T04:18:56.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f67127fc640 1 -- 172.21.15.22:0/2210990656 >> v1:172.21.15.22:6789/0 conn(0x7f67140b5550 legacy=0x7f6714160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:56.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f67127fc640 1 --2- 172.21.15.22:0/2210990656 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f67000039c0 0x7f6700003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:56.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f67127fc640 1 --2- 172.21.15.22:0/2210990656 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6700004810 0x7f6700004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:56.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f67127fc640 1 --2- 172.21.15.22:0/2210990656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67000051d0 0x7f670000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:56.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f67127fc640 1 -- 172.21.15.22:0/2210990656 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6714160d20 con 0x7f67000051d0 2024-03-29T04:18:56.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f67127fc640 1 -- 172.21.15.22:0/2210990656 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f670000c070 con 0x7f6700004810 2024-03-29T04:18:56.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f67127fc640 1 -- 172.21.15.22:0/2210990656 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f670000c270 con 0x7f67000039c0 2024-03-29T04:18:56.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f6713fff640 1 --2- 172.21.15.22:0/2210990656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67000051d0 0x7f670000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:56.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f67137fe640 1 --2- 172.21.15.22:0/2210990656 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f67000039c0 0x7f6700003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:56.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f6712ffd640 1 --2- 172.21.15.22:0/2210990656 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6700004810 0x7f6700004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:56.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f6713fff640 1 -- 172.21.15.22:0/2210990656 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f67000039c0 msgr2=0x7f6700003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:56.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f6713fff640 1 --2- 172.21.15.22:0/2210990656 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f67000039c0 0x7f6700003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f6713fff640 1 -- 172.21.15.22:0/2210990656 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6700004810 msgr2=0x7f6700004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:56.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f6713fff640 1 --2- 172.21.15.22:0/2210990656 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6700004810 0x7f6700004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.160+0000 7f6713fff640 1 -- 172.21.15.22:0/2210990656 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6700003530 con 0x7f67000051d0 2024-03-29T04:18:56.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f6713fff640 1 -- 172.21.15.22:0/2210990656 reap_dead start 2024-03-29T04:18:56.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f6713fff640 1 --2- 172.21.15.22:0/2210990656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67000051d0 0x7f670000b9d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f670c009fc0 tx=0x7f670c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9944a16bec7355fa server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:56.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f67127fc640 1 -- 172.21.15.22:0/2210990656 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c009c10 con 0x7f67000051d0 2024-03-29T04:18:56.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f67127fc640 1 -- 172.21.15.22:0/2210990656 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f670c0a5d10 con 0x7f67000051d0 2024-03-29T04:18:56.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f67127fc640 1 -- 172.21.15.22:0/2210990656 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c003be0 con 0x7f67000051d0 2024-03-29T04:18:56.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f6719eef640 1 -- 172.21.15.22:0/2210990656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67000051d0 msgr2=0x7f670000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:56.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f6719eef640 1 --2- 172.21.15.22:0/2210990656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67000051d0 0x7f670000b9d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f670c009fc0 tx=0x7f670c01ccc0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f6719eef640 1 -- 172.21.15.22:0/2210990656 shutdown_connections 2024-03-29T04:18:56.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f6719eef640 1 --2- 172.21.15.22:0/2210990656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67000051d0 0x7f670000b9d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f6719eef640 1 -- 172.21.15.22:0/2210990656 >> 172.21.15.22:0/2210990656 conn(0x7f67140b1b80 msgr2=0x7f67140b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:56.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f6719eef640 1 -- 172.21.15.22:0/2210990656 shutdown_connections 2024-03-29T04:18:56.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f6719eef640 1 -- 172.21.15.22:0/2210990656 wait complete. 2024-03-29T04:18:56.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f6719eef640 1 Processor -- start 2024-03-29T04:18:56.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f6719eef640 1 -- start start 2024-03-29T04:18:56.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f6719eef640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67000051d0 0x7f67141448a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f6719eef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6714155ae0 0x7f671406ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f6719eef640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6714155f60 0x7f671406bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f6719eef640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6714160610 con 0x7f6714155ae0 2024-03-29T04:18:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f6719eef640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f671415d650 con 0x7f67000051d0 2024-03-29T04:18:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f6719eef640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f6714160790 con 0x7f6714155f60 2024-03-29T04:18:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f6712ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6714155ae0 0x7f671406ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:56.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f6712ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6714155ae0 0x7f671406ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44186/0 (socket says 172.21.15.22:44186) 2024-03-29T04:18:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.164+0000 7f6712ffd640 1 -- 172.21.15.22:0/995264066 learned_addr learned my addr 172.21.15.22:0/995264066 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.168+0000 7f67137fe640 1 --2- 172.21.15.22:0/995264066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67000051d0 0x7f67141448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.168+0000 7f6713fff640 1 --2- 172.21.15.22:0/995264066 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6714155f60 0x7f671406bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.168+0000 7f6712ffd640 1 -- 172.21.15.22:0/995264066 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6714155f60 msgr2=0x7f671406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.168+0000 7f6712ffd640 1 --2- 172.21.15.22:0/995264066 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6714155f60 0x7f671406bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.168+0000 7f6712ffd640 1 -- 172.21.15.22:0/995264066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67000051d0 msgr2=0x7f67141448a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.168+0000 7f6712ffd640 1 --2- 172.21.15.22:0/995264066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67000051d0 0x7f67141448a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.168+0000 7f6712ffd640 1 -- 172.21.15.22:0/995264066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67141520f0 con 0x7f6714155ae0 2024-03-29T04:18:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.168+0000 7f67137fe640 1 --2- 172.21.15.22:0/995264066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67000051d0 0x7f67141448a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.168+0000 7f6713fff640 1 --2- 172.21.15.22:0/995264066 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6714155f60 0x7f671406bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.168+0000 7f6712ffd640 1 --2- 172.21.15.22:0/995264066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6714155ae0 0x7f671406ba40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f670400e440 tx=0x7f670400e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.168+0000 7f670bfff640 1 -- 172.21.15.22:0/995264066 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67040052e0 con 0x7f6714155ae0 2024-03-29T04:18:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.168+0000 7f670bfff640 1 -- 172.21.15.22:0/995264066 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6704018070 con 0x7f6714155ae0 2024-03-29T04:18:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.168+0000 7f6719eef640 1 -- 172.21.15.22:0/995264066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6714158500 con 0x7f6714155ae0 2024-03-29T04:18:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.168+0000 7f670bfff640 1 -- 172.21.15.22:0/995264066 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704005480 con 0x7f6714155ae0 2024-03-29T04:18:56.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.168+0000 7f6719eef640 1 -- 172.21.15.22:0/995264066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6714158ac0 con 0x7f6714155ae0 2024-03-29T04:18:56.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.168+0000 7f6719eef640 1 -- 172.21.15.22:0/995264066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67140687f0 con 0x7f6714155ae0 2024-03-29T04:18:56.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.168+0000 7f670bfff640 1 -- 172.21.15.22:0/995264066 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f6704003840 con 0x7f6714155ae0 2024-03-29T04:18:56.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.168+0000 7f670bfff640 1 --2- 172.21.15.22:0/995264066 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f66f4067fc0 0x7f66f406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:56.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.168+0000 7f670bfff640 1 -- 172.21.15.22:0/995264066 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f6704085fe0 con 0x7f6714155ae0 2024-03-29T04:18:56.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.168+0000 7f67137fe640 1 --2- 172.21.15.22:0/995264066 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f66f4067fc0 0x7f66f406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:56.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.168+0000 7f67137fe640 1 --2- 172.21.15.22:0/995264066 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f66f4067fc0 0x7f66f406a480 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0017c0 tx=0x7f66fc000ee0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:56.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.172+0000 7f670bfff640 1 -- 172.21.15.22:0/995264066 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f67141520f0 con 0x7f6714155ae0 2024-03-29T04:18:56.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.328+0000 7f6719eef640 1 -- 172.21.15.22:0/995264066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f67140b0090 con 0x7f6714155ae0 2024-03-29T04:18:56.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.604+0000 7f670bfff640 1 -- 172.21.15.22:0/995264066 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) ==== 138+0+0 (secure 0 0 0) 0x7f6704056910 con 0x7f6714155ae0 2024-03-29T04:18:56.587 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' created 2024-03-29T04:18:56.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.608+0000 7f6719eef640 1 -- 172.21.15.22:0/995264066 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f66f4067fc0 msgr2=0x7f66f406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:56.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.608+0000 7f6719eef640 1 --2- 172.21.15.22:0/995264066 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f66f4067fc0 0x7f66f406a480 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0017c0 tx=0x7f66fc000ee0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.608+0000 7f6719eef640 1 -- 172.21.15.22:0/995264066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6714155ae0 msgr2=0x7f671406ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:56.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.608+0000 7f6719eef640 1 --2- 172.21.15.22:0/995264066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6714155ae0 0x7f671406ba40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f670400e440 tx=0x7f670400e900 comp rx=0 tx=0).stop 2024-03-29T04:18:56.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.608+0000 7f6719eef640 1 -- 172.21.15.22:0/995264066 shutdown_connections 2024-03-29T04:18:56.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.608+0000 7f6719eef640 1 --2- 172.21.15.22:0/995264066 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6714155f60 0x7f671406bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.608+0000 7f6719eef640 1 --2- 172.21.15.22:0/995264066 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f66f4067fc0 0x7f66f406a480 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.608+0000 7f6719eef640 1 --2- 172.21.15.22:0/995264066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6714155ae0 0x7f671406ba40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.608+0000 7f6719eef640 1 --2- 172.21.15.22:0/995264066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67000051d0 0x7f67141448a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.608+0000 7f6719eef640 1 -- 172.21.15.22:0/995264066 >> 172.21.15.22:0/995264066 conn(0x7f67140b1b80 msgr2=0x7f67140b0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:56.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.608+0000 7f6719eef640 1 -- 172.21.15.22:0/995264066 shutdown_connections 2024-03-29T04:18:56.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.608+0000 7f6719eef640 1 -- 172.21.15.22:0/995264066 wait complete. 2024-03-29T04:18:56.614 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-29T04:18:56.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.764+0000 7fc5e6dcf640 1 Processor -- start 2024-03-29T04:18:56.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.764+0000 7fc5e6dcf640 1 -- start start 2024-03-29T04:18:56.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5e6dcf640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5e00b31c0 0x7fc5e015f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:56.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5e6dcf640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc5e00b6a50 0x7fc5e00b9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:56.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5e6dcf640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5e00cf580 con 0x7fc5e00b3860 2024-03-29T04:18:56.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5e6dcf640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fc5e015f8d0 con 0x7fc5e00b6a50 2024-03-29T04:18:56.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5e6dcf640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc5e00b3d40 con 0x7fc5e00b31c0 2024-03-29T04:18:56.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5dffff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fc5e00b3860 0x7fc5e00b6340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:52166/0 (socket says 172.21.15.22:52166) 2024-03-29T04:18:56.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5dffff640 1 -- 172.21.15.22:0/352877655 learned_addr learned my addr 172.21.15.22:0/352877655 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:56.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5e536c640 1 --2- 172.21.15.22:0/352877655 >> v2:172.21.15.151:3300/0 conn(0x7fc5e00b6a50 0x7fc5e00b9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:56.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5e4b6b640 1 --2- 172.21.15.22:0/352877655 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5e00b31c0 0x7fc5e015f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:56.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5df7fe640 1 -- 172.21.15.22:0/352877655 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 167306619 0 0) 0x7fc5e00cf580 con 0x7fc5e00b3860 2024-03-29T04:18:56.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5df7fe640 1 -- 172.21.15.22:0/352877655 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5d0003530 con 0x7fc5e00b3860 2024-03-29T04:18:56.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5df7fe640 1 -- 172.21.15.22:0/352877655 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3338388642 0 0) 0x7fc5d0003530 con 0x7fc5e00b3860 2024-03-29T04:18:56.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5df7fe640 1 -- 172.21.15.22:0/352877655 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5e00b31c0 msgr2=0x7fc5e015f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:56.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5df7fe640 1 --2- 172.21.15.22:0/352877655 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5e00b31c0 0x7fc5e015f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5df7fe640 1 -- 172.21.15.22:0/352877655 >> v2:172.21.15.151:3300/0 conn(0x7fc5e00b6a50 msgr2=0x7fc5e00b9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:56.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5df7fe640 1 --2- 172.21.15.22:0/352877655 >> v2:172.21.15.151:3300/0 conn(0x7fc5e00b6a50 0x7fc5e00b9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5df7fe640 1 -- 172.21.15.22:0/352877655 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e00b4020 con 0x7fc5e00b3860 2024-03-29T04:18:56.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5df7fe640 1 -- 172.21.15.22:0/352877655 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7fc5cc0016a0 con 0x7fc5e00b3860 2024-03-29T04:18:56.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5df7fe640 1 -- 172.21.15.22:0/352877655 >> v1:172.21.15.22:6789/0 conn(0x7fc5e00b3860 legacy=0x7fc5e00b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:56.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5df7fe640 1 --2- 172.21.15.22:0/352877655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5d00039c0 0x7fc5d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:56.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5df7fe640 1 --2- 172.21.15.22:0/352877655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5d0004810 0x7fc5d0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:56.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5df7fe640 1 --2- 172.21.15.22:0/352877655 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5e00b31c0 0x7fc5d00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:56.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5df7fe640 1 -- 172.21.15.22:0/352877655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e015f8d0 con 0x7fc5d0004810 2024-03-29T04:18:56.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5df7fe640 1 -- 172.21.15.22:0/352877655 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e00b3d40 con 0x7fc5d00039c0 2024-03-29T04:18:56.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5df7fe640 1 -- 172.21.15.22:0/352877655 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc5d000b9d0 con 0x7fc5e00b31c0 2024-03-29T04:18:56.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5dffff640 1 --2- 172.21.15.22:0/352877655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5d0004810 0x7fc5d0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:56.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5e4b6b640 1 --2- 172.21.15.22:0/352877655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5d00039c0 0x7fc5d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:56.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5e536c640 1 --2- 172.21.15.22:0/352877655 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5e00b31c0 0x7fc5d00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:56.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5dffff640 1 -- 172.21.15.22:0/352877655 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5e00b31c0 msgr2=0x7fc5d00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:56.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5dffff640 1 --2- 172.21.15.22:0/352877655 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5e00b31c0 0x7fc5d00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5dffff640 1 -- 172.21.15.22:0/352877655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5d00039c0 msgr2=0x7fc5d0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:56.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5dffff640 1 --2- 172.21.15.22:0/352877655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5d00039c0 0x7fc5d0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.768+0000 7fc5dffff640 1 -- 172.21.15.22:0/352877655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d0003530 con 0x7fc5d0004810 2024-03-29T04:18:56.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5dffff640 1 --2- 172.21.15.22:0/352877655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5d0004810 0x7fc5d0004c90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00cf750 tx=0x7fc5cc01f2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62bd026b4313a64 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:56.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5df7fe640 1 -- 172.21.15.22:0/352877655 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc007cb0 con 0x7fc5d0004810 2024-03-29T04:18:56.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5df7fe640 1 -- 172.21.15.22:0/352877655 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5cc0ac020 con 0x7fc5d0004810 2024-03-29T04:18:56.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5df7fe640 1 -- 172.21.15.22:0/352877655 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc0058e0 con 0x7fc5d0004810 2024-03-29T04:18:56.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5e6dcf640 1 -- 172.21.15.22:0/352877655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5d0004810 msgr2=0x7fc5d0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:56.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5e6dcf640 1 --2- 172.21.15.22:0/352877655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5d0004810 0x7fc5d0004c90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00cf750 tx=0x7fc5cc01f2b0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5e536c640 1 -- 172.21.15.22:0/352877655 reap_dead start 2024-03-29T04:18:56.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5e6dcf640 1 -- 172.21.15.22:0/352877655 shutdown_connections 2024-03-29T04:18:56.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5e6dcf640 1 -- 172.21.15.22:0/352877655 >> 172.21.15.22:0/352877655 conn(0x7fc5e00d11e0 msgr2=0x7fc5e00d15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:56.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5e6dcf640 1 -- 172.21.15.22:0/352877655 shutdown_connections 2024-03-29T04:18:56.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5e6dcf640 1 -- 172.21.15.22:0/352877655 wait complete. 2024-03-29T04:18:56.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5e6dcf640 1 Processor -- start 2024-03-29T04:18:56.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5e6dcf640 1 -- start start 2024-03-29T04:18:56.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5e6dcf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5d0004810 0x7fc5e00cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:56.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5e6dcf640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5e015db60 0x7fc5e00c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:56.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5e6dcf640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5e01e7d50 0x7fc5e00c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:56.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5e6dcf640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e00b48c0 con 0x7fc5d0004810 2024-03-29T04:18:56.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5e6dcf640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e00b45e0 con 0x7fc5e01e7d50 2024-03-29T04:18:56.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5e6dcf640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc5e00b35a0 con 0x7fc5e015db60 2024-03-29T04:18:56.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5e4b6b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5d0004810 0x7fc5e00cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:56.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5e4b6b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5d0004810 0x7fc5e00cdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44212/0 (socket says 172.21.15.22:44212) 2024-03-29T04:18:56.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.772+0000 7fc5e4b6b640 1 -- 172.21.15.22:0/1338372522 learned_addr learned my addr 172.21.15.22:0/1338372522 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:56.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.776+0000 7fc5e536c640 1 --2- 172.21.15.22:0/1338372522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5e01e7d50 0x7fc5e00c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:56.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.776+0000 7fc5dffff640 1 --2- 172.21.15.22:0/1338372522 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5e015db60 0x7fc5e00c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:56.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.776+0000 7fc5e4b6b640 1 -- 172.21.15.22:0/1338372522 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5e015db60 msgr2=0x7fc5e00c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:56.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.776+0000 7fc5e4b6b640 1 --2- 172.21.15.22:0/1338372522 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5e015db60 0x7fc5e00c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.776+0000 7fc5e4b6b640 1 -- 172.21.15.22:0/1338372522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5e01e7d50 msgr2=0x7fc5e00c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:56.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.776+0000 7fc5e4b6b640 1 --2- 172.21.15.22:0/1338372522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5e01e7d50 0x7fc5e00c7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:56.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.776+0000 7fc5e4b6b640 1 -- 172.21.15.22:0/1338372522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e015cb20 con 0x7fc5d0004810 2024-03-29T04:18:56.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.776+0000 7fc5e4b6b640 1 --2- 172.21.15.22:0/1338372522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5d0004810 0x7fc5e00cdf10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4011590 tx=0x7fc5d4011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:56.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.776+0000 7fc5dd7fa640 1 -- 172.21.15.22:0/1338372522 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4015930 con 0x7fc5d0004810 2024-03-29T04:18:56.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.776+0000 7fc5dd7fa640 1 -- 172.21.15.22:0/1338372522 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5d4015ad0 con 0x7fc5d0004810 2024-03-29T04:18:56.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.776+0000 7fc5e6dcf640 1 -- 172.21.15.22:0/1338372522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e00c79d0 con 0x7fc5d0004810 2024-03-29T04:18:56.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.776+0000 7fc5dd7fa640 1 -- 172.21.15.22:0/1338372522 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d400f460 con 0x7fc5d0004810 2024-03-29T04:18:56.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.776+0000 7fc5e6dcf640 1 -- 172.21.15.22:0/1338372522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e015d080 con 0x7fc5d0004810 2024-03-29T04:18:56.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.776+0000 7fc5e6dcf640 1 -- 172.21.15.22:0/1338372522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5e0068900 con 0x7fc5d0004810 2024-03-29T04:18:56.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.776+0000 7fc5dd7fa640 1 -- 172.21.15.22:0/1338372522 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fc5d400f600 con 0x7fc5d0004810 2024-03-29T04:18:56.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.776+0000 7fc5dd7fa640 1 --2- 172.21.15.22:0/1338372522 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc5c4067ff0 0x7fc5c406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:56.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.776+0000 7fc5dd7fa640 1 -- 172.21.15.22:0/1338372522 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5619+0+0 (secure 0 0 0) 0x7fc5d4086c60 con 0x7fc5d0004810 2024-03-29T04:18:56.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.776+0000 7fc5dffff640 1 --2- 172.21.15.22:0/1338372522 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc5c4067ff0 0x7fc5c406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:56.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.776+0000 7fc5dffff640 1 --2- 172.21.15.22:0/1338372522 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc5c4067ff0 0x7fc5c406a4b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc0ad000 tx=0x7fc5cc0019a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:56.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.780+0000 7fc5dd7fa640 1 -- 172.21.15.22:0/1338372522 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fc5e015cb20 con 0x7fc5d0004810 2024-03-29T04:18:56.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:56.940+0000 7fc5e6dcf640 1 -- 172.21.15.22:0/1338372522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc5e015df40 con 0x7fc5d0004810 2024-03-29T04:18:57.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.607+0000 7fc5dd7fa640 1 -- 172.21.15.22:0/1338372522 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) ==== 170+0+0 (secure 0 0 0) 0x7fc5d4019030 con 0x7fc5d0004810 2024-03-29T04:18:57.591 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' created 2024-03-29T04:18:57.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.615+0000 7fc5e6dcf640 1 -- 172.21.15.22:0/1338372522 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc5c4067ff0 msgr2=0x7fc5c406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:57.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.615+0000 7fc5e6dcf640 1 --2- 172.21.15.22:0/1338372522 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc5c4067ff0 0x7fc5c406a4b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc0ad000 tx=0x7fc5cc0019a0 comp rx=0 tx=0).stop 2024-03-29T04:18:57.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.615+0000 7fc5e6dcf640 1 -- 172.21.15.22:0/1338372522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5d0004810 msgr2=0x7fc5e00cdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:57.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.615+0000 7fc5e6dcf640 1 --2- 172.21.15.22:0/1338372522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5d0004810 0x7fc5e00cdf10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4011590 tx=0x7fc5d4011a50 comp rx=0 tx=0).stop 2024-03-29T04:18:57.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.615+0000 7fc5e6dcf640 1 -- 172.21.15.22:0/1338372522 shutdown_connections 2024-03-29T04:18:57.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.615+0000 7fc5e6dcf640 1 --2- 172.21.15.22:0/1338372522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5e01e7d50 0x7fc5e00c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:57.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.615+0000 7fc5e6dcf640 1 --2- 172.21.15.22:0/1338372522 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc5e015db60 0x7fc5e00c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:57.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.615+0000 7fc5e6dcf640 1 --2- 172.21.15.22:0/1338372522 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc5c4067ff0 0x7fc5c406a4b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:57.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.615+0000 7fc5e6dcf640 1 --2- 172.21.15.22:0/1338372522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5d0004810 0x7fc5e00cdf10 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:57.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.615+0000 7fc5e6dcf640 1 -- 172.21.15.22:0/1338372522 >> 172.21.15.22:0/1338372522 conn(0x7fc5e00d11e0 msgr2=0x7fc5e006cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:57.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.619+0000 7fc5e6dcf640 1 -- 172.21.15.22:0/1338372522 shutdown_connections 2024-03-29T04:18:57.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.627+0000 7fc5e6dcf640 1 -- 172.21.15.22:0/1338372522 wait complete. 2024-03-29T04:18:57.636 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-29T04:18:57.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.775+0000 7f497f3ed640 1 Processor -- start 2024-03-29T04:18:57.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.775+0000 7f497f3ed640 1 -- start start 2024-03-29T04:18:57.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.775+0000 7f497f3ed640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f497815e4c0 0x7f49780b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:57.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.775+0000 7f497f3ed640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f49780cf6c0 0x7f49780d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:57.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.775+0000 7f497f3ed640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f497806c850 con 0x7f497806c070 2024-03-29T04:18:57.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.775+0000 7f497f3ed640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f497815e8a0 con 0x7f497815e4c0 2024-03-29T04:18:57.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.775+0000 7f497f3ed640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f497806cb30 con 0x7f49780cf6c0 2024-03-29T04:18:57.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.775+0000 7f497d189640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f497815e4c0 0x7f49780b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:57.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.775+0000 7f497c988640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f497806c070 0x7f49780cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:52182/0 (socket says 172.21.15.22:52182) 2024-03-29T04:18:57.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.775+0000 7f497c988640 1 -- 172.21.15.22:0/3058820126 learned_addr learned my addr 172.21.15.22:0/3058820126 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:18:57.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.775+0000 7f497d98a640 1 --2- 172.21.15.22:0/3058820126 >> v2:172.21.15.151:3300/0 conn(0x7f49780cf6c0 0x7f49780d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:57.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.775+0000 7f497d189640 1 -- 172.21.15.22:0/3058820126 >> v2:172.21.15.151:3300/0 conn(0x7f49780cf6c0 msgr2=0x7f49780d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:57.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.775+0000 7f497d189640 1 --2- 172.21.15.22:0/3058820126 >> v2:172.21.15.151:3300/0 conn(0x7f49780cf6c0 0x7f49780d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:57.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.775+0000 7f497d189640 1 -- 172.21.15.22:0/3058820126 >> v1:172.21.15.22:6789/0 conn(0x7f497806c070 legacy=0x7f49780cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:57.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.775+0000 7f496ffff640 1 -- 172.21.15.22:0/3058820126 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 911786548 0 0) 0x7f497806c850 con 0x7f497806c070 2024-03-29T04:18:57.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.775+0000 7f497d189640 1 -- 172.21.15.22:0/3058820126 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497806ce10 con 0x7f497815e4c0 2024-03-29T04:18:57.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.779+0000 7f497d189640 1 --2- 172.21.15.22:0/3058820126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f497815e4c0 0x7f49780b9e30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f49780cb340 tx=0x7f497401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6f381695db110e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:57.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.779+0000 7f496ffff640 1 -- 172.21.15.22:0/3058820126 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49740b0070 con 0x7f497815e4c0 2024-03-29T04:18:57.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.779+0000 7f496ffff640 1 -- 172.21.15.22:0/3058820126 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49740ab600 con 0x7f497815e4c0 2024-03-29T04:18:57.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.779+0000 7f496ffff640 1 -- 172.21.15.22:0/3058820126 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49740ab9d0 con 0x7f497815e4c0 2024-03-29T04:18:57.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.779+0000 7f497f3ed640 1 -- 172.21.15.22:0/3058820126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f497815e4c0 msgr2=0x7f49780b9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:57.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.779+0000 7f497f3ed640 1 --2- 172.21.15.22:0/3058820126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f497815e4c0 0x7f49780b9e30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f49780cb340 tx=0x7f497401c480 comp rx=0 tx=0).stop 2024-03-29T04:18:57.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.779+0000 7f497f3ed640 1 -- 172.21.15.22:0/3058820126 shutdown_connections 2024-03-29T04:18:57.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.779+0000 7f497f3ed640 1 --2- 172.21.15.22:0/3058820126 >> v2:172.21.15.151:3300/0 conn(0x7f49780cf6c0 0x7f49780d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:57.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.779+0000 7f497f3ed640 1 --2- 172.21.15.22:0/3058820126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f497815e4c0 0x7f49780b9e30 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:57.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.779+0000 7f497f3ed640 1 -- 172.21.15.22:0/3058820126 >> 172.21.15.22:0/3058820126 conn(0x7f49780b6410 msgr2=0x7f49780d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:57.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.779+0000 7f497f3ed640 1 -- 172.21.15.22:0/3058820126 shutdown_connections 2024-03-29T04:18:57.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.779+0000 7f497f3ed640 1 -- 172.21.15.22:0/3058820126 wait complete. 2024-03-29T04:18:57.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497f3ed640 1 Processor -- start 2024-03-29T04:18:57.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497f3ed640 1 -- start start 2024-03-29T04:18:57.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497f3ed640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f497806c070 0x7f4978160fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:57.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497f3ed640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49780cf6c0 0x7f497815fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:57.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497f3ed640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f497815e4c0 0x7f49781602c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:57.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497f3ed640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f49780d3e90 con 0x7f497806c070 2024-03-29T04:18:57.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497f3ed640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f49780d3d10 con 0x7f49780cf6c0 2024-03-29T04:18:57.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497f3ed640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f49780d4010 con 0x7f497815e4c0 2024-03-29T04:18:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497d189640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f497806c070 0x7f4978160fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497d189640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f497806c070 0x7f4978160fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44230/0 (socket says 172.21.15.22:44230) 2024-03-29T04:18:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497d189640 1 -- 172.21.15.22:0/1140793983 learned_addr learned my addr 172.21.15.22:0/1140793983 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497c988640 1 --2- 172.21.15.22:0/1140793983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49780cf6c0 0x7f497815fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497d98a640 1 --2- 172.21.15.22:0/1140793983 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f497815e4c0 0x7f49781602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497d189640 1 -- 172.21.15.22:0/1140793983 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f497815e4c0 msgr2=0x7f49781602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497d189640 1 --2- 172.21.15.22:0/1140793983 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f497815e4c0 0x7f49781602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497d189640 1 -- 172.21.15.22:0/1140793983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49780cf6c0 msgr2=0x7f497815fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497d189640 1 --2- 172.21.15.22:0/1140793983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49780cf6c0 0x7f497815fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497d189640 1 -- 172.21.15.22:0/1140793983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497815ea40 con 0x7f497806c070 2024-03-29T04:18:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497d189640 1 --2- 172.21.15.22:0/1140793983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f497806c070 0x7f4978160fd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f497401a0b0 tx=0x7f497401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f496dffb640 1 -- 172.21.15.22:0/1140793983 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49740bd070 con 0x7f497806c070 2024-03-29T04:18:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f496dffb640 1 -- 172.21.15.22:0/1140793983 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4974004230 con 0x7f497806c070 2024-03-29T04:18:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f496dffb640 1 -- 172.21.15.22:0/1140793983 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49740b0040 con 0x7f497806c070 2024-03-29T04:18:57.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497f3ed640 1 -- 172.21.15.22:0/1140793983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f497815ece0 con 0x7f497806c070 2024-03-29T04:18:57.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497f3ed640 1 -- 172.21.15.22:0/1140793983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f497815f2a0 con 0x7f497806c070 2024-03-29T04:18:57.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.783+0000 7f497f3ed640 1 -- 172.21.15.22:0/1140793983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4978068900 con 0x7f497806c070 2024-03-29T04:18:57.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.787+0000 7f496dffb640 1 -- 172.21.15.22:0/1140793983 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f49740ab4c0 con 0x7f497806c070 2024-03-29T04:18:57.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.787+0000 7f496dffb640 1 --2- 172.21.15.22:0/1140793983 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f495c067fc0 0x7f495c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:57.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.787+0000 7f496dffb640 1 -- 172.21.15.22:0/1140793983 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6001+0+0 (secure 0 0 0) 0x7f49741203f0 con 0x7f497806c070 2024-03-29T04:18:57.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.787+0000 7f497c988640 1 --2- 172.21.15.22:0/1140793983 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f495c067fc0 0x7f495c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:57.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.787+0000 7f497c988640 1 --2- 172.21.15.22:0/1140793983 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f495c067fc0 0x7f495c06a480 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f49780c76d0 tx=0x7f4968007b60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:57.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.791+0000 7f496dffb640 1 -- 172.21.15.22:0/1140793983 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f4974016070 con 0x7f497806c070 2024-03-29T04:18:57.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:57.983+0000 7f497f3ed640 1 -- 172.21.15.22:0/1140793983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4978074470 con 0x7f497806c070 2024-03-29T04:18:58.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.663+0000 7f496dffb640 1 -- 172.21.15.22:0/1140793983 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f49740f0a70 con 0x7f497806c070 2024-03-29T04:18:58.651 INFO:teuthology.orchestra.run.smithi022.stderr:new fs with metadata pool 5 and data pool 6 2024-03-29T04:18:58.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.671+0000 7f497f3ed640 1 -- 172.21.15.22:0/1140793983 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f495c067fc0 msgr2=0x7f495c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:58.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.671+0000 7f497f3ed640 1 --2- 172.21.15.22:0/1140793983 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f495c067fc0 0x7f495c06a480 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f49780c76d0 tx=0x7f4968007b60 comp rx=0 tx=0).stop 2024-03-29T04:18:58.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.671+0000 7f497f3ed640 1 -- 172.21.15.22:0/1140793983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f497806c070 msgr2=0x7f4978160fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:58.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.671+0000 7f497f3ed640 1 --2- 172.21.15.22:0/1140793983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f497806c070 0x7f4978160fd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f497401a0b0 tx=0x7f497401acc0 comp rx=0 tx=0).stop 2024-03-29T04:18:58.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.675+0000 7f497f3ed640 1 -- 172.21.15.22:0/1140793983 shutdown_connections 2024-03-29T04:18:58.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.675+0000 7f497f3ed640 1 --2- 172.21.15.22:0/1140793983 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f497815e4c0 0x7f49781602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:58.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.675+0000 7f497f3ed640 1 --2- 172.21.15.22:0/1140793983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49780cf6c0 0x7f497815fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:58.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.675+0000 7f497f3ed640 1 --2- 172.21.15.22:0/1140793983 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f495c067fc0 0x7f495c06a480 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:58.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.675+0000 7f497f3ed640 1 --2- 172.21.15.22:0/1140793983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f497806c070 0x7f4978160fd0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:58.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.675+0000 7f497f3ed640 1 -- 172.21.15.22:0/1140793983 >> 172.21.15.22:0/1140793983 conn(0x7f49780b6410 msgr2=0x7f497815d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:58.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.679+0000 7f497f3ed640 1 -- 172.21.15.22:0/1140793983 shutdown_connections 2024-03-29T04:18:58.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.679+0000 7f497f3ed640 1 -- 172.21.15.22:0/1140793983 wait complete. 2024-03-29T04:18:58.686 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:58.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.815+0000 7f94ff6ad640 1 Processor -- start 2024-03-29T04:18:58.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.815+0000 7f94ff6ad640 1 -- start start 2024-03-29T04:18:58.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.815+0000 7f94ff6ad640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f94f806c670 0x7f94f80cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:58.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.815+0000 7f94ff6ad640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f94f80b41f0 0x7f94f806e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:58.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.815+0000 7f94ff6ad640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94f806ca50 0x7f94f815fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:58.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.815+0000 7f94ff6ad640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f94f80b34c0 con 0x7f94f806c670 2024-03-29T04:18:58.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.815+0000 7f94ff6ad640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f94f806e5c0 con 0x7f94f80b41f0 2024-03-29T04:18:58.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.815+0000 7f94ff6ad640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f94f80ccef0 con 0x7f94f806ca50 2024-03-29T04:18:58.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.815+0000 7f94fd449640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f94f806c670 0x7f94f80cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:58.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.815+0000 7f94fd449640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f94f806c670 0x7f94f80cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44240/0 (socket says 172.21.15.22:44240) 2024-03-29T04:18:58.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.815+0000 7f94fd449640 1 -- 172.21.15.22:0/1393527403 learned_addr learned my addr 172.21.15.22:0/1393527403 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:58.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.815+0000 7f94fd449640 1 -- 172.21.15.22:0/1393527403 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94f806ca50 msgr2=0x7f94f815fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:58.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.815+0000 7f94fd449640 1 --2- 172.21.15.22:0/1393527403 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94f806ca50 0x7f94f815fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:58.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.815+0000 7f94fd449640 1 -- 172.21.15.22:0/1393527403 >> v2:172.21.15.151:3300/0 conn(0x7f94f80b41f0 msgr2=0x7f94f806e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:58.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.815+0000 7f94fcc48640 1 --2- 172.21.15.22:0/1393527403 >> v2:172.21.15.151:3300/0 conn(0x7f94f80b41f0 0x7f94f806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:58.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.815+0000 7f94fd449640 1 --2- 172.21.15.22:0/1393527403 >> v2:172.21.15.151:3300/0 conn(0x7f94f80b41f0 0x7f94f806e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:58.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.815+0000 7f94fd449640 1 -- 172.21.15.22:0/1393527403 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f80b3710 con 0x7f94f806c670 2024-03-29T04:18:58.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.815+0000 7f94fcc48640 1 --2- 172.21.15.22:0/1393527403 >> v2:172.21.15.151:3300/0 conn(0x7f94f80b41f0 0x7f94f806e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:18:58.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94fd449640 1 --2- 172.21.15.22:0/1393527403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94f806c670 0x7f94f80cc910 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f94f806ebe0 tx=0x7f94f001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb325e7962eb4f6c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:58.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94effff640 1 -- 172.21.15.22:0/1393527403 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f00b0070 con 0x7f94f806c670 2024-03-29T04:18:58.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94effff640 1 -- 172.21.15.22:0/1393527403 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94f00ab600 con 0x7f94f806c670 2024-03-29T04:18:58.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94effff640 1 -- 172.21.15.22:0/1393527403 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f00ab9d0 con 0x7f94f806c670 2024-03-29T04:18:58.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94ff6ad640 1 -- 172.21.15.22:0/1393527403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94f806c670 msgr2=0x7f94f80cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:58.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94ff6ad640 1 --2- 172.21.15.22:0/1393527403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94f806c670 0x7f94f80cc910 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f94f806ebe0 tx=0x7f94f001c260 comp rx=0 tx=0).stop 2024-03-29T04:18:58.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94ff6ad640 1 -- 172.21.15.22:0/1393527403 shutdown_connections 2024-03-29T04:18:58.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94ff6ad640 1 --2- 172.21.15.22:0/1393527403 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94f806ca50 0x7f94f815fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:58.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94ff6ad640 1 --2- 172.21.15.22:0/1393527403 >> v2:172.21.15.151:3300/0 conn(0x7f94f80b41f0 0x7f94f806e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:58.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94ff6ad640 1 --2- 172.21.15.22:0/1393527403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94f806c670 0x7f94f80cc910 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:58.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94ff6ad640 1 -- 172.21.15.22:0/1393527403 >> 172.21.15.22:0/1393527403 conn(0x7f94f80cfee0 msgr2=0x7f94f80d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:58.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94ff6ad640 1 -- 172.21.15.22:0/1393527403 shutdown_connections 2024-03-29T04:18:58.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94ff6ad640 1 -- 172.21.15.22:0/1393527403 wait complete. 2024-03-29T04:18:58.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94ff6ad640 1 Processor -- start 2024-03-29T04:18:58.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94ff6ad640 1 -- start start 2024-03-29T04:18:58.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94ff6ad640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94f806c670 0x7f94f8159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:58.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94ff6ad640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94f806ca50 0x7f94f81faeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:58.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94ff6ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94f80b41f0 0x7f94f813ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:58.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94ff6ad640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f94f806cfe0 con 0x7f94f80b41f0 2024-03-29T04:18:58.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94ff6ad640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f94f806ce60 con 0x7f94f806ca50 2024-03-29T04:18:58.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94ff6ad640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f94f80ce5b0 con 0x7f94f806c670 2024-03-29T04:18:58.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94fdc4a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94f80b41f0 0x7f94f813ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:58.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94fdc4a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94f80b41f0 0x7f94f813ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44252/0 (socket says 172.21.15.22:44252) 2024-03-29T04:18:58.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94fdc4a640 1 -- 172.21.15.22:0/635060936 learned_addr learned my addr 172.21.15.22:0/635060936 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:58.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94fcc48640 1 --2- 172.21.15.22:0/635060936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94f806ca50 0x7f94f81faeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:58.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94fd449640 1 --2- 172.21.15.22:0/635060936 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94f806c670 0x7f94f8159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:58.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94fdc4a640 1 -- 172.21.15.22:0/635060936 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94f806c670 msgr2=0x7f94f8159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:58.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94fdc4a640 1 --2- 172.21.15.22:0/635060936 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94f806c670 0x7f94f8159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:58.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94fdc4a640 1 -- 172.21.15.22:0/635060936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94f806ca50 msgr2=0x7f94f81faeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:58.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94fdc4a640 1 --2- 172.21.15.22:0/635060936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94f806ca50 0x7f94f81faeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:58.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94fdc4a640 1 -- 172.21.15.22:0/635060936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f81fa410 con 0x7f94f80b41f0 2024-03-29T04:18:58.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.819+0000 7f94fdc4a640 1 --2- 172.21.15.22:0/635060936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94f80b41f0 0x7f94f813ae30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f94f400b970 tx=0x7f94f400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:58.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.823+0000 7f94edffb640 1 -- 172.21.15.22:0/635060936 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f4019070 con 0x7f94f80b41f0 2024-03-29T04:18:58.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.823+0000 7f94edffb640 1 -- 172.21.15.22:0/635060936 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94f400cb60 con 0x7f94f80b41f0 2024-03-29T04:18:58.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.823+0000 7f94ff6ad640 1 -- 172.21.15.22:0/635060936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f815be40 con 0x7f94f80b41f0 2024-03-29T04:18:58.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.823+0000 7f94edffb640 1 -- 172.21.15.22:0/635060936 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f4014440 con 0x7f94f80b41f0 2024-03-29T04:18:58.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.823+0000 7f94ff6ad640 1 -- 172.21.15.22:0/635060936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f81fa990 con 0x7f94f80b41f0 2024-03-29T04:18:58.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.823+0000 7f94ff6ad640 1 -- 172.21.15.22:0/635060936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94f81fab90 con 0x7f94f80b41f0 2024-03-29T04:18:58.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.823+0000 7f94edffb640 1 -- 172.21.15.22:0/635060936 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f94f401c070 con 0x7f94f80b41f0 2024-03-29T04:18:58.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.823+0000 7f94edffb640 1 --2- 172.21.15.22:0/635060936 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f94dc067fc0 0x7f94dc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:58.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.823+0000 7f94edffb640 1 -- 172.21.15.22:0/635060936 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f94f4086340 con 0x7f94f80b41f0 2024-03-29T04:18:58.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.823+0000 7f94fd449640 1 --2- 172.21.15.22:0/635060936 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f94dc067fc0 0x7f94dc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:58.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.823+0000 7f94fd449640 1 --2- 172.21.15.22:0/635060936 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f94dc067fc0 0x7f94dc06a480 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f94f00ad000 tx=0x7f94f00a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:58.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.827+0000 7f94edffb640 1 -- 172.21.15.22:0/635060936 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f94f81fab90 con 0x7f94f80b41f0 2024-03-29T04:18:58.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.991+0000 7f94ff6ad640 1 -- 172.21.15.22:0/635060936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94f815a1b0 con 0x7f94f80b41f0 2024-03-29T04:18:58.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.991+0000 7f94edffb640 1 -- 172.21.15.22:0/635060936 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16955 (secure 0 0 0) 0x7f94f40569f0 con 0x7f94f80b41f0 2024-03-29T04:18:58.974 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:58.974 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":39,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:18:57.989672+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:18:56.336876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:18:56.950670+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:58.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.995+0000 7f94ff6ad640 1 -- 172.21.15.22:0/635060936 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f94dc067fc0 msgr2=0x7f94dc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:58.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.995+0000 7f94ff6ad640 1 --2- 172.21.15.22:0/635060936 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f94dc067fc0 0x7f94dc06a480 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f94f00ad000 tx=0x7f94f00a6000 comp rx=0 tx=0).stop 2024-03-29T04:18:58.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.995+0000 7f94ff6ad640 1 -- 172.21.15.22:0/635060936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94f80b41f0 msgr2=0x7f94f813ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:58.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.995+0000 7f94ff6ad640 1 --2- 172.21.15.22:0/635060936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94f80b41f0 0x7f94f813ae30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f94f400b970 tx=0x7f94f400be30 comp rx=0 tx=0).stop 2024-03-29T04:18:58.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.995+0000 7f94ff6ad640 1 -- 172.21.15.22:0/635060936 shutdown_connections 2024-03-29T04:18:58.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.995+0000 7f94ff6ad640 1 --2- 172.21.15.22:0/635060936 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f94dc067fc0 0x7f94dc06a480 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:58.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.995+0000 7f94ff6ad640 1 --2- 172.21.15.22:0/635060936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94f80b41f0 0x7f94f813ae30 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:58.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.995+0000 7f94ff6ad640 1 --2- 172.21.15.22:0/635060936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94f806ca50 0x7f94f81faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:58.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.995+0000 7f94ff6ad640 1 --2- 172.21.15.22:0/635060936 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f94f806c670 0x7f94f8159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:58.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.995+0000 7f94ff6ad640 1 -- 172.21.15.22:0/635060936 >> 172.21.15.22:0/635060936 conn(0x7f94f80cfee0 msgr2=0x7f94f815dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:58.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.995+0000 7f94ff6ad640 1 -- 172.21.15.22:0/635060936 shutdown_connections 2024-03-29T04:18:58.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:58.995+0000 7f94ff6ad640 1 -- 172.21.15.22:0/635060936 wait complete. 2024-03-29T04:18:58.994 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:18:59.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.139+0000 7fb0a8fa4640 1 Processor -- start 2024-03-29T04:18:59.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.139+0000 7fb0a8fa4640 1 -- start start 2024-03-29T04:18:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.139+0000 7fb0a8fa4640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb0a406c670 0x7fb0a40cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.139+0000 7fb0a8fa4640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fb0a406ca50 0x7fb0a40ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.139+0000 7fb0a8fa4640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0a40ccef0 con 0x7fb0a40b41f0 2024-03-29T04:18:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.139+0000 7fb0a8fa4640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fb0a40b34c0 con 0x7fb0a406c670 2024-03-29T04:18:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.139+0000 7fb0a8fa4640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fb0a40b3710 con 0x7fb0a406ca50 2024-03-29T04:18:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.139+0000 7fb0a259c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb0a406c670 0x7fb0a40cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.139+0000 7fb0a259c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb0a406c670 0x7fb0a40cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44264/0 (socket says 172.21.15.22:44264) 2024-03-29T04:18:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.139+0000 7fb0a259c640 1 -- 172.21.15.22:0/3981671944 learned_addr learned my addr 172.21.15.22:0/3981671944 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.139+0000 7fb0a2d9d640 1 --2- 172.21.15.22:0/3981671944 >> v2:172.21.15.151:3300/0 conn(0x7fb0a406ca50 0x7fb0a40ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.139+0000 7fb0a259c640 1 -- 172.21.15.22:0/3981671944 >> v2:172.21.15.151:3300/0 conn(0x7fb0a406ca50 msgr2=0x7fb0a40ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.139+0000 7fb0a259c640 1 --2- 172.21.15.22:0/3981671944 >> v2:172.21.15.151:3300/0 conn(0x7fb0a406ca50 0x7fb0a40ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.139+0000 7fb0a259c640 1 -- 172.21.15.22:0/3981671944 >> v1:172.21.15.151:6789/0 conn(0x7fb0a40b41f0 legacy=0x7fb0a406e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.139+0000 7fb0a259c640 1 -- 172.21.15.22:0/3981671944 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a40b39f0 con 0x7fb0a406c670 2024-03-29T04:18:59.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a259c640 1 --2- 172.21.15.22:0/3981671944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0a406c670 0x7fb0a40cc910 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb0a40d0e40 tx=0x7fb09401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d70f8bc9228c41 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:59.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a159a640 1 -- 172.21.15.22:0/3981671944 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0940ab430 con 0x7fb0a406c670 2024-03-29T04:18:59.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a159a640 1 -- 172.21.15.22:0/3981671944 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0940b0070 con 0x7fb0a406c670 2024-03-29T04:18:59.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a159a640 1 -- 172.21.15.22:0/3981671944 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0940ab7f0 con 0x7fb0a406c670 2024-03-29T04:18:59.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a8fa4640 1 -- 172.21.15.22:0/3981671944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0a406c670 msgr2=0x7fb0a40cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:59.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a8fa4640 1 --2- 172.21.15.22:0/3981671944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0a406c670 0x7fb0a40cc910 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb0a40d0e40 tx=0x7fb09401c520 comp rx=0 tx=0).stop 2024-03-29T04:18:59.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a8fa4640 1 -- 172.21.15.22:0/3981671944 shutdown_connections 2024-03-29T04:18:59.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a8fa4640 1 --2- 172.21.15.22:0/3981671944 >> v2:172.21.15.151:3300/0 conn(0x7fb0a406ca50 0x7fb0a40ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:59.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a8fa4640 1 --2- 172.21.15.22:0/3981671944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0a406c670 0x7fb0a40cc910 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:59.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a8fa4640 1 -- 172.21.15.22:0/3981671944 >> 172.21.15.22:0/3981671944 conn(0x7fb0a40cfee0 msgr2=0x7fb0a40d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:59.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a8fa4640 1 -- 172.21.15.22:0/3981671944 shutdown_connections 2024-03-29T04:18:59.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a8fa4640 1 -- 172.21.15.22:0/3981671944 wait complete. 2024-03-29T04:18:59.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a8fa4640 1 Processor -- start 2024-03-29T04:18:59.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a8fa4640 1 -- start start 2024-03-29T04:18:59.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a8fa4640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0a406c670 0x7fb0a41354c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:59.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a8fa4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0a406ca50 0x7fb0a415b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:59.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a8fa4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0a40b41f0 0x7fb0a41f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:59.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a8fa4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a4169070 con 0x7fb0a40b41f0 2024-03-29T04:18:59.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a8fa4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a4168ef0 con 0x7fb0a406ca50 2024-03-29T04:18:59.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a8fa4640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb0a41691f0 con 0x7fb0a406c670 2024-03-29T04:18:59.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a2d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0a40b41f0 0x7fb0a41f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:59.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a2d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0a40b41f0 0x7fb0a41f56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44272/0 (socket says 172.21.15.22:44272) 2024-03-29T04:18:59.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a2d9d640 1 -- 172.21.15.22:0/67343253 learned_addr learned my addr 172.21.15.22:0/67343253 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:59.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a259c640 1 --2- 172.21.15.22:0/67343253 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0a406c670 0x7fb0a41354c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:59.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a1d9b640 1 --2- 172.21.15.22:0/67343253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0a406ca50 0x7fb0a415b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:59.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a2d9d640 1 -- 172.21.15.22:0/67343253 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0a406c670 msgr2=0x7fb0a41354c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:59.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a2d9d640 1 --2- 172.21.15.22:0/67343253 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0a406c670 0x7fb0a41354c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:59.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a2d9d640 1 -- 172.21.15.22:0/67343253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0a406ca50 msgr2=0x7fb0a415b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:59.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a2d9d640 1 --2- 172.21.15.22:0/67343253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0a406ca50 0x7fb0a415b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:59.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a2d9d640 1 -- 172.21.15.22:0/67343253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a4135a00 con 0x7fb0a40b41f0 2024-03-29T04:18:59.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.143+0000 7fb0a2d9d640 1 --2- 172.21.15.22:0/67343253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0a40b41f0 0x7fb0a41f56c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb09c007e90 tx=0x7fb09c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:59.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.147+0000 7fb092ffd640 1 -- 172.21.15.22:0/67343253 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c014420 con 0x7fb0a40b41f0 2024-03-29T04:18:59.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.147+0000 7fb092ffd640 1 -- 172.21.15.22:0/67343253 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb09c019070 con 0x7fb0a40b41f0 2024-03-29T04:18:59.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.147+0000 7fb0a8fa4640 1 -- 172.21.15.22:0/67343253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a415bee0 con 0x7fb0a40b41f0 2024-03-29T04:18:59.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.147+0000 7fb092ffd640 1 -- 172.21.15.22:0/67343253 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c014800 con 0x7fb0a40b41f0 2024-03-29T04:18:59.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.147+0000 7fb0a8fa4640 1 -- 172.21.15.22:0/67343253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a4136150 con 0x7fb0a40b41f0 2024-03-29T04:18:59.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.147+0000 7fb0a8fa4640 1 -- 172.21.15.22:0/67343253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a4068890 con 0x7fb0a40b41f0 2024-03-29T04:18:59.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.147+0000 7fb092ffd640 1 -- 172.21.15.22:0/67343253 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb09c0149a0 con 0x7fb0a40b41f0 2024-03-29T04:18:59.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.147+0000 7fb092ffd640 1 --2- 172.21.15.22:0/67343253 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb084067fc0 0x7fb08406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:59.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.147+0000 7fb092ffd640 1 -- 172.21.15.22:0/67343253 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fb09c086e30 con 0x7fb0a40b41f0 2024-03-29T04:18:59.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.147+0000 7fb0a259c640 1 --2- 172.21.15.22:0/67343253 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb084067fc0 0x7fb08406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:59.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.147+0000 7fb0a259c640 1 --2- 172.21.15.22:0/67343253 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb084067fc0 0x7fb08406a480 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb0940af000 tx=0x7fb0940a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:59.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.151+0000 7fb092ffd640 1 -- 172.21.15.22:0/67343253 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fb09c011260 con 0x7fb0a40b41f0 2024-03-29T04:18:59.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.315+0000 7fb0a8fa4640 1 -- 172.21.15.22:0/67343253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0a40b32a0 con 0x7fb0a40b41f0 2024-03-29T04:18:59.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.319+0000 7fb092ffd640 1 -- 172.21.15.22:0/67343253 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16955 (secure 0 0 0) 0x7fb09c0574e0 con 0x7fb0a40b41f0 2024-03-29T04:18:59.299 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:18:59.300 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":39,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:18:57.989672+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:18:56.336876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:18:56.950670+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:18:59.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.319+0000 7fb0a8fa4640 1 -- 172.21.15.22:0/67343253 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb084067fc0 msgr2=0x7fb08406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:59.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.319+0000 7fb0a8fa4640 1 --2- 172.21.15.22:0/67343253 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb084067fc0 0x7fb08406a480 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb0940af000 tx=0x7fb0940a7000 comp rx=0 tx=0).stop 2024-03-29T04:18:59.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.319+0000 7fb0a8fa4640 1 -- 172.21.15.22:0/67343253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0a40b41f0 msgr2=0x7fb0a41f56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:59.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.319+0000 7fb0a8fa4640 1 --2- 172.21.15.22:0/67343253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0a40b41f0 0x7fb0a41f56c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb09c007e90 tx=0x7fb09c00e4b0 comp rx=0 tx=0).stop 2024-03-29T04:18:59.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.319+0000 7fb0a8fa4640 1 -- 172.21.15.22:0/67343253 shutdown_connections 2024-03-29T04:18:59.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.319+0000 7fb0a8fa4640 1 --2- 172.21.15.22:0/67343253 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb084067fc0 0x7fb08406a480 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:59.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.319+0000 7fb0a8fa4640 1 --2- 172.21.15.22:0/67343253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0a40b41f0 0x7fb0a41f56c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:59.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.319+0000 7fb0a8fa4640 1 --2- 172.21.15.22:0/67343253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0a406ca50 0x7fb0a415b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:59.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.319+0000 7fb0a8fa4640 1 --2- 172.21.15.22:0/67343253 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb0a406c670 0x7fb0a41354c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:59.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.319+0000 7fb0a8fa4640 1 -- 172.21.15.22:0/67343253 >> 172.21.15.22:0/67343253 conn(0x7fb0a40cfee0 msgr2=0x7fb0a4160500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:59.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.319+0000 7fb0a8fa4640 1 -- 172.21.15.22:0/67343253 shutdown_connections 2024-03-29T04:18:59.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.319+0000 7fb0a8fa4640 1 -- 172.21.15.22:0/67343253 wait complete. 2024-03-29T04:18:59.319 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-29T04:18:59.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.459+0000 7f617bb8c640 1 Processor -- start 2024-03-29T04:18:59.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.459+0000 7f617bb8c640 1 -- start start 2024-03-29T04:18:59.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.463+0000 7f617bb8c640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f617413f140 0x7f61741606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:59.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.463+0000 7f617bb8c640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f617415ff40 0x7f617406c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:59.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.463+0000 7f617bb8c640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f617406d710 con 0x7f617406cbb0 2024-03-29T04:18:59.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.463+0000 7f617bb8c640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f617406d000 con 0x7f617413f140 2024-03-29T04:18:59.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.463+0000 7f617bb8c640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f617406e320 con 0x7f617415ff40 2024-03-29T04:18:59.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.463+0000 7f6179928640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f617413f140 0x7f61741606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:59.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.463+0000 7f6179127640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f617415ff40 0x7f617406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:59.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.463+0000 7f6179928640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f617413f140 0x7f61741606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:51346/0 (socket says 172.21.15.22:51346) 2024-03-29T04:18:59.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.463+0000 7f617a129640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f617406cbb0 0x7f6174141a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.22:39820/0 (socket says 172.21.15.22:39820) 2024-03-29T04:18:59.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.463+0000 7f6179928640 1 -- 172.21.15.22:0/1301614285 learned_addr learned my addr 172.21.15.22:0/1301614285 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:59.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.463+0000 7f6179928640 1 -- 172.21.15.22:0/1301614285 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f617415ff40 msgr2=0x7f617406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:18:59.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.463+0000 7f6179928640 1 --2- 172.21.15.22:0/1301614285 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f617415ff40 0x7f617406c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:59.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.463+0000 7f6179928640 1 -- 172.21.15.22:0/1301614285 >> v1:172.21.15.151:6789/0 conn(0x7f617406cbb0 legacy=0x7f6174141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:59.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.463+0000 7f6179928640 1 -- 172.21.15.22:0/1301614285 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617406d9f0 con 0x7f617413f140 2024-03-29T04:18:59.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f6179928640 1 --2- 172.21.15.22:0/1301614285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f617413f140 0x7f61741606d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6170009e50 tx=0x7f617001ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f27154e99339d2d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:59.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f6178926640 1 -- 172.21.15.22:0/1301614285 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61700a58d0 con 0x7f617413f140 2024-03-29T04:18:59.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f6178926640 1 -- 172.21.15.22:0/1301614285 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61700a5a70 con 0x7f617413f140 2024-03-29T04:18:59.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f6178926640 1 -- 172.21.15.22:0/1301614285 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61700a5e40 con 0x7f617413f140 2024-03-29T04:18:59.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f617bb8c640 1 -- 172.21.15.22:0/1301614285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f617413f140 msgr2=0x7f61741606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:59.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f617bb8c640 1 --2- 172.21.15.22:0/1301614285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f617413f140 0x7f61741606d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6170009e50 tx=0x7f617001ca00 comp rx=0 tx=0).stop 2024-03-29T04:18:59.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f617bb8c640 1 -- 172.21.15.22:0/1301614285 shutdown_connections 2024-03-29T04:18:59.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f617bb8c640 1 --2- 172.21.15.22:0/1301614285 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f617415ff40 0x7f617406c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:59.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f617bb8c640 1 --2- 172.21.15.22:0/1301614285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f617413f140 0x7f61741606d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:59.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f617bb8c640 1 -- 172.21.15.22:0/1301614285 >> 172.21.15.22:0/1301614285 conn(0x7f61740d00a0 msgr2=0x7f61740d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:18:59.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f617bb8c640 1 -- 172.21.15.22:0/1301614285 shutdown_connections 2024-03-29T04:18:59.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f617bb8c640 1 -- 172.21.15.22:0/1301614285 wait complete. 2024-03-29T04:18:59.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f617bb8c640 1 Processor -- start 2024-03-29T04:18:59.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f617bb8c640 1 -- start start 2024-03-29T04:18:59.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f617bb8c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f617406cbb0 0x7f6174073110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:59.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f617bb8c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f617413f140 0x7f61741525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:59.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f617bb8c640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f617415ff40 0x7f6174153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:59.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f617bb8c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6174160d90 con 0x7f617413f140 2024-03-29T04:18:59.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f617bb8c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6174160c10 con 0x7f617406cbb0 2024-03-29T04:18:59.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f617bb8c640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f6174168ff0 con 0x7f617415ff40 2024-03-29T04:18:59.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f6179127640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f617413f140 0x7f61741525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:59.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f6179127640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f617413f140 0x7f61741525a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44274/0 (socket says 172.21.15.22:44274) 2024-03-29T04:18:59.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f6179127640 1 -- 172.21.15.22:0/3987883522 learned_addr learned my addr 172.21.15.22:0/3987883522 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:18:59.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f6179928640 1 --2- 172.21.15.22:0/3987883522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f617406cbb0 0x7f6174073110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:59.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.467+0000 7f617a129640 1 --2- 172.21.15.22:0/3987883522 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f617415ff40 0x7f6174153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:59.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.471+0000 7f6179127640 1 -- 172.21.15.22:0/3987883522 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f617415ff40 msgr2=0x7f6174153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:59.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.471+0000 7f6179127640 1 --2- 172.21.15.22:0/3987883522 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f617415ff40 0x7f6174153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:59.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.471+0000 7f6179127640 1 -- 172.21.15.22:0/3987883522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f617406cbb0 msgr2=0x7f6174073110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:18:59.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.471+0000 7f6179127640 1 --2- 172.21.15.22:0/3987883522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f617406cbb0 0x7f6174073110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:18:59.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.471+0000 7f6179127640 1 -- 172.21.15.22:0/3987883522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6174154470 con 0x7f617413f140 2024-03-29T04:18:59.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.471+0000 7f6179928640 1 --2- 172.21.15.22:0/3987883522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f617406cbb0 0x7f6174073110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:18:59.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.471+0000 7f6179127640 1 --2- 172.21.15.22:0/3987883522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f617413f140 0x7f61741525a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f616400e930 tx=0x7f616400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:59.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.471+0000 7f616a7fc640 1 -- 172.21.15.22:0/3987883522 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164017070 con 0x7f617413f140 2024-03-29T04:18:59.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.471+0000 7f616a7fc640 1 -- 172.21.15.22:0/3987883522 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f616400ca00 con 0x7f617413f140 2024-03-29T04:18:59.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.471+0000 7f617bb8c640 1 -- 172.21.15.22:0/3987883522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6174072350 con 0x7f617413f140 2024-03-29T04:18:59.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.471+0000 7f617bb8c640 1 -- 172.21.15.22:0/3987883522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6174072910 con 0x7f617413f140 2024-03-29T04:18:59.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.471+0000 7f616a7fc640 1 -- 172.21.15.22:0/3987883522 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616400cdb0 con 0x7f617413f140 2024-03-29T04:18:59.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.471+0000 7f617bb8c640 1 -- 172.21.15.22:0/3987883522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f615c0052a0 con 0x7f617413f140 2024-03-29T04:18:59.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.471+0000 7f616a7fc640 1 -- 172.21.15.22:0/3987883522 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f616401a070 con 0x7f617413f140 2024-03-29T04:18:59.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.471+0000 7f616a7fc640 1 --2- 172.21.15.22:0/3987883522 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6158067fc0 0x7f615806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:18:59.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.471+0000 7f616a7fc640 1 -- 172.21.15.22:0/3987883522 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f6164086d30 con 0x7f617413f140 2024-03-29T04:18:59.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.471+0000 7f6179928640 1 --2- 172.21.15.22:0/3987883522 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6158067fc0 0x7f615806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:18:59.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.471+0000 7f6179928640 1 --2- 172.21.15.22:0/3987883522 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6158067fc0 0x7f615806a480 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f61700ae000 tx=0x7f61700a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:18:59.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.475+0000 7f616a7fc640 1 -- 172.21.15.22:0/3987883522 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f61640574a0 con 0x7f617413f140 2024-03-29T04:18:59.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:18:59.683+0000 7f617bb8c640 1 -- 172.21.15.22:0/3987883522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f615c005860 con 0x7f617413f140 2024-03-29T04:19:00.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.679+0000 7f616a7fc640 1 -- 172.21.15.22:0/3987883522 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f6164056da0 con 0x7f617413f140 2024-03-29T04:19:00.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.683+0000 7f617bb8c640 1 -- 172.21.15.22:0/3987883522 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6158067fc0 msgr2=0x7f615806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:00.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.683+0000 7f617bb8c640 1 --2- 172.21.15.22:0/3987883522 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6158067fc0 0x7f615806a480 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f61700ae000 tx=0x7f61700a7070 comp rx=0 tx=0).stop 2024-03-29T04:19:00.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.683+0000 7f617bb8c640 1 -- 172.21.15.22:0/3987883522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f617413f140 msgr2=0x7f61741525a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:00.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.683+0000 7f617bb8c640 1 --2- 172.21.15.22:0/3987883522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f617413f140 0x7f61741525a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f616400e930 tx=0x7f616400edf0 comp rx=0 tx=0).stop 2024-03-29T04:19:00.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.683+0000 7f617bb8c640 1 -- 172.21.15.22:0/3987883522 shutdown_connections 2024-03-29T04:19:00.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.683+0000 7f617bb8c640 1 --2- 172.21.15.22:0/3987883522 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f617415ff40 0x7f6174153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:00.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.683+0000 7f617bb8c640 1 --2- 172.21.15.22:0/3987883522 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6158067fc0 0x7f615806a480 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:00.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.683+0000 7f617bb8c640 1 --2- 172.21.15.22:0/3987883522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f617413f140 0x7f61741525a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:00.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.683+0000 7f617bb8c640 1 --2- 172.21.15.22:0/3987883522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f617406cbb0 0x7f6174073110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:00.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.683+0000 7f617bb8c640 1 -- 172.21.15.22:0/3987883522 >> 172.21.15.22:0/3987883522 conn(0x7f61740d00a0 msgr2=0x7f61741411b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:00.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.683+0000 7f617bb8c640 1 -- 172.21.15.22:0/3987883522 shutdown_connections 2024-03-29T04:19:00.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.683+0000 7f617bb8c640 1 -- 172.21.15.22:0/3987883522 wait complete. 2024-03-29T04:19:00.687 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-29T04:19:00.687 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-29T04:19:00.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.831+0000 7f572de15640 1 Processor -- start 2024-03-29T04:19:00.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.831+0000 7f572de15640 1 -- start start 2024-03-29T04:19:00.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.831+0000 7f572de15640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f5728158b40 0x7f57280b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:00.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.831+0000 7f572de15640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f57280b5530 0x7f57280cc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:00.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.831+0000 7f572de15640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f57280ccc90 0x7f57280d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:00.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.831+0000 7f572de15640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f5728162d00 con 0x7f57280b5530 2024-03-29T04:19:00.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.831+0000 7f572de15640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f57280cd0d0 con 0x7f5728158b40 2024-03-29T04:19:00.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.831+0000 7f572de15640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5728158f20 con 0x7f57280ccc90 2024-03-29T04:19:00.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.831+0000 7f5726ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f57280b5530 0x7f57280cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:00.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.831+0000 7f5726ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f57280b5530 0x7f57280cc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44286/0 (socket says 172.21.15.22:44286) 2024-03-29T04:19:00.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.831+0000 7f5726ffd640 1 -- 172.21.15.22:0/128486880 learned_addr learned my addr 172.21.15.22:0/128486880 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:00.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.831+0000 7f57277fe640 1 --2- 172.21.15.22:0/128486880 >> v2:172.21.15.151:3300/0 conn(0x7f5728158b40 0x7f57280b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:00.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.831+0000 7f5727fff640 1 --2- 172.21.15.22:0/128486880 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f57280ccc90 0x7f57280d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:00.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.831+0000 7f5726ffd640 1 -- 172.21.15.22:0/128486880 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f57280ccc90 msgr2=0x7f57280d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:00.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.831+0000 7f5726ffd640 1 --2- 172.21.15.22:0/128486880 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f57280ccc90 0x7f57280d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:00.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.831+0000 7f5726ffd640 1 -- 172.21.15.22:0/128486880 >> v2:172.21.15.151:3300/0 conn(0x7f5728158b40 msgr2=0x7f57280b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:00.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.831+0000 7f5726ffd640 1 --2- 172.21.15.22:0/128486880 >> v2:172.21.15.151:3300/0 conn(0x7f5728158b40 0x7f57280b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:00.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.831+0000 7f5726ffd640 1 -- 172.21.15.22:0/128486880 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57281620f0 con 0x7f57280b5530 2024-03-29T04:19:00.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.831+0000 7f57277fe640 1 --2- 172.21.15.22:0/128486880 >> v2:172.21.15.151:3300/0 conn(0x7f5728158b40 0x7f57280b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:19:00.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.831+0000 7f5727fff640 1 --2- 172.21.15.22:0/128486880 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f57280ccc90 0x7f57280d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:19:00.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.835+0000 7f5726ffd640 1 --2- 172.21.15.22:0/128486880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57280b5530 0x7f57280cc750 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f57280b4860 tx=0x7f571c01c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ad4197f8a87305 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:00.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.835+0000 7f57267fc640 1 -- 172.21.15.22:0/128486880 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c0b0070 con 0x7f57280b5530 2024-03-29T04:19:00.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.835+0000 7f57267fc640 1 -- 172.21.15.22:0/128486880 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f571c0ab600 con 0x7f57280b5530 2024-03-29T04:19:00.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.835+0000 7f57267fc640 1 -- 172.21.15.22:0/128486880 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c0ab9d0 con 0x7f57280b5530 2024-03-29T04:19:00.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.839+0000 7f572de15640 1 -- 172.21.15.22:0/128486880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57280b5530 msgr2=0x7f57280cc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:00.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.839+0000 7f572de15640 1 --2- 172.21.15.22:0/128486880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57280b5530 0x7f57280cc750 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f57280b4860 tx=0x7f571c01c1a0 comp rx=0 tx=0).stop 2024-03-29T04:19:00.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.839+0000 7f572de15640 1 -- 172.21.15.22:0/128486880 shutdown_connections 2024-03-29T04:19:00.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.839+0000 7f572de15640 1 --2- 172.21.15.22:0/128486880 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f57280ccc90 0x7f57280d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:00.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.839+0000 7f572de15640 1 --2- 172.21.15.22:0/128486880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57280b5530 0x7f57280cc750 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:00.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.839+0000 7f572de15640 1 --2- 172.21.15.22:0/128486880 >> v2:172.21.15.151:3300/0 conn(0x7f5728158b40 0x7f57280b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:00.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.839+0000 7f572de15640 1 -- 172.21.15.22:0/128486880 >> 172.21.15.22:0/128486880 conn(0x7f57280b3cf0 msgr2=0x7f57280b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:00.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.839+0000 7f572de15640 1 -- 172.21.15.22:0/128486880 shutdown_connections 2024-03-29T04:19:00.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.839+0000 7f572de15640 1 -- 172.21.15.22:0/128486880 wait complete. 2024-03-29T04:19:00.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.839+0000 7f572de15640 1 Processor -- start 2024-03-29T04:19:00.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.839+0000 7f572de15640 1 -- start start 2024-03-29T04:19:00.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f572de15640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57280b5530 0x7f57280c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:00.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f572de15640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57280ccc90 0x7f5728152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:00.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f572de15640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5728158b40 0x7f57281528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:00.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f572de15640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f57280d1eb0 con 0x7f57280ccc90 2024-03-29T04:19:00.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f572de15640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f57280d1d30 con 0x7f57280b5530 2024-03-29T04:19:00.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f572de15640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f57280b6c50 con 0x7f5728158b40 2024-03-29T04:19:00.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f5726ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57280ccc90 0x7f5728152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:00.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f5726ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57280ccc90 0x7f5728152380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44290/0 (socket says 172.21.15.22:44290) 2024-03-29T04:19:00.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f5726ffd640 1 -- 172.21.15.22:0/3397765731 learned_addr learned my addr 172.21.15.22:0/3397765731 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:00.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f5727fff640 1 --2- 172.21.15.22:0/3397765731 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5728158b40 0x7f57281528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:00.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f57277fe640 1 --2- 172.21.15.22:0/3397765731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57280b5530 0x7f57280c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:00.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f5726ffd640 1 -- 172.21.15.22:0/3397765731 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5728158b40 msgr2=0x7f57281528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:00.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f5726ffd640 1 --2- 172.21.15.22:0/3397765731 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5728158b40 0x7f57281528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:00.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f5726ffd640 1 -- 172.21.15.22:0/3397765731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57280b5530 msgr2=0x7f57280c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:00.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f5726ffd640 1 --2- 172.21.15.22:0/3397765731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57280b5530 0x7f57280c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:00.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f5726ffd640 1 -- 172.21.15.22:0/3397765731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57281f74c0 con 0x7f57280ccc90 2024-03-29T04:19:00.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f57277fe640 1 --2- 172.21.15.22:0/3397765731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57280b5530 0x7f57280c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:19:00.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f5726ffd640 1 --2- 172.21.15.22:0/3397765731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57280ccc90 0x7f5728152380 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f571c01a850 tx=0x7f571c018f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:00.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f570ffff640 1 -- 172.21.15.22:0/3397765731 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c0b9070 con 0x7f57280ccc90 2024-03-29T04:19:00.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f570ffff640 1 -- 172.21.15.22:0/3397765731 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f571c0b4400 con 0x7f57280ccc90 2024-03-29T04:19:00.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f572de15640 1 -- 172.21.15.22:0/3397765731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5728153020 con 0x7f57280ccc90 2024-03-29T04:19:00.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f572de15640 1 -- 172.21.15.22:0/3397765731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57281535e0 con 0x7f57280ccc90 2024-03-29T04:19:00.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f570ffff640 1 -- 172.21.15.22:0/3397765731 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c0b0040 con 0x7f57280ccc90 2024-03-29T04:19:00.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f572de15640 1 -- 172.21.15.22:0/3397765731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57080052a0 con 0x7f57280ccc90 2024-03-29T04:19:00.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.843+0000 7f570ffff640 1 -- 172.21.15.22:0/3397765731 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f571c0bc070 con 0x7f57280ccc90 2024-03-29T04:19:00.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.847+0000 7f570ffff640 1 --2- 172.21.15.22:0/3397765731 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5714067fc0 0x7f571406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:00.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.847+0000 7f570ffff640 1 -- 172.21.15.22:0/3397765731 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f571c11f980 con 0x7f57280ccc90 2024-03-29T04:19:00.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.847+0000 7f57277fe640 1 --2- 172.21.15.22:0/3397765731 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5714067fc0 0x7f571406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:00.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.847+0000 7f57277fe640 1 --2- 172.21.15.22:0/3397765731 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5714067fc0 0x7f571406a480 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f57200097a0 tx=0x7f5720008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:00.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:00.851+0000 7f570ffff640 1 -- 172.21.15.22:0/3397765731 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f571c016110 con 0x7f57280ccc90 2024-03-29T04:19:01.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.047+0000 7f572de15640 1 -- 172.21.15.22:0/3397765731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f5708005590 con 0x7f57280ccc90 2024-03-29T04:19:01.479 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:19:01.482 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:19:01.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.679+0000 7f570ffff640 1 -- 172.21.15.22:0/3397765731 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f571c0effa0 con 0x7f57280ccc90 2024-03-29T04:19:01.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.683+0000 7f572de15640 1 -- 172.21.15.22:0/3397765731 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5714067fc0 msgr2=0x7f571406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:01.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.683+0000 7f572de15640 1 --2- 172.21.15.22:0/3397765731 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5714067fc0 0x7f571406a480 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f57200097a0 tx=0x7f5720008040 comp rx=0 tx=0).stop 2024-03-29T04:19:01.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.683+0000 7f572de15640 1 -- 172.21.15.22:0/3397765731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57280ccc90 msgr2=0x7f5728152380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:01.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.683+0000 7f572de15640 1 --2- 172.21.15.22:0/3397765731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57280ccc90 0x7f5728152380 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f571c01a850 tx=0x7f571c018f40 comp rx=0 tx=0).stop 2024-03-29T04:19:01.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.683+0000 7f572de15640 1 -- 172.21.15.22:0/3397765731 shutdown_connections 2024-03-29T04:19:01.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.683+0000 7f572de15640 1 --2- 172.21.15.22:0/3397765731 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5728158b40 0x7f57281528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:01.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.683+0000 7f572de15640 1 --2- 172.21.15.22:0/3397765731 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5714067fc0 0x7f571406a480 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:01.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.683+0000 7f572de15640 1 --2- 172.21.15.22:0/3397765731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57280ccc90 0x7f5728152380 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:01.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.683+0000 7f572de15640 1 --2- 172.21.15.22:0/3397765731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57280b5530 0x7f57280c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:01.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.683+0000 7f572de15640 1 -- 172.21.15.22:0/3397765731 >> 172.21.15.22:0/3397765731 conn(0x7f57280b3cf0 msgr2=0x7f57280b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:01.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.683+0000 7f572de15640 1 -- 172.21.15.22:0/3397765731 shutdown_connections 2024-03-29T04:19:01.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.683+0000 7f572de15640 1 -- 172.21.15.22:0/3397765731 wait complete. 2024-03-29T04:19:01.686 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-29T04:19:01.686 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:19:01.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.831+0000 7f9ed1b20640 1 Processor -- start 2024-03-29T04:19:01.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.831+0000 7f9ed1b20640 1 -- start start 2024-03-29T04:19:01.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.835+0000 7f9ed1b20640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f9ecc15ec40 0x7f9ecc15f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:01.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.835+0000 7f9ed1b20640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9ecc13e820 0x7f9ecc15f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:01.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.835+0000 7f9ed1b20640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ecc06fd10 con 0x7f9ecc06ef90 2024-03-29T04:19:01.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.835+0000 7f9ed1b20640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f9ecc06fad0 con 0x7f9ecc13e820 2024-03-29T04:19:01.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.835+0000 7f9ed1b20640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f9ecc15faa0 con 0x7f9ecc15ec40 2024-03-29T04:19:01.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.835+0000 7f9ecaffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9ecc13e820 0x7f9ecc15f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:01.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.835+0000 7f9ecbfff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f9ecc06ef90 0x7f9ecc06f3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:52196/0 (socket says 172.21.15.22:52196) 2024-03-29T04:19:01.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.835+0000 7f9ecaffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9ecc13e820 0x7f9ecc15f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44304/0 (socket says 172.21.15.22:44304) 2024-03-29T04:19:01.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.835+0000 7f9ecbfff640 1 -- 172.21.15.22:0/1363722955 learned_addr learned my addr 172.21.15.22:0/1363722955 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:19:01.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.835+0000 7f9ecb7fe640 1 --2- 172.21.15.22:0/1363722955 >> v2:172.21.15.151:3300/0 conn(0x7f9ecc15ec40 0x7f9ecc15f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:01.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.835+0000 7f9ecaffd640 1 -- 172.21.15.22:0/1363722955 >> v2:172.21.15.151:3300/0 conn(0x7f9ecc15ec40 msgr2=0x7f9ecc15f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:01.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.835+0000 7f9ecaffd640 1 --2- 172.21.15.22:0/1363722955 >> v2:172.21.15.151:3300/0 conn(0x7f9ecc15ec40 0x7f9ecc15f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:01.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.835+0000 7f9eca7fc640 1 -- 172.21.15.22:0/1363722955 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 870571786 0 0) 0x7f9ecc06fd10 con 0x7f9ecc06ef90 2024-03-29T04:19:01.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.835+0000 7f9ecaffd640 1 -- 172.21.15.22:0/1363722955 >> v1:172.21.15.22:6789/0 conn(0x7f9ecc06ef90 legacy=0x7f9ecc06f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:01.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.835+0000 7f9ecaffd640 1 -- 172.21.15.22:0/1363722955 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ecc141200 con 0x7f9ecc13e820 2024-03-29T04:19:01.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.835+0000 7f9ecaffd640 1 --2- 172.21.15.22:0/1363722955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ecc13e820 0x7f9ecc15f560 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc071970 tx=0x7f9ebc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2dccacd2f9c4aea server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:01.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.835+0000 7f9eca7fc640 1 -- 172.21.15.22:0/1363722955 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc0ab650 con 0x7f9ecc13e820 2024-03-29T04:19:01.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9eca7fc640 1 -- 172.21.15.22:0/1363722955 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ebc0b0070 con 0x7f9ecc13e820 2024-03-29T04:19:01.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9eca7fc640 1 -- 172.21.15.22:0/1363722955 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc0aba10 con 0x7f9ecc13e820 2024-03-29T04:19:01.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ed1b20640 1 -- 172.21.15.22:0/1363722955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ecc13e820 msgr2=0x7f9ecc15f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:01.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ed1b20640 1 --2- 172.21.15.22:0/1363722955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ecc13e820 0x7f9ecc15f560 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc071970 tx=0x7f9ebc01c450 comp rx=0 tx=0).stop 2024-03-29T04:19:01.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ed1b20640 1 -- 172.21.15.22:0/1363722955 shutdown_connections 2024-03-29T04:19:01.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ed1b20640 1 --2- 172.21.15.22:0/1363722955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ecc13e820 0x7f9ecc15f560 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:01.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ed1b20640 1 --2- 172.21.15.22:0/1363722955 >> v2:172.21.15.151:3300/0 conn(0x7f9ecc15ec40 0x7f9ecc15f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:01.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ed1b20640 1 -- 172.21.15.22:0/1363722955 >> 172.21.15.22:0/1363722955 conn(0x7f9ecc0b2c70 msgr2=0x7f9ecc0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:01.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ed1b20640 1 -- 172.21.15.22:0/1363722955 shutdown_connections 2024-03-29T04:19:01.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ed1b20640 1 -- 172.21.15.22:0/1363722955 wait complete. 2024-03-29T04:19:01.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ed1b20640 1 Processor -- start 2024-03-29T04:19:01.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ed1b20640 1 -- start start 2024-03-29T04:19:01.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ed1b20640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ecc06ef90 0x7f9ecc0d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:01.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ed1b20640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ecc13e820 0x7f9ecc0b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:01.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ed1b20640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ecc15ec40 0x7f9ecc0b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:01.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ed1b20640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc140b80 con 0x7f9ecc15ec40 2024-03-29T04:19:01.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ed1b20640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc140a00 con 0x7f9ecc06ef90 2024-03-29T04:19:01.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ed1b20640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9ecc140d00 con 0x7f9ecc13e820 2024-03-29T04:19:01.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ecbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ecc15ec40 0x7f9ecc0b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:01.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ecbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ecc15ec40 0x7f9ecc0b1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44314/0 (socket says 172.21.15.22:44314) 2024-03-29T04:19:01.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ecbfff640 1 -- 172.21.15.22:0/3190901404 learned_addr learned my addr 172.21.15.22:0/3190901404 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:01.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ecaffd640 1 --2- 172.21.15.22:0/3190901404 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ecc13e820 0x7f9ecc0b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:01.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ecb7fe640 1 --2- 172.21.15.22:0/3190901404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ecc06ef90 0x7f9ecc0d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:01.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ecbfff640 1 -- 172.21.15.22:0/3190901404 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ecc13e820 msgr2=0x7f9ecc0b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:01.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ecbfff640 1 --2- 172.21.15.22:0/3190901404 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ecc13e820 0x7f9ecc0b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:01.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ecbfff640 1 -- 172.21.15.22:0/3190901404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ecc06ef90 msgr2=0x7f9ecc0d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:01.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ecbfff640 1 --2- 172.21.15.22:0/3190901404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ecc06ef90 0x7f9ecc0d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:01.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ecbfff640 1 -- 172.21.15.22:0/3190901404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ecc0b4c50 con 0x7f9ecc15ec40 2024-03-29T04:19:01.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ecaffd640 1 --2- 172.21.15.22:0/3190901404 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ecc13e820 0x7f9ecc0b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:19:01.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ecb7fe640 1 --2- 172.21.15.22:0/3190901404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ecc06ef90 0x7f9ecc0d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:19:01.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ecbfff640 1 --2- 172.21.15.22:0/3190901404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ecc15ec40 0x7f9ecc0b1f80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400c7a0 tx=0x7f9ec400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:01.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9eb3fff640 1 -- 172.21.15.22:0/3190901404 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4018070 con 0x7f9ecc15ec40 2024-03-29T04:19:01.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9eb3fff640 1 -- 172.21.15.22:0/3190901404 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec400a640 con 0x7f9ecc15ec40 2024-03-29T04:19:01.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ed1b20640 1 -- 172.21.15.22:0/3190901404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ecc0b4ef0 con 0x7f9ecc15ec40 2024-03-29T04:19:01.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9eb3fff640 1 -- 172.21.15.22:0/3190901404 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec400aab0 con 0x7f9ecc15ec40 2024-03-29T04:19:01.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.839+0000 7f9ed1b20640 1 -- 172.21.15.22:0/3190901404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ecc0b54b0 con 0x7f9ecc15ec40 2024-03-29T04:19:01.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.843+0000 7f9ed1b20640 1 -- 172.21.15.22:0/3190901404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ecc068890 con 0x7f9ecc15ec40 2024-03-29T04:19:01.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.843+0000 7f9eb3fff640 1 -- 172.21.15.22:0/3190901404 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9ec401b070 con 0x7f9ecc15ec40 2024-03-29T04:19:01.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.843+0000 7f9eb3fff640 1 --2- 172.21.15.22:0/3190901404 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9eb8067fc0 0x7f9eb806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:01.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.843+0000 7f9eb3fff640 1 -- 172.21.15.22:0/3190901404 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f9ec4085c00 con 0x7f9ecc15ec40 2024-03-29T04:19:01.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.843+0000 7f9ecb7fe640 1 --2- 172.21.15.22:0/3190901404 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9eb8067fc0 0x7f9eb806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:01.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.843+0000 7f9ecb7fe640 1 --2- 172.21.15.22:0/3190901404 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9eb8067fc0 0x7f9eb806a480 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc06c020 tx=0x7f9ec0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:01.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:01.847+0000 7f9eb3fff640 1 -- 172.21.15.22:0/3190901404 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9ec408c080 con 0x7f9ecc15ec40 2024-03-29T04:19:02.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.043+0000 7f9ed1b20640 1 -- 172.21.15.22:0/3190901404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ecc0b31f0 con 0x7f9ecc15ec40 2024-03-29T04:19:02.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.043+0000 7f9eb3fff640 1 -- 172.21.15.22:0/3190901404 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5181 (secure 0 0 0) 0x7f9ec40562b0 con 0x7f9ecc15ec40 2024-03-29T04:19:02.025 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:19:02.025 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/3112156479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":3112156479},{"type":"v1","addr":"172.21.15.151:6835","nonce":3112156479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5308,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/491273202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":491273202},{"type":"v1","addr":"172.21.15.151:6837","nonce":491273202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/434199693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":434199693},{"type":"v1","addr":"172.21.15.22:6833","nonce":434199693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:18:57.990202+0000","modified":"2024-03-29T04:19:01.052542+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6835/2966923505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2966923505},{"type":"v1","addr":"172.21.15.22:6835","nonce":2966923505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529]},"id":2}]} 2024-03-29T04:19:02.026 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 18 2024-03-29T04:19:02.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.047+0000 7f9ed1b20640 1 -- 172.21.15.22:0/3190901404 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9eb8067fc0 msgr2=0x7f9eb806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:02.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.047+0000 7f9ed1b20640 1 --2- 172.21.15.22:0/3190901404 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9eb8067fc0 0x7f9eb806a480 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc06c020 tx=0x7f9ec0008040 comp rx=0 tx=0).stop 2024-03-29T04:19:02.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.047+0000 7f9ed1b20640 1 -- 172.21.15.22:0/3190901404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ecc15ec40 msgr2=0x7f9ecc0b1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:02.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.047+0000 7f9ed1b20640 1 --2- 172.21.15.22:0/3190901404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ecc15ec40 0x7f9ecc0b1f80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400c7a0 tx=0x7f9ec400cc60 comp rx=0 tx=0).stop 2024-03-29T04:19:02.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.047+0000 7f9ed1b20640 1 -- 172.21.15.22:0/3190901404 shutdown_connections 2024-03-29T04:19:02.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.047+0000 7f9ed1b20640 1 --2- 172.21.15.22:0/3190901404 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9eb8067fc0 0x7f9eb806a480 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.047+0000 7f9ed1b20640 1 --2- 172.21.15.22:0/3190901404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ecc15ec40 0x7f9ecc0b1f80 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.047+0000 7f9ed1b20640 1 --2- 172.21.15.22:0/3190901404 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9ecc13e820 0x7f9ecc0b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.047+0000 7f9ed1b20640 1 --2- 172.21.15.22:0/3190901404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ecc06ef90 0x7f9ecc0d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.047+0000 7f9ed1b20640 1 -- 172.21.15.22:0/3190901404 >> 172.21.15.22:0/3190901404 conn(0x7f9ecc0b2c70 msgr2=0x7f9ecc06b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:02.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.047+0000 7f9ed1b20640 1 -- 172.21.15.22:0/3190901404 shutdown_connections 2024-03-29T04:19:02.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.047+0000 7f9ed1b20640 1 -- 172.21.15.22:0/3190901404 wait complete. 2024-03-29T04:19:02.043 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:19:02.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f022d086640 1 Processor -- start 2024-03-29T04:19:02.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f022d086640 1 -- start start 2024-03-29T04:19:02.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f022d086640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f02280b5b20 0x7f022806d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f022d086640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f022806dcc0 0x7f022806e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f022d086640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f022815edd0 con 0x7f0228070490 2024-03-29T04:19:02.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f022d086640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f022815f0b0 con 0x7f02280b5b20 2024-03-29T04:19:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f022d086640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f022815f390 con 0x7f022806dcc0 2024-03-29T04:19:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f0226d9d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f0228070490 0x7f022815c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:52212/0 (socket says 172.21.15.22:52212) 2024-03-29T04:19:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f0226d9d640 1 -- 172.21.15.22:0/3425901708 learned_addr learned my addr 172.21.15.22:0/3425901708 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:19:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f022659c640 1 --2- 172.21.15.22:0/3425901708 >> v2:172.21.15.151:3300/0 conn(0x7f02280b5b20 0x7f022806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f022759e640 1 --2- 172.21.15.22:0/3425901708 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f022806dcc0 0x7f022806e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f0225d9b640 1 -- 172.21.15.22:0/3425901708 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3642259012 0 0) 0x7f022815edd0 con 0x7f0228070490 2024-03-29T04:19:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f0225d9b640 1 -- 172.21.15.22:0/3425901708 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0210003530 con 0x7f0228070490 2024-03-29T04:19:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f0225d9b640 1 -- 172.21.15.22:0/3425901708 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2284569809 0 0) 0x7f0210003530 con 0x7f0228070490 2024-03-29T04:19:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f0225d9b640 1 -- 172.21.15.22:0/3425901708 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f022806dcc0 msgr2=0x7f022806e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f0225d9b640 1 --2- 172.21.15.22:0/3425901708 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f022806dcc0 0x7f022806e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f0225d9b640 1 -- 172.21.15.22:0/3425901708 >> v2:172.21.15.151:3300/0 conn(0x7f02280b5b20 msgr2=0x7f022806d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f0225d9b640 1 --2- 172.21.15.22:0/3425901708 >> v2:172.21.15.151:3300/0 conn(0x7f02280b5b20 0x7f022806d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f0225d9b640 1 -- 172.21.15.22:0/3425901708 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022815f670 con 0x7f0228070490 2024-03-29T04:19:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f0225d9b640 1 -- 172.21.15.22:0/3425901708 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f0218001680 con 0x7f0228070490 2024-03-29T04:19:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f0225d9b640 1 -- 172.21.15.22:0/3425901708 >> v1:172.21.15.22:6789/0 conn(0x7f0228070490 legacy=0x7f022815c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f0225d9b640 1 --2- 172.21.15.22:0/3425901708 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f022806dcc0 0x7f02100039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f0225d9b640 1 --2- 172.21.15.22:0/3425901708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0210003f40 0x7f0210004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f0225d9b640 1 --2- 172.21.15.22:0/3425901708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0210004d50 0x7f02100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f0225d9b640 1 -- 172.21.15.22:0/3425901708 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f022815f0b0 con 0x7f0210004d50 2024-03-29T04:19:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f0225d9b640 1 -- 172.21.15.22:0/3425901708 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f022815f390 con 0x7f0210003f40 2024-03-29T04:19:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f0225d9b640 1 -- 172.21.15.22:0/3425901708 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f021000dcc0 con 0x7f022806dcc0 2024-03-29T04:19:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f022759e640 1 --2- 172.21.15.22:0/3425901708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0210004d50 0x7f02100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f0226d9d640 1 --2- 172.21.15.22:0/3425901708 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f022806dcc0 0x7f02100039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f022659c640 1 --2- 172.21.15.22:0/3425901708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0210003f40 0x7f0210004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f022759e640 1 -- 172.21.15.22:0/3425901708 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f022806dcc0 msgr2=0x7f02100039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f022759e640 1 --2- 172.21.15.22:0/3425901708 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f022806dcc0 0x7f02100039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f022759e640 1 -- 172.21.15.22:0/3425901708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0210003f40 msgr2=0x7f0210004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f022759e640 1 --2- 172.21.15.22:0/3425901708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0210003f40 0x7f0210004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f022759e640 1 -- 172.21.15.22:0/3425901708 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0210003530 con 0x7f0210004d50 2024-03-29T04:19:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.187+0000 7f022659c640 1 --2- 172.21.15.22:0/3425901708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0210003f40 0x7f0210004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:19:02.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.191+0000 7f022759e640 1 --2- 172.21.15.22:0/3425901708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0210004d50 0x7f02100051d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0220009fc0 tx=0x7f022001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910641991f62a187 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:02.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.191+0000 7f0225d9b640 1 -- 172.21.15.22:0/3425901708 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02200b2070 con 0x7f0210004d50 2024-03-29T04:19:02.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.191+0000 7f0225d9b640 1 -- 172.21.15.22:0/3425901708 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02200a5970 con 0x7f0210004d50 2024-03-29T04:19:02.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.191+0000 7f0225d9b640 1 -- 172.21.15.22:0/3425901708 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220009c10 con 0x7f0210004d50 2024-03-29T04:19:02.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.191+0000 7f022d086640 1 -- 172.21.15.22:0/3425901708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0210004d50 msgr2=0x7f02100051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:02.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.191+0000 7f022d086640 1 --2- 172.21.15.22:0/3425901708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0210004d50 0x7f02100051d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0220009fc0 tx=0x7f022001cc80 comp rx=0 tx=0).stop 2024-03-29T04:19:02.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.191+0000 7f022759e640 1 -- 172.21.15.22:0/3425901708 reap_dead start 2024-03-29T04:19:02.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.191+0000 7f022d086640 1 -- 172.21.15.22:0/3425901708 shutdown_connections 2024-03-29T04:19:02.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.191+0000 7f022d086640 1 -- 172.21.15.22:0/3425901708 >> 172.21.15.22:0/3425901708 conn(0x7f0228072500 msgr2=0x7f0228072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:02.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.191+0000 7f022d086640 1 -- 172.21.15.22:0/3425901708 shutdown_connections 2024-03-29T04:19:02.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.191+0000 7f022d086640 1 -- 172.21.15.22:0/3425901708 wait complete. 2024-03-29T04:19:02.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022d086640 1 Processor -- start 2024-03-29T04:19:02.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022d086640 1 -- start start 2024-03-29T04:19:02.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022d086640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0228147950 0x7f022806cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022d086640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0228147d30 0x7f022806d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022d086640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0228162530 0x7f02281629e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022d086640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f022806c240 con 0x7f0228147d30 2024-03-29T04:19:02.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022d086640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f022806c0c0 con 0x7f0228162530 2024-03-29T04:19:02.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022d086640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f022806c3c0 con 0x7f0228147950 2024-03-29T04:19:02.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0228147d30 0x7f022806d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0228147d30 0x7f022806d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44338/0 (socket says 172.21.15.22:44338) 2024-03-29T04:19:02.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022659c640 1 -- 172.21.15.22:0/1176337454 learned_addr learned my addr 172.21.15.22:0/1176337454 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022759e640 1 --2- 172.21.15.22:0/1176337454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0228162530 0x7f02281629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022659c640 1 -- 172.21.15.22:0/1176337454 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0228147950 msgr2=0x7f022806cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f0226d9d640 1 --2- 172.21.15.22:0/1176337454 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0228147950 0x7f022806cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022659c640 1 --2- 172.21.15.22:0/1176337454 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0228147950 0x7f022806cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022659c640 1 -- 172.21.15.22:0/1176337454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0228162530 msgr2=0x7f02281629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022659c640 1 --2- 172.21.15.22:0/1176337454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0228162530 0x7f02281629e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022659c640 1 -- 172.21.15.22:0/1176337454 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0228161ab0 con 0x7f0228147d30 2024-03-29T04:19:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022759e640 1 --2- 172.21.15.22:0/1176337454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0228162530 0x7f02281629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:19:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022659c640 1 --2- 172.21.15.22:0/1176337454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0228147d30 0x7f022806d280 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f021c011580 tx=0x7f021c011a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f020f7fe640 1 -- 172.21.15.22:0/1176337454 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c015870 con 0x7f0228147d30 2024-03-29T04:19:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f020f7fe640 1 -- 172.21.15.22:0/1176337454 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f021c015a10 con 0x7f0228147d30 2024-03-29T04:19:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022d086640 1 -- 172.21.15.22:0/1176337454 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0228160a10 con 0x7f0228147d30 2024-03-29T04:19:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f020f7fe640 1 -- 172.21.15.22:0/1176337454 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c015e00 con 0x7f0228147d30 2024-03-29T04:19:02.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022d086640 1 -- 172.21.15.22:0/1176337454 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0228160fd0 con 0x7f0228147d30 2024-03-29T04:19:02.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f022d086640 1 -- 172.21.15.22:0/1176337454 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0228068890 con 0x7f0228147d30 2024-03-29T04:19:02.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.195+0000 7f020f7fe640 1 -- 172.21.15.22:0/1176337454 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f021c00f460 con 0x7f0228147d30 2024-03-29T04:19:02.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.199+0000 7f020f7fe640 1 --2- 172.21.15.22:0/1176337454 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0214067ff0 0x7f021406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.199+0000 7f020f7fe640 1 -- 172.21.15.22:0/1176337454 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f021c086fb0 con 0x7f0228147d30 2024-03-29T04:19:02.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.199+0000 7f0226d9d640 1 --2- 172.21.15.22:0/1176337454 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0214067ff0 0x7f021406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.199+0000 7f0226d9d640 1 --2- 172.21.15.22:0/1176337454 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0214067ff0 0x7f021406a4b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f02180010f0 tx=0x7f0218009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:02.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.207+0000 7f020f7fe640 1 -- 172.21.15.22:0/1176337454 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f021c019030 con 0x7f0228147d30 2024-03-29T04:19:02.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.367+0000 7f022d086640 1 -- 172.21.15.22:0/1176337454 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02280b0540 con 0x7f0228147d30 2024-03-29T04:19:02.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.371+0000 7f020f7fe640 1 -- 172.21.15.22:0/1176337454 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16942 (secure 0 0 0) 0x7f021c0575e0 con 0x7f0228147d30 2024-03-29T04:19:02.351 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:19:02.351 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":40,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:18:59.668139+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:18:56.336876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:18:56.950670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:19:02.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.371+0000 7f022d086640 1 -- 172.21.15.22:0/1176337454 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0214067ff0 msgr2=0x7f021406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:02.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.371+0000 7f022d086640 1 --2- 172.21.15.22:0/1176337454 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0214067ff0 0x7f021406a4b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f02180010f0 tx=0x7f0218009040 comp rx=0 tx=0).stop 2024-03-29T04:19:02.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.371+0000 7f022d086640 1 -- 172.21.15.22:0/1176337454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0228147d30 msgr2=0x7f022806d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:02.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.371+0000 7f022d086640 1 --2- 172.21.15.22:0/1176337454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0228147d30 0x7f022806d280 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f021c011580 tx=0x7f021c011a40 comp rx=0 tx=0).stop 2024-03-29T04:19:02.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.371+0000 7f022d086640 1 -- 172.21.15.22:0/1176337454 shutdown_connections 2024-03-29T04:19:02.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.371+0000 7f022d086640 1 --2- 172.21.15.22:0/1176337454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0228162530 0x7f02281629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.371+0000 7f022d086640 1 --2- 172.21.15.22:0/1176337454 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0214067ff0 0x7f021406a4b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.371+0000 7f022d086640 1 --2- 172.21.15.22:0/1176337454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0228147d30 0x7f022806d280 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.371+0000 7f022d086640 1 --2- 172.21.15.22:0/1176337454 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0228147950 0x7f022806cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.371+0000 7f022d086640 1 -- 172.21.15.22:0/1176337454 >> 172.21.15.22:0/1176337454 conn(0x7f0228072500 msgr2=0x7f0228156e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:02.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.371+0000 7f022d086640 1 -- 172.21.15.22:0/1176337454 shutdown_connections 2024-03-29T04:19:02.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.371+0000 7f022d086640 1 -- 172.21.15.22:0/1176337454 wait complete. 2024-03-29T04:19:02.374 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-29T04:19:02.374 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:19:02.442 INFO:tasks.ceph.mds.a.smithi022.stderr:2024-03-29T04:19:02.459+0000 7fcfda2d4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T04:19:02.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56e172b640 1 Processor -- start 2024-03-29T04:19:02.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56e172b640 1 -- start start 2024-03-29T04:19:02.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56e172b640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56dc0b3f50 0x7f56dc169310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56e172b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56dc0b4500 con 0x7f56dc0cef80 2024-03-29T04:19:02.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56e172b640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56dc0b47e0 con 0x7f56dc0b55a0 2024-03-29T04:19:02.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56e172b640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f56dc0b4ac0 con 0x7f56dc0b3f50 2024-03-29T04:19:02.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56daffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f56dc0cef80 0x7f56dc0cf360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:52214/0 (socket says 172.21.15.22:52214) 2024-03-29T04:19:02.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56daffd640 1 -- 172.21.15.22:0/2106692117 learned_addr learned my addr 172.21.15.22:0/2106692117 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:19:02.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56db7fe640 1 --2- 172.21.15.22:0/2106692117 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56dc0b3f50 0x7f56dc169310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56d9ffb640 1 -- 172.21.15.22:0/2106692117 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1064364583 0 0) 0x7f56dc0b4500 con 0x7f56dc0cef80 2024-03-29T04:19:02.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56d9ffb640 1 -- 172.21.15.22:0/2106692117 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56bc003530 con 0x7f56dc0cef80 2024-03-29T04:19:02.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56d9ffb640 1 -- 172.21.15.22:0/2106692117 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3353458051 0 0) 0x7f56bc003530 con 0x7f56dc0cef80 2024-03-29T04:19:02.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56d9ffb640 1 -- 172.21.15.22:0/2106692117 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56dc0b3f50 msgr2=0x7f56dc169310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:02.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56d9ffb640 1 --2- 172.21.15.22:0/2106692117 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56dc0b3f50 0x7f56dc169310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56d9ffb640 1 -- 172.21.15.22:0/2106692117 >> v1:172.21.15.151:6789/0 conn(0x7f56dc0b55a0 legacy=0x7f56dc0b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:02.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56d9ffb640 1 -- 172.21.15.22:0/2106692117 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56dc0b4da0 con 0x7f56dc0cef80 2024-03-29T04:19:02.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56d9ffb640 1 -- 172.21.15.22:0/2106692117 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f56c4001a00 con 0x7f56dc0cef80 2024-03-29T04:19:02.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56d9ffb640 1 -- 172.21.15.22:0/2106692117 >> v1:172.21.15.22:6789/0 conn(0x7f56dc0cef80 legacy=0x7f56dc0cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:02.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56d9ffb640 1 --2- 172.21.15.22:0/2106692117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56bc0039f0 0x7f56bc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56d9ffb640 1 --2- 172.21.15.22:0/2106692117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56bc004810 0x7f56bc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56d9ffb640 1 --2- 172.21.15.22:0/2106692117 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56dc0b3f50 0x7f56bc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56d9ffb640 1 -- 172.21.15.22:0/2106692117 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc0b4ac0 con 0x7f56bc004810 2024-03-29T04:19:02.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56d9ffb640 1 -- 172.21.15.22:0/2106692117 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc00ba80 con 0x7f56bc0039f0 2024-03-29T04:19:02.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56d9ffb640 1 -- 172.21.15.22:0/2106692117 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f56bc00bc80 con 0x7f56dc0b3f50 2024-03-29T04:19:02.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56da7fc640 1 --2- 172.21.15.22:0/2106692117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56bc004810 0x7f56bc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56daffd640 1 --2- 172.21.15.22:0/2106692117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56bc0039f0 0x7f56bc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56db7fe640 1 --2- 172.21.15.22:0/2106692117 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56dc0b3f50 0x7f56bc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56da7fc640 1 -- 172.21.15.22:0/2106692117 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56dc0b3f50 msgr2=0x7f56bc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:02.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56da7fc640 1 --2- 172.21.15.22:0/2106692117 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56dc0b3f50 0x7f56bc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56da7fc640 1 -- 172.21.15.22:0/2106692117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56bc0039f0 msgr2=0x7f56bc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:02.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56da7fc640 1 --2- 172.21.15.22:0/2106692117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56bc0039f0 0x7f56bc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56da7fc640 1 -- 172.21.15.22:0/2106692117 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc003530 con 0x7f56bc004810 2024-03-29T04:19:02.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.519+0000 7f56db7fe640 1 --2- 172.21.15.22:0/2106692117 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56dc0b3f50 0x7f56bc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:19:02.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.523+0000 7f56da7fc640 1 --2- 172.21.15.22:0/2106692117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56bc004810 0x7f56bc004c90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f56cc001750 tx=0x7f56cc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e721d61b2070bce server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:02.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.523+0000 7f56d9ffb640 1 -- 172.21.15.22:0/2106692117 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc0013a0 con 0x7f56bc004810 2024-03-29T04:19:02.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.523+0000 7f56d9ffb640 1 -- 172.21.15.22:0/2106692117 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56cc0052f0 con 0x7f56bc004810 2024-03-29T04:19:02.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.523+0000 7f56d9ffb640 1 -- 172.21.15.22:0/2106692117 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc003840 con 0x7f56bc004810 2024-03-29T04:19:02.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.523+0000 7f56e172b640 1 -- 172.21.15.22:0/2106692117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56bc004810 msgr2=0x7f56bc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:02.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.523+0000 7f56e172b640 1 --2- 172.21.15.22:0/2106692117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56bc004810 0x7f56bc004c90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f56cc001750 tx=0x7f56cc01c3c0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.523+0000 7f56db7fe640 1 -- 172.21.15.22:0/2106692117 reap_dead start 2024-03-29T04:19:02.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.523+0000 7f56e172b640 1 -- 172.21.15.22:0/2106692117 shutdown_connections 2024-03-29T04:19:02.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.523+0000 7f56e172b640 1 -- 172.21.15.22:0/2106692117 >> 172.21.15.22:0/2106692117 conn(0x7f56dc0d4310 msgr2=0x7f56dc0d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:02.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.523+0000 7f56e172b640 1 -- 172.21.15.22:0/2106692117 shutdown_connections 2024-03-29T04:19:02.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.523+0000 7f56e172b640 1 -- 172.21.15.22:0/2106692117 wait complete. 2024-03-29T04:19:02.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56e172b640 1 Processor -- start 2024-03-29T04:19:02.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56e172b640 1 -- start start 2024-03-29T04:19:02.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56e172b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56bc004810 0x7f56dc152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56e172b640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56dc15e260 0x7f56dc0cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56e172b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56dc152680 0x7f56dc15b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56e172b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc167300 con 0x7f56dc152680 2024-03-29T04:19:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56e172b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc0cfa70 con 0x7f56bc004810 2024-03-29T04:19:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56e172b640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f56dc167480 con 0x7f56dc15e260 2024-03-29T04:19:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56db7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56dc152680 0x7f56dc15b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56db7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56dc152680 0x7f56dc15b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44348/0 (socket says 172.21.15.22:44348) 2024-03-29T04:19:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56db7fe640 1 -- 172.21.15.22:0/543414566 learned_addr learned my addr 172.21.15.22:0/543414566 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56daffd640 1 --2- 172.21.15.22:0/543414566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56bc004810 0x7f56dc152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56da7fc640 1 --2- 172.21.15.22:0/543414566 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56dc15e260 0x7f56dc0cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56db7fe640 1 -- 172.21.15.22:0/543414566 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56dc15e260 msgr2=0x7f56dc0cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56db7fe640 1 --2- 172.21.15.22:0/543414566 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56dc15e260 0x7f56dc0cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56db7fe640 1 -- 172.21.15.22:0/543414566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56bc004810 msgr2=0x7f56dc152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56db7fe640 1 --2- 172.21.15.22:0/543414566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56bc004810 0x7f56dc152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56db7fe640 1 -- 172.21.15.22:0/543414566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56dc0cc140 con 0x7f56dc152680 2024-03-29T04:19:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56daffd640 1 --2- 172.21.15.22:0/543414566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56bc004810 0x7f56dc152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:19:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56db7fe640 1 --2- 172.21.15.22:0/543414566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56dc152680 0x7f56dc15b0e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f56dc08c9e0 tx=0x7f56d4010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56d37fe640 1 -- 172.21.15.22:0/543414566 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d400f040 con 0x7f56dc152680 2024-03-29T04:19:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56d37fe640 1 -- 172.21.15.22:0/543414566 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56d400e870 con 0x7f56dc152680 2024-03-29T04:19:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56e172b640 1 -- 172.21.15.22:0/543414566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56dc0cc3e0 con 0x7f56dc152680 2024-03-29T04:19:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56d37fe640 1 -- 172.21.15.22:0/543414566 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d400ec40 con 0x7f56dc152680 2024-03-29T04:19:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56e172b640 1 -- 172.21.15.22:0/543414566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56dc0cc9a0 con 0x7f56dc152680 2024-03-29T04:19:02.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.527+0000 7f56e172b640 1 -- 172.21.15.22:0/543414566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56dc068890 con 0x7f56dc152680 2024-03-29T04:19:02.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.531+0000 7f56d37fe640 1 -- 172.21.15.22:0/543414566 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f56d401c400 con 0x7f56dc152680 2024-03-29T04:19:02.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.531+0000 7f56d37fe640 1 --2- 172.21.15.22:0/543414566 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f56c0067fc0 0x7f56c006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.531+0000 7f56d37fe640 1 -- 172.21.15.22:0/543414566 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f56d4086dc0 con 0x7f56dc152680 2024-03-29T04:19:02.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.531+0000 7f56daffd640 1 --2- 172.21.15.22:0/543414566 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f56c0067fc0 0x7f56c006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.531+0000 7f56daffd640 1 --2- 172.21.15.22:0/543414566 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f56c0067fc0 0x7f56c006a480 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f56c40013e0 tx=0x7f56c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:02.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.535+0000 7f56d37fe640 1 -- 172.21.15.22:0/543414566 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f56d4018030 con 0x7f56dc152680 2024-03-29T04:19:02.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.687+0000 7f56e172b640 1 -- 172.21.15.22:0/543414566 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f56dc0b8db0 con 0x7f56c0067fc0 2024-03-29T04:19:02.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.707+0000 7f56d37fe640 1 -- 172.21.15.22:0/543414566 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246633 (secure 0 0 0) 0x7f56dc0b8db0 con 0x7f56c0067fc0 2024-03-29T04:19:02.691 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:19:02.694 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-29T04:19:02.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.711+0000 7f56e172b640 1 -- 172.21.15.22:0/543414566 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f56c0067fc0 msgr2=0x7f56c006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:02.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.711+0000 7f56e172b640 1 --2- 172.21.15.22:0/543414566 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f56c0067fc0 0x7f56c006a480 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f56c40013e0 tx=0x7f56c4009040 comp rx=0 tx=0).stop 2024-03-29T04:19:02.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.711+0000 7f56e172b640 1 -- 172.21.15.22:0/543414566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56dc152680 msgr2=0x7f56dc15b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:02.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.711+0000 7f56e172b640 1 --2- 172.21.15.22:0/543414566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56dc152680 0x7f56dc15b0e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f56dc08c9e0 tx=0x7f56d4010c20 comp rx=0 tx=0).stop 2024-03-29T04:19:02.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.711+0000 7f56e172b640 1 -- 172.21.15.22:0/543414566 shutdown_connections 2024-03-29T04:19:02.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.711+0000 7f56e172b640 1 --2- 172.21.15.22:0/543414566 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f56c0067fc0 0x7f56c006a480 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.711+0000 7f56e172b640 1 --2- 172.21.15.22:0/543414566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56dc152680 0x7f56dc15b0e0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.711+0000 7f56e172b640 1 --2- 172.21.15.22:0/543414566 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f56dc15e260 0x7f56dc0cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.711+0000 7f56e172b640 1 --2- 172.21.15.22:0/543414566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56bc004810 0x7f56dc152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.711+0000 7f56e172b640 1 -- 172.21.15.22:0/543414566 >> 172.21.15.22:0/543414566 conn(0x7f56dc0d4310 msgr2=0x7f56dc0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:02.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.711+0000 7f56e172b640 1 -- 172.21.15.22:0/543414566 shutdown_connections 2024-03-29T04:19:02.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.711+0000 7f56e172b640 1 -- 172.21.15.22:0/543414566 wait complete. 2024-03-29T04:19:02.707 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-03-29T04:19:01.721733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241488,"kb_used_data":4524,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733232,"statfs":{"total":773094113280,"available":772846829568,"internally_reserved":0,"allocated":4632576,"data_stored":2651217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.108925"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.625563+0000","last_change":"2024-03-29T04:18:57.625563+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.625563+0000","last_undegraded":"2024-03-29T04:18:57.625563+0000","last_fullsized":"2024-03-29T04:18:57.625563+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.625314+0000","last_change":"2024-03-29T04:18:57.625314+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.625314+0000","last_undegraded":"2024-03-29T04:18:57.625314+0000","last_fullsized":"2024-03-29T04:18:57.625314+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.622173+0000","last_change":"2024-03-29T04:18:57.622173+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.622173+0000","last_undegraded":"2024-03-29T04:18:57.622173+0000","last_fullsized":"2024-03-29T04:18:57.622173+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.628289+0000","last_change":"2024-03-29T04:18:57.628289+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.628289+0000","last_undegraded":"2024-03-29T04:18:57.628289+0000","last_fullsized":"2024-03-29T04:18:57.628289+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.627884+0000","last_change":"2024-03-29T04:18:57.627884+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.627884+0000","last_undegraded":"2024-03-29T04:18:57.627884+0000","last_fullsized":"2024-03-29T04:18:57.627884+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.629072+0000","last_change":"2024-03-29T04:18:57.629072+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.629072+0000","last_undegraded":"2024-03-29T04:18:57.629072+0000","last_fullsized":"2024-03-29T04:18:57.629072+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.638167+0000","last_change":"2024-03-29T04:18:57.638167+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.638167+0000","last_undegraded":"2024-03-29T04:18:57.638167+0000","last_fullsized":"2024-03-29T04:18:57.638167+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.624752+0000","last_change":"2024-03-29T04:18:57.624752+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.624752+0000","last_undegraded":"2024-03-29T04:18:57.624752+0000","last_fullsized":"2024-03-29T04:18:57.624752+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.621806+0000","last_change":"2024-03-29T04:18:57.621806+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.621806+0000","last_undegraded":"2024-03-29T04:18:57.621806+0000","last_fullsized":"2024-03-29T04:18:57.621806+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.635701+0000","last_change":"2024-03-29T04:18:57.635701+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.635701+0000","last_undegraded":"2024-03-29T04:18:57.635701+0000","last_fullsized":"2024-03-29T04:18:57.635701+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.632529+0000","last_change":"2024-03-29T04:18:57.632529+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.632529+0000","last_undegraded":"2024-03-29T04:18:57.632529+0000","last_fullsized":"2024-03-29T04:18:57.632529+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.621716+0000","last_change":"2024-03-29T04:18:57.621716+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.621716+0000","last_undegraded":"2024-03-29T04:18:57.621716+0000","last_fullsized":"2024-03-29T04:18:57.621716+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.636959+0000","last_change":"2024-03-29T04:18:57.636959+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.636959+0000","last_undegraded":"2024-03-29T04:18:57.636959+0000","last_fullsized":"2024-03-29T04:18:57.636959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.651723+0000","last_change":"2024-03-29T04:18:57.651723+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.651723+0000","last_undegraded":"2024-03-29T04:18:57.651723+0000","last_fullsized":"2024-03-29T04:18:57.651723+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.636989+0000","last_change":"2024-03-29T04:18:57.636989+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.636989+0000","last_undegraded":"2024-03-29T04:18:57.636989+0000","last_fullsized":"2024-03-29T04:18:57.636989+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.638252+0000","last_change":"2024-03-29T04:18:57.638252+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.638252+0000","last_undegraded":"2024-03-29T04:18:57.638252+0000","last_fullsized":"2024-03-29T04:18:57.638252+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.632094+0000","last_change":"2024-03-29T04:18:57.632094+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.632094+0000","last_undegraded":"2024-03-29T04:18:57.632094+0000","last_fullsized":"2024-03-29T04:18:57.632094+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.623561+0000","last_change":"2024-03-29T04:18:57.623561+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.623561+0000","last_undegraded":"2024-03-29T04:18:57.623561+0000","last_fullsized":"2024-03-29T04:18:57.623561+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.627332+0000","last_change":"2024-03-29T04:18:57.627332+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.627332+0000","last_undegraded":"2024-03-29T04:18:57.627332+0000","last_fullsized":"2024-03-29T04:18:57.627332+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.622588+0000","last_change":"2024-03-29T04:18:57.622588+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.622588+0000","last_undegraded":"2024-03-29T04:18:57.622588+0000","last_fullsized":"2024-03-29T04:18:57.622588+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.635894+0000","last_change":"2024-03-29T04:18:57.635894+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.635894+0000","last_undegraded":"2024-03-29T04:18:57.635894+0000","last_fullsized":"2024-03-29T04:18:57.635894+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.643079+0000","last_change":"2024-03-29T04:18:57.643079+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.643079+0000","last_undegraded":"2024-03-29T04:18:57.643079+0000","last_fullsized":"2024-03-29T04:18:57.643079+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.635697+0000","last_change":"2024-03-29T04:18:57.635697+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.635697+0000","last_undegraded":"2024-03-29T04:18:57.635697+0000","last_fullsized":"2024-03-29T04:18:57.635697+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.627435+0000","last_change":"2024-03-29T04:18:57.627435+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.627435+0000","last_undegraded":"2024-03-29T04:18:57.627435+0000","last_fullsized":"2024-03-29T04:18:57.627435+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.624839+0000","last_change":"2024-03-29T04:18:57.624839+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.624839+0000","last_undegraded":"2024-03-29T04:18:57.624839+0000","last_fullsized":"2024-03-29T04:18:57.624839+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.627398+0000","last_change":"2024-03-29T04:18:57.627398+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.627398+0000","last_undegraded":"2024-03-29T04:18:57.627398+0000","last_fullsized":"2024-03-29T04:18:57.627398+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.623666+0000","last_change":"2024-03-29T04:18:57.623666+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.623666+0000","last_undegraded":"2024-03-29T04:18:57.623666+0000","last_fullsized":"2024-03-29T04:18:57.623666+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.630493+0000","last_change":"2024-03-29T04:18:57.630493+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.630493+0000","last_undegraded":"2024-03-29T04:18:57.630493+0000","last_fullsized":"2024-03-29T04:18:57.630493+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.625750+0000","last_change":"2024-03-29T04:18:57.625750+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.625750+0000","last_undegraded":"2024-03-29T04:18:57.625750+0000","last_fullsized":"2024-03-29T04:18:57.625750+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.622621+0000","last_change":"2024-03-29T04:18:57.622621+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.622621+0000","last_undegraded":"2024-03-29T04:18:57.622621+0000","last_fullsized":"2024-03-29T04:18:57.622621+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.631780+0000","last_change":"2024-03-29T04:18:57.631780+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.631780+0000","last_undegraded":"2024-03-29T04:18:57.631780+0000","last_fullsized":"2024-03-29T04:18:57.631780+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.627065+0000","last_change":"2024-03-29T04:18:57.627065+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.627065+0000","last_undegraded":"2024-03-29T04:18:57.627065+0000","last_fullsized":"2024-03-29T04:18:57.627065+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.639988+0000","last_change":"2024-03-29T04:18:57.639988+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.639988+0000","last_undegraded":"2024-03-29T04:18:57.639988+0000","last_fullsized":"2024-03-29T04:18:57.639988+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.656457+0000","last_change":"2024-03-29T04:18:57.652890+0000","last_active":"2024-03-29T04:18:57.656457+0000","last_peered":"2024-03-29T04:18:57.656457+0000","last_clean":"2024-03-29T04:18:57.656457+0000","last_became_active":"2024-03-29T04:18:57.652244+0000","last_became_peered":"2024-03-29T04:18:57.652244+0000","last_unstale":"2024-03-29T04:18:57.656457+0000","last_undegraded":"2024-03-29T04:18:57.656457+0000","last_fullsized":"2024-03-29T04:18:57.656457+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:30:52.040122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.630597+0000","last_change":"2024-03-29T04:18:57.630597+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.630597+0000","last_undegraded":"2024-03-29T04:18:57.630597+0000","last_fullsized":"2024-03-29T04:18:57.630597+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.647365+0000","last_change":"2024-03-29T04:18:57.646783+0000","last_active":"2024-03-29T04:18:57.647365+0000","last_peered":"2024-03-29T04:18:57.647365+0000","last_clean":"2024-03-29T04:18:57.647365+0000","last_became_active":"2024-03-29T04:18:57.646495+0000","last_became_peered":"2024-03-29T04:18:57.646495+0000","last_unstale":"2024-03-29T04:18:57.647365+0000","last_undegraded":"2024-03-29T04:18:57.647365+0000","last_fullsized":"2024-03-29T04:18:57.647365+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:56:06.055281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.637914+0000","last_change":"2024-03-29T04:18:57.637914+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.637914+0000","last_undegraded":"2024-03-29T04:18:57.637914+0000","last_fullsized":"2024-03-29T04:18:57.637914+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.654763+0000","last_change":"2024-03-29T04:18:57.654191+0000","last_active":"2024-03-29T04:18:57.654763+0000","last_peered":"2024-03-29T04:18:57.654763+0000","last_clean":"2024-03-29T04:18:57.654763+0000","last_became_active":"2024-03-29T04:18:57.653911+0000","last_became_peered":"2024-03-29T04:18:57.653911+0000","last_unstale":"2024-03-29T04:18:57.654763+0000","last_undegraded":"2024-03-29T04:18:57.654763+0000","last_fullsized":"2024-03-29T04:18:57.654763+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:57.166873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.624736+0000","last_change":"2024-03-29T04:18:57.624736+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.624736+0000","last_undegraded":"2024-03-29T04:18:57.624736+0000","last_fullsized":"2024-03-29T04:18:57.624736+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.662129+0000","last_change":"2024-03-29T04:18:57.651021+0000","last_active":"2024-03-29T04:18:57.662129+0000","last_peered":"2024-03-29T04:18:57.662129+0000","last_clean":"2024-03-29T04:18:57.662129+0000","last_became_active":"2024-03-29T04:18:57.644336+0000","last_became_peered":"2024-03-29T04:18:57.644336+0000","last_unstale":"2024-03-29T04:18:57.662129+0000","last_undegraded":"2024-03-29T04:18:57.662129+0000","last_fullsized":"2024-03-29T04:18:57.662129+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:56:06.055281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.623072+0000","last_change":"2024-03-29T04:18:57.623072+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.623072+0000","last_undegraded":"2024-03-29T04:18:57.623072+0000","last_fullsized":"2024-03-29T04:18:57.623072+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.662014+0000","last_change":"2024-03-29T04:18:57.650116+0000","last_active":"2024-03-29T04:18:57.662014+0000","last_peered":"2024-03-29T04:18:57.662014+0000","last_clean":"2024-03-29T04:18:57.662014+0000","last_became_active":"2024-03-29T04:18:57.638212+0000","last_became_peered":"2024-03-29T04:18:57.638212+0000","last_unstale":"2024-03-29T04:18:57.662014+0000","last_undegraded":"2024-03-29T04:18:57.662014+0000","last_fullsized":"2024-03-29T04:18:57.662014+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:57.166873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.632972+0000","last_change":"2024-03-29T04:18:57.632972+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.632972+0000","last_undegraded":"2024-03-29T04:18:57.632972+0000","last_fullsized":"2024-03-29T04:18:57.632972+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.666853+0000","last_change":"2024-03-29T04:18:57.665159+0000","last_active":"2024-03-29T04:18:57.666853+0000","last_peered":"2024-03-29T04:18:57.666853+0000","last_clean":"2024-03-29T04:18:57.666853+0000","last_became_active":"2024-03-29T04:18:57.664442+0000","last_became_peered":"2024-03-29T04:18:57.664442+0000","last_unstale":"2024-03-29T04:18:57.666853+0000","last_undegraded":"2024-03-29T04:18:57.666853+0000","last_fullsized":"2024-03-29T04:18:57.666853+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:57.166873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.641871+0000","last_change":"2024-03-29T04:18:57.641871+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.641871+0000","last_undegraded":"2024-03-29T04:18:57.641871+0000","last_fullsized":"2024-03-29T04:18:57.641871+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.663425+0000","last_change":"2024-03-29T04:18:57.662680+0000","last_active":"2024-03-29T04:18:57.663425+0000","last_peered":"2024-03-29T04:18:57.663425+0000","last_clean":"2024-03-29T04:18:57.663425+0000","last_became_active":"2024-03-29T04:18:57.662148+0000","last_became_peered":"2024-03-29T04:18:57.662148+0000","last_unstale":"2024-03-29T04:18:57.663425+0000","last_undegraded":"2024-03-29T04:18:57.663425+0000","last_fullsized":"2024-03-29T04:18:57.663425+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:08:44.644431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.626138+0000","last_change":"2024-03-29T04:18:57.626138+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.626138+0000","last_undegraded":"2024-03-29T04:18:57.626138+0000","last_fullsized":"2024-03-29T04:18:57.626138+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.662773+0000","last_change":"2024-03-29T04:18:57.652332+0000","last_active":"2024-03-29T04:18:57.662773+0000","last_peered":"2024-03-29T04:18:57.662773+0000","last_clean":"2024-03-29T04:18:57.662773+0000","last_became_active":"2024-03-29T04:18:57.651656+0000","last_became_peered":"2024-03-29T04:18:57.651656+0000","last_unstale":"2024-03-29T04:18:57.662773+0000","last_undegraded":"2024-03-29T04:18:57.662773+0000","last_fullsized":"2024-03-29T04:18:57.662773+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:08:44.644431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.627438+0000","last_change":"2024-03-29T04:18:57.627438+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.627438+0000","last_undegraded":"2024-03-29T04:18:57.627438+0000","last_fullsized":"2024-03-29T04:18:57.627438+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.635115+0000","last_change":"2024-03-29T04:18:57.633865+0000","last_active":"2024-03-29T04:18:57.635115+0000","last_peered":"2024-03-29T04:18:57.635115+0000","last_clean":"2024-03-29T04:18:57.635115+0000","last_became_active":"2024-03-29T04:18:57.633449+0000","last_became_peered":"2024-03-29T04:18:57.633449+0000","last_unstale":"2024-03-29T04:18:57.635115+0000","last_undegraded":"2024-03-29T04:18:57.635115+0000","last_fullsized":"2024-03-29T04:18:57.635115+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:07:09.642311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.634764+0000","last_change":"2024-03-29T04:18:57.634764+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.634764+0000","last_undegraded":"2024-03-29T04:18:57.634764+0000","last_fullsized":"2024-03-29T04:18:57.634764+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.669540+0000","last_change":"2024-03-29T04:18:57.665734+0000","last_active":"2024-03-29T04:18:57.669540+0000","last_peered":"2024-03-29T04:18:57.669540+0000","last_clean":"2024-03-29T04:18:57.669540+0000","last_became_active":"2024-03-29T04:18:57.662309+0000","last_became_peered":"2024-03-29T04:18:57.662309+0000","last_unstale":"2024-03-29T04:18:57.669540+0000","last_undegraded":"2024-03-29T04:18:57.669540+0000","last_fullsized":"2024-03-29T04:18:57.669540+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:32.935677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.621183+0000","last_change":"2024-03-29T04:18:57.621183+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.621183+0000","last_undegraded":"2024-03-29T04:18:57.621183+0000","last_fullsized":"2024-03-29T04:18:57.621183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.666094+0000","last_change":"2024-03-29T04:18:57.660878+0000","last_active":"2024-03-29T04:18:57.666094+0000","last_peered":"2024-03-29T04:18:57.666094+0000","last_clean":"2024-03-29T04:18:57.666094+0000","last_became_active":"2024-03-29T04:18:57.659621+0000","last_became_peered":"2024-03-29T04:18:57.659621+0000","last_unstale":"2024-03-29T04:18:57.666094+0000","last_undegraded":"2024-03-29T04:18:57.666094+0000","last_fullsized":"2024-03-29T04:18:57.666094+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:08:44.644431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.634774+0000","last_change":"2024-03-29T04:18:57.634774+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.634774+0000","last_undegraded":"2024-03-29T04:18:57.634774+0000","last_fullsized":"2024-03-29T04:18:57.634774+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.650916+0000","last_change":"2024-03-29T04:18:57.650361+0000","last_active":"2024-03-29T04:18:57.650916+0000","last_peered":"2024-03-29T04:18:57.650916+0000","last_clean":"2024-03-29T04:18:57.650916+0000","last_became_active":"2024-03-29T04:18:57.650006+0000","last_became_peered":"2024-03-29T04:18:57.650006+0000","last_unstale":"2024-03-29T04:18:57.650916+0000","last_undegraded":"2024-03-29T04:18:57.650916+0000","last_fullsized":"2024-03-29T04:18:57.650916+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:34.306196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.622013+0000","last_change":"2024-03-29T04:18:57.622013+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.622013+0000","last_undegraded":"2024-03-29T04:18:57.622013+0000","last_fullsized":"2024-03-29T04:18:57.622013+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.634732+0000","last_change":"2024-03-29T04:18:57.633188+0000","last_active":"2024-03-29T04:18:57.634732+0000","last_peered":"2024-03-29T04:18:57.634732+0000","last_clean":"2024-03-29T04:18:57.634732+0000","last_became_active":"2024-03-29T04:18:57.632810+0000","last_became_peered":"2024-03-29T04:18:57.632810+0000","last_unstale":"2024-03-29T04:18:57.634732+0000","last_undegraded":"2024-03-29T04:18:57.634732+0000","last_fullsized":"2024-03-29T04:18:57.634732+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:58:15.229696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.624837+0000","last_change":"2024-03-29T04:18:57.624837+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.624837+0000","last_undegraded":"2024-03-29T04:18:57.624837+0000","last_fullsized":"2024-03-29T04:18:57.624837+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.667382+0000","last_change":"2024-03-29T04:18:57.666809+0000","last_active":"2024-03-29T04:18:57.667382+0000","last_peered":"2024-03-29T04:18:57.667382+0000","last_clean":"2024-03-29T04:18:57.667382+0000","last_became_active":"2024-03-29T04:18:57.666382+0000","last_became_peered":"2024-03-29T04:18:57.666382+0000","last_unstale":"2024-03-29T04:18:57.667382+0000","last_undegraded":"2024-03-29T04:18:57.667382+0000","last_fullsized":"2024-03-29T04:18:57.667382+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:32.935677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.630791+0000","last_change":"2024-03-29T04:18:57.630791+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.630791+0000","last_undegraded":"2024-03-29T04:18:57.630791+0000","last_fullsized":"2024-03-29T04:18:57.630791+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.655700+0000","last_change":"2024-03-29T04:18:57.653609+0000","last_active":"2024-03-29T04:18:57.655700+0000","last_peered":"2024-03-29T04:18:57.655700+0000","last_clean":"2024-03-29T04:18:57.655700+0000","last_became_active":"2024-03-29T04:18:57.653049+0000","last_became_peered":"2024-03-29T04:18:57.653049+0000","last_unstale":"2024-03-29T04:18:57.655700+0000","last_undegraded":"2024-03-29T04:18:57.655700+0000","last_fullsized":"2024-03-29T04:18:57.655700+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:14:26.382626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.625716+0000","last_change":"2024-03-29T04:18:57.625716+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.625716+0000","last_undegraded":"2024-03-29T04:18:57.625716+0000","last_fullsized":"2024-03-29T04:18:57.625716+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.667279+0000","last_change":"2024-03-29T04:18:57.664980+0000","last_active":"2024-03-29T04:18:57.667279+0000","last_peered":"2024-03-29T04:18:57.667279+0000","last_clean":"2024-03-29T04:18:57.667279+0000","last_became_active":"2024-03-29T04:18:57.664377+0000","last_became_peered":"2024-03-29T04:18:57.664377+0000","last_unstale":"2024-03-29T04:18:57.667279+0000","last_undegraded":"2024-03-29T04:18:57.667279+0000","last_fullsized":"2024-03-29T04:18:57.667279+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:37.267037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.624394+0000","last_change":"2024-03-29T04:18:57.624394+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.624394+0000","last_undegraded":"2024-03-29T04:18:57.624394+0000","last_fullsized":"2024-03-29T04:18:57.624394+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.669561+0000","last_change":"2024-03-29T04:18:57.666734+0000","last_active":"2024-03-29T04:18:57.669561+0000","last_peered":"2024-03-29T04:18:57.669561+0000","last_clean":"2024-03-29T04:18:57.669561+0000","last_became_active":"2024-03-29T04:18:57.666091+0000","last_became_peered":"2024-03-29T04:18:57.666091+0000","last_unstale":"2024-03-29T04:18:57.669561+0000","last_undegraded":"2024-03-29T04:18:57.669561+0000","last_fullsized":"2024-03-29T04:18:57.669561+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:51.553019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.631448+0000","last_change":"2024-03-29T04:18:57.631448+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.631448+0000","last_undegraded":"2024-03-29T04:18:57.631448+0000","last_fullsized":"2024-03-29T04:18:57.631448+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.670139+0000","last_change":"2024-03-29T04:18:57.666956+0000","last_active":"2024-03-29T04:18:57.670139+0000","last_peered":"2024-03-29T04:18:57.670139+0000","last_clean":"2024-03-29T04:18:57.670139+0000","last_became_active":"2024-03-29T04:18:57.666383+0000","last_became_peered":"2024-03-29T04:18:57.666383+0000","last_unstale":"2024-03-29T04:18:57.670139+0000","last_undegraded":"2024-03-29T04:18:57.670139+0000","last_fullsized":"2024-03-29T04:18:57.670139+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:32.935677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.639038+0000","last_change":"2024-03-29T04:18:57.639038+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.639038+0000","last_undegraded":"2024-03-29T04:18:57.639038+0000","last_fullsized":"2024-03-29T04:18:57.639038+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.650794+0000","last_change":"2024-03-29T04:18:57.649911+0000","last_active":"2024-03-29T04:18:57.650794+0000","last_peered":"2024-03-29T04:18:57.650794+0000","last_clean":"2024-03-29T04:18:57.650794+0000","last_became_active":"2024-03-29T04:18:57.649663+0000","last_became_peered":"2024-03-29T04:18:57.649663+0000","last_unstale":"2024-03-29T04:18:57.650794+0000","last_undegraded":"2024-03-29T04:18:57.650794+0000","last_fullsized":"2024-03-29T04:18:57.650794+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:05:33.397788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.627310+0000","last_change":"2024-03-29T04:18:57.627310+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.627310+0000","last_undegraded":"2024-03-29T04:18:57.627310+0000","last_fullsized":"2024-03-29T04:18:57.627310+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.655035+0000","last_change":"2024-03-29T04:18:57.654452+0000","last_active":"2024-03-29T04:18:57.655035+0000","last_peered":"2024-03-29T04:18:57.655035+0000","last_clean":"2024-03-29T04:18:57.655035+0000","last_became_active":"2024-03-29T04:18:57.654084+0000","last_became_peered":"2024-03-29T04:18:57.654084+0000","last_unstale":"2024-03-29T04:18:57.655035+0000","last_undegraded":"2024-03-29T04:18:57.655035+0000","last_fullsized":"2024-03-29T04:18:57.655035+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:56:06.055281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.632889+0000","last_change":"2024-03-29T04:18:57.632889+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.632889+0000","last_undegraded":"2024-03-29T04:18:57.632889+0000","last_fullsized":"2024-03-29T04:18:57.632889+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.635307+0000","last_change":"2024-03-29T04:18:57.633652+0000","last_active":"2024-03-29T04:18:57.635307+0000","last_peered":"2024-03-29T04:18:57.635307+0000","last_clean":"2024-03-29T04:18:57.635307+0000","last_became_active":"2024-03-29T04:18:57.633156+0000","last_became_peered":"2024-03-29T04:18:57.633156+0000","last_unstale":"2024-03-29T04:18:57.635307+0000","last_undegraded":"2024-03-29T04:18:57.635307+0000","last_fullsized":"2024-03-29T04:18:57.635307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:29:47.004303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.622136+0000","last_change":"2024-03-29T04:18:57.622136+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.622136+0000","last_undegraded":"2024-03-29T04:18:57.622136+0000","last_fullsized":"2024-03-29T04:18:57.622136+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.664068+0000","last_change":"2024-03-29T04:18:57.663600+0000","last_active":"2024-03-29T04:18:57.664068+0000","last_peered":"2024-03-29T04:18:57.664068+0000","last_clean":"2024-03-29T04:18:57.664068+0000","last_became_active":"2024-03-29T04:18:57.663253+0000","last_became_peered":"2024-03-29T04:18:57.663253+0000","last_unstale":"2024-03-29T04:18:57.664068+0000","last_undegraded":"2024-03-29T04:18:57.664068+0000","last_fullsized":"2024-03-29T04:18:57.664068+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:02:24.387380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.649106+0000","last_change":"2024-03-29T04:18:57.649106+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.649106+0000","last_undegraded":"2024-03-29T04:18:57.649106+0000","last_fullsized":"2024-03-29T04:18:57.649106+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.655742+0000","last_change":"2024-03-29T04:18:57.647273+0000","last_active":"2024-03-29T04:18:57.655742+0000","last_peered":"2024-03-29T04:18:57.655742+0000","last_clean":"2024-03-29T04:18:57.655742+0000","last_became_active":"2024-03-29T04:18:57.646106+0000","last_became_peered":"2024-03-29T04:18:57.646106+0000","last_unstale":"2024-03-29T04:18:57.655742+0000","last_undegraded":"2024-03-29T04:18:57.655742+0000","last_fullsized":"2024-03-29T04:18:57.655742+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:32:54.500069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.625166+0000","last_change":"2024-03-29T04:18:57.625166+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.625166+0000","last_undegraded":"2024-03-29T04:18:57.625166+0000","last_fullsized":"2024-03-29T04:18:57.625166+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.634816+0000","last_change":"2024-03-29T04:18:57.633301+0000","last_active":"2024-03-29T04:18:57.634816+0000","last_peered":"2024-03-29T04:18:57.634816+0000","last_clean":"2024-03-29T04:18:57.634816+0000","last_became_active":"2024-03-29T04:18:57.633009+0000","last_became_peered":"2024-03-29T04:18:57.633009+0000","last_unstale":"2024-03-29T04:18:57.634816+0000","last_undegraded":"2024-03-29T04:18:57.634816+0000","last_fullsized":"2024-03-29T04:18:57.634816+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:08:15.678077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.631700+0000","last_change":"2024-03-29T04:18:57.631700+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.631700+0000","last_undegraded":"2024-03-29T04:18:57.631700+0000","last_fullsized":"2024-03-29T04:18:57.631700+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.667043+0000","last_change":"2024-03-29T04:18:57.660454+0000","last_active":"2024-03-29T04:18:57.667043+0000","last_peered":"2024-03-29T04:18:57.667043+0000","last_clean":"2024-03-29T04:18:57.667043+0000","last_became_active":"2024-03-29T04:18:57.650654+0000","last_became_peered":"2024-03-29T04:18:57.650654+0000","last_unstale":"2024-03-29T04:18:57.667043+0000","last_undegraded":"2024-03-29T04:18:57.667043+0000","last_fullsized":"2024-03-29T04:18:57.667043+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:56:06.055281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.633061+0000","last_change":"2024-03-29T04:18:57.633061+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.633061+0000","last_undegraded":"2024-03-29T04:18:57.633061+0000","last_fullsized":"2024-03-29T04:18:57.633061+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.663471+0000","last_change":"2024-03-29T04:18:57.662501+0000","last_active":"2024-03-29T04:18:57.663471+0000","last_peered":"2024-03-29T04:18:57.663471+0000","last_clean":"2024-03-29T04:18:57.663471+0000","last_became_active":"2024-03-29T04:18:57.662145+0000","last_became_peered":"2024-03-29T04:18:57.662145+0000","last_unstale":"2024-03-29T04:18:57.663471+0000","last_undegraded":"2024-03-29T04:18:57.663471+0000","last_fullsized":"2024-03-29T04:18:57.663471+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:06:54.580779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.624796+0000","last_change":"2024-03-29T04:18:57.624796+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.624796+0000","last_undegraded":"2024-03-29T04:18:57.624796+0000","last_fullsized":"2024-03-29T04:18:57.624796+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.636288+0000","last_change":"2024-03-29T04:18:57.635473+0000","last_active":"2024-03-29T04:18:57.636288+0000","last_peered":"2024-03-29T04:18:57.636288+0000","last_clean":"2024-03-29T04:18:57.636288+0000","last_became_active":"2024-03-29T04:18:57.635129+0000","last_became_peered":"2024-03-29T04:18:57.635129+0000","last_unstale":"2024-03-29T04:18:57.636288+0000","last_undegraded":"2024-03-29T04:18:57.636288+0000","last_fullsized":"2024-03-29T04:18:57.636288+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:37.267037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.621515+0000","last_change":"2024-03-29T04:18:57.621515+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.621515+0000","last_undegraded":"2024-03-29T04:18:57.621515+0000","last_fullsized":"2024-03-29T04:18:57.621515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.642130+0000","last_change":"2024-03-29T04:18:57.641648+0000","last_active":"2024-03-29T04:18:57.642130+0000","last_peered":"2024-03-29T04:18:57.642130+0000","last_clean":"2024-03-29T04:18:57.642130+0000","last_became_active":"2024-03-29T04:18:57.641274+0000","last_became_peered":"2024-03-29T04:18:57.641274+0000","last_unstale":"2024-03-29T04:18:57.642130+0000","last_undegraded":"2024-03-29T04:18:57.642130+0000","last_fullsized":"2024-03-29T04:18:57.642130+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:57.166873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.628402+0000","last_change":"2024-03-29T04:18:57.628402+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.628402+0000","last_undegraded":"2024-03-29T04:18:57.628402+0000","last_fullsized":"2024-03-29T04:18:57.628402+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.642242+0000","last_change":"2024-03-29T04:18:57.636067+0000","last_active":"2024-03-29T04:18:57.642242+0000","last_peered":"2024-03-29T04:18:57.642242+0000","last_clean":"2024-03-29T04:18:57.642242+0000","last_became_active":"2024-03-29T04:18:57.635270+0000","last_became_peered":"2024-03-29T04:18:57.635270+0000","last_unstale":"2024-03-29T04:18:57.642242+0000","last_undegraded":"2024-03-29T04:18:57.642242+0000","last_fullsized":"2024-03-29T04:18:57.642242+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:45.160719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.644151+0000","last_change":"2024-03-29T04:18:57.644151+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.644151+0000","last_undegraded":"2024-03-29T04:18:57.644151+0000","last_fullsized":"2024-03-29T04:18:57.644151+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.661753+0000","last_change":"2024-03-29T04:18:57.643649+0000","last_active":"2024-03-29T04:18:57.661753+0000","last_peered":"2024-03-29T04:18:57.661753+0000","last_clean":"2024-03-29T04:18:57.661753+0000","last_became_active":"2024-03-29T04:18:57.642751+0000","last_became_peered":"2024-03-29T04:18:57.642751+0000","last_unstale":"2024-03-29T04:18:57.661753+0000","last_undegraded":"2024-03-29T04:18:57.661753+0000","last_fullsized":"2024-03-29T04:18:57.661753+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:57.166873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.630826+0000","last_change":"2024-03-29T04:18:57.630826+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.630826+0000","last_undegraded":"2024-03-29T04:18:57.630826+0000","last_fullsized":"2024-03-29T04:18:57.630826+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.634944+0000","last_change":"2024-03-29T04:18:57.633269+0000","last_active":"2024-03-29T04:18:57.634944+0000","last_peered":"2024-03-29T04:18:57.634944+0000","last_clean":"2024-03-29T04:18:57.634944+0000","last_became_active":"2024-03-29T04:18:57.632982+0000","last_became_peered":"2024-03-29T04:18:57.632982+0000","last_unstale":"2024-03-29T04:18:57.634944+0000","last_undegraded":"2024-03-29T04:18:57.634944+0000","last_fullsized":"2024-03-29T04:18:57.634944+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:56:45.800534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.647972+0000","last_change":"2024-03-29T04:18:57.647972+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.647972+0000","last_undegraded":"2024-03-29T04:18:57.647972+0000","last_fullsized":"2024-03-29T04:18:57.647972+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.658040+0000","last_change":"2024-03-29T04:18:57.655959+0000","last_active":"2024-03-29T04:18:57.658040+0000","last_peered":"2024-03-29T04:18:57.658040+0000","last_clean":"2024-03-29T04:18:57.658040+0000","last_became_active":"2024-03-29T04:18:57.651470+0000","last_became_peered":"2024-03-29T04:18:57.651470+0000","last_unstale":"2024-03-29T04:18:57.658040+0000","last_undegraded":"2024-03-29T04:18:57.658040+0000","last_fullsized":"2024-03-29T04:18:57.658040+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:18:30.179360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.639032+0000","last_change":"2024-03-29T04:17:25.361826+0000","last_active":"2024-03-29T04:18:57.639032+0000","last_peered":"2024-03-29T04:18:57.639032+0000","last_clean":"2024-03-29T04:18:57.639032+0000","last_became_active":"2024-03-29T04:17:25.361259+0000","last_became_peered":"2024-03-29T04:17:25.361259+0000","last_unstale":"2024-03-29T04:18:57.639032+0000","last_undegraded":"2024-03-29T04:18:57.639032+0000","last_fullsized":"2024-03-29T04:18:57.639032+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_clean_scrub_stamp":"2024-03-29T04:17:24.334639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:40.027873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1019904,"data_stored":771678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":181310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":438272,"data_stored":187977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":181310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":438272,"data_stored":187977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":438272,"data_stored":187977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":425984,"data_stored":181310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1019904,"data_stored":771678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:19:02.712 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:19:02.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.887+0000 7f407dbdd640 1 Processor -- start 2024-03-29T04:19:02.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.887+0000 7f407dbdd640 1 -- start start 2024-03-29T04:19:02.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.887+0000 7f407dbdd640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f4078145d20 0x7f40780cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.887+0000 7f407dbdd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f40780700f0 0x7f407806b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.887+0000 7f407dbdd640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40780cfc00 con 0x7f4078070a00 2024-03-29T04:19:02.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.887+0000 7f407dbdd640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f407806e110 con 0x7f40780700f0 2024-03-29T04:19:02.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.887+0000 7f407dbdd640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f4078146100 con 0x7f4078145d20 2024-03-29T04:19:02.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.887+0000 7f4077fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f40780700f0 0x7f407806b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.887+0000 7f4076ffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f4078070a00 0x7f407806f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:52228/0 (socket says 172.21.15.22:52228) 2024-03-29T04:19:02.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.887+0000 7f4076ffd640 1 -- 172.21.15.22:0/3825132204 learned_addr learned my addr 172.21.15.22:0/3825132204 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:19:02.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.887+0000 7f40777fe640 1 --2- 172.21.15.22:0/3825132204 >> v2:172.21.15.151:3300/0 conn(0x7f4078145d20 0x7f40780cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.887+0000 7f40767fc640 1 -- 172.21.15.22:0/3825132204 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 657082925 0 0) 0x7f406c001120 con 0x7f4078070a00 2024-03-29T04:19:02.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.887+0000 7f40767fc640 1 -- 172.21.15.22:0/3825132204 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4060003530 con 0x7f4078070a00 2024-03-29T04:19:02.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.887+0000 7f4077fff640 1 -- 172.21.15.22:0/3825132204 >> v2:172.21.15.151:3300/0 conn(0x7f4078145d20 msgr2=0x7f40780cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:02.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.887+0000 7f4077fff640 1 --2- 172.21.15.22:0/3825132204 >> v2:172.21.15.151:3300/0 conn(0x7f4078145d20 0x7f40780cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.887+0000 7f4077fff640 1 -- 172.21.15.22:0/3825132204 >> v1:172.21.15.22:6789/0 conn(0x7f4078070a00 legacy=0x7f407806f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:02.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.887+0000 7f4077fff640 1 -- 172.21.15.22:0/3825132204 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40780cfe80 con 0x7f40780700f0 2024-03-29T04:19:02.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f4077fff640 1 --2- 172.21.15.22:0/3825132204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40780700f0 0x7f407806b920 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f40780b6610 tx=0x7f407001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4d65aba9698820 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:02.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f40767fc640 1 -- 172.21.15.22:0/3825132204 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40700b2070 con 0x7f40780700f0 2024-03-29T04:19:02.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f40767fc640 1 -- 172.21.15.22:0/3825132204 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40700a5850 con 0x7f40780700f0 2024-03-29T04:19:02.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f40767fc640 1 -- 172.21.15.22:0/3825132204 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40700a5c20 con 0x7f40780700f0 2024-03-29T04:19:02.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f407dbdd640 1 -- 172.21.15.22:0/3825132204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40780700f0 msgr2=0x7f407806b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:02.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f407dbdd640 1 --2- 172.21.15.22:0/3825132204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40780700f0 0x7f407806b920 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f40780b6610 tx=0x7f407001c8a0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f407dbdd640 1 -- 172.21.15.22:0/3825132204 shutdown_connections 2024-03-29T04:19:02.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f407dbdd640 1 --2- 172.21.15.22:0/3825132204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40780700f0 0x7f407806b920 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f407dbdd640 1 --2- 172.21.15.22:0/3825132204 >> v2:172.21.15.151:3300/0 conn(0x7f4078145d20 0x7f40780cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f407dbdd640 1 -- 172.21.15.22:0/3825132204 >> 172.21.15.22:0/3825132204 conn(0x7f407806e410 msgr2=0x7f407806e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:02.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f407dbdd640 1 -- 172.21.15.22:0/3825132204 shutdown_connections 2024-03-29T04:19:02.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f407dbdd640 1 -- 172.21.15.22:0/3825132204 wait complete. 2024-03-29T04:19:02.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f407dbdd640 1 Processor -- start 2024-03-29T04:19:02.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f407dbdd640 1 -- start start 2024-03-29T04:19:02.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f407dbdd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40780700f0 0x7f407813d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f407dbdd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4078070a00 0x7f407815cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f407dbdd640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4078145d20 0x7f4078158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f407dbdd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f407806c7f0 con 0x7f40780700f0 2024-03-29T04:19:02.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f407dbdd640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f407806c670 con 0x7f4078070a00 2024-03-29T04:19:02.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f407dbdd640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f407806c970 con 0x7f4078145d20 2024-03-29T04:19:02.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f40777fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40780700f0 0x7f407813d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f40777fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40780700f0 0x7f407813d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44372/0 (socket says 172.21.15.22:44372) 2024-03-29T04:19:02.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.891+0000 7f40777fe640 1 -- 172.21.15.22:0/2475576872 learned_addr learned my addr 172.21.15.22:0/2475576872 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:02.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.895+0000 7f4076ffd640 1 --2- 172.21.15.22:0/2475576872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4078070a00 0x7f407815cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.895+0000 7f4077fff640 1 --2- 172.21.15.22:0/2475576872 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4078145d20 0x7f4078158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.895+0000 7f40777fe640 1 -- 172.21.15.22:0/2475576872 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4078145d20 msgr2=0x7f4078158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:02.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.895+0000 7f40777fe640 1 --2- 172.21.15.22:0/2475576872 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4078145d20 0x7f4078158e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.895+0000 7f40777fe640 1 -- 172.21.15.22:0/2475576872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4078070a00 msgr2=0x7f407815cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:02.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.895+0000 7f40777fe640 1 --2- 172.21.15.22:0/2475576872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4078070a00 0x7f407815cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:02.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.895+0000 7f40777fe640 1 -- 172.21.15.22:0/2475576872 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407815e750 con 0x7f40780700f0 2024-03-29T04:19:02.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.895+0000 7f40777fe640 1 --2- 172.21.15.22:0/2475576872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40780700f0 0x7f407813d470 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f406800d5f0 tx=0x7f406800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:02.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.895+0000 7f405ffff640 1 -- 172.21.15.22:0/2475576872 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40680125f0 con 0x7f40780700f0 2024-03-29T04:19:02.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.895+0000 7f405ffff640 1 -- 172.21.15.22:0/2475576872 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4068019070 con 0x7f40780700f0 2024-03-29T04:19:02.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.895+0000 7f407dbdd640 1 -- 172.21.15.22:0/2475576872 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40780b4c20 con 0x7f40780700f0 2024-03-29T04:19:02.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.895+0000 7f405ffff640 1 -- 172.21.15.22:0/2475576872 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40680129b0 con 0x7f40780700f0 2024-03-29T04:19:02.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.895+0000 7f407dbdd640 1 -- 172.21.15.22:0/2475576872 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40780b4f00 con 0x7f40780700f0 2024-03-29T04:19:02.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.895+0000 7f407dbdd640 1 -- 172.21.15.22:0/2475576872 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4078068890 con 0x7f40780700f0 2024-03-29T04:19:02.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.895+0000 7f405ffff640 1 -- 172.21.15.22:0/2475576872 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f4068012b50 con 0x7f40780700f0 2024-03-29T04:19:02.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.895+0000 7f405ffff640 1 --2- 172.21.15.22:0/2475576872 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4064067ff0 0x7f406406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:02.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.895+0000 7f405ffff640 1 -- 172.21.15.22:0/2475576872 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f4068086b20 con 0x7f40780700f0 2024-03-29T04:19:02.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.895+0000 7f4076ffd640 1 --2- 172.21.15.22:0/2475576872 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4064067ff0 0x7f406406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:02.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.899+0000 7f4076ffd640 1 --2- 172.21.15.22:0/2475576872 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4064067ff0 0x7f406406a4b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4078160a30 tx=0x7f406c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:02.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:02.899+0000 7f405ffff640 1 -- 172.21.15.22:0/2475576872 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f40680570a0 con 0x7f40780700f0 2024-03-29T04:19:03.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.059+0000 7f407dbdd640 1 -- 172.21.15.22:0/2475576872 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f407806f410 con 0x7f4064067ff0 2024-03-29T04:19:03.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.075+0000 7f405ffff640 1 -- 172.21.15.22:0/2475576872 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246633 (secure 0 0 0) 0x7f407806f410 con 0x7f4064067ff0 2024-03-29T04:19:03.061 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:19:03.062 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-29T04:19:03.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.079+0000 7f407dbdd640 1 -- 172.21.15.22:0/2475576872 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4064067ff0 msgr2=0x7f406406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:03.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.079+0000 7f407dbdd640 1 --2- 172.21.15.22:0/2475576872 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4064067ff0 0x7f406406a4b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4078160a30 tx=0x7f406c009040 comp rx=0 tx=0).stop 2024-03-29T04:19:03.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.079+0000 7f407dbdd640 1 -- 172.21.15.22:0/2475576872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40780700f0 msgr2=0x7f407813d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:03.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.079+0000 7f407dbdd640 1 --2- 172.21.15.22:0/2475576872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40780700f0 0x7f407813d470 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f406800d5f0 tx=0x7f406800dab0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.079+0000 7f407dbdd640 1 -- 172.21.15.22:0/2475576872 shutdown_connections 2024-03-29T04:19:03.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.079+0000 7f407dbdd640 1 --2- 172.21.15.22:0/2475576872 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4078145d20 0x7f4078158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.079+0000 7f407dbdd640 1 --2- 172.21.15.22:0/2475576872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4078070a00 0x7f407815cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.079+0000 7f407dbdd640 1 --2- 172.21.15.22:0/2475576872 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4064067ff0 0x7f406406a4b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.079+0000 7f407dbdd640 1 --2- 172.21.15.22:0/2475576872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40780700f0 0x7f407813d470 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.079+0000 7f407dbdd640 1 -- 172.21.15.22:0/2475576872 >> 172.21.15.22:0/2475576872 conn(0x7f407806e410 msgr2=0x7f40780b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:03.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.079+0000 7f407dbdd640 1 -- 172.21.15.22:0/2475576872 shutdown_connections 2024-03-29T04:19:03.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.079+0000 7f407dbdd640 1 -- 172.21.15.22:0/2475576872 wait complete. 2024-03-29T04:19:03.080 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-03-29T04:19:01.721733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241488,"kb_used_data":4524,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733232,"statfs":{"total":773094113280,"available":772846829568,"internally_reserved":0,"allocated":4632576,"data_stored":2651217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.108925"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.625563+0000","last_change":"2024-03-29T04:18:57.625563+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.625563+0000","last_undegraded":"2024-03-29T04:18:57.625563+0000","last_fullsized":"2024-03-29T04:18:57.625563+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.625314+0000","last_change":"2024-03-29T04:18:57.625314+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.625314+0000","last_undegraded":"2024-03-29T04:18:57.625314+0000","last_fullsized":"2024-03-29T04:18:57.625314+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.622173+0000","last_change":"2024-03-29T04:18:57.622173+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.622173+0000","last_undegraded":"2024-03-29T04:18:57.622173+0000","last_fullsized":"2024-03-29T04:18:57.622173+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.628289+0000","last_change":"2024-03-29T04:18:57.628289+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.628289+0000","last_undegraded":"2024-03-29T04:18:57.628289+0000","last_fullsized":"2024-03-29T04:18:57.628289+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.627884+0000","last_change":"2024-03-29T04:18:57.627884+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.627884+0000","last_undegraded":"2024-03-29T04:18:57.627884+0000","last_fullsized":"2024-03-29T04:18:57.627884+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.629072+0000","last_change":"2024-03-29T04:18:57.629072+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.629072+0000","last_undegraded":"2024-03-29T04:18:57.629072+0000","last_fullsized":"2024-03-29T04:18:57.629072+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.638167+0000","last_change":"2024-03-29T04:18:57.638167+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.638167+0000","last_undegraded":"2024-03-29T04:18:57.638167+0000","last_fullsized":"2024-03-29T04:18:57.638167+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.624752+0000","last_change":"2024-03-29T04:18:57.624752+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.624752+0000","last_undegraded":"2024-03-29T04:18:57.624752+0000","last_fullsized":"2024-03-29T04:18:57.624752+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.621806+0000","last_change":"2024-03-29T04:18:57.621806+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.621806+0000","last_undegraded":"2024-03-29T04:18:57.621806+0000","last_fullsized":"2024-03-29T04:18:57.621806+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.635701+0000","last_change":"2024-03-29T04:18:57.635701+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.635701+0000","last_undegraded":"2024-03-29T04:18:57.635701+0000","last_fullsized":"2024-03-29T04:18:57.635701+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.632529+0000","last_change":"2024-03-29T04:18:57.632529+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.632529+0000","last_undegraded":"2024-03-29T04:18:57.632529+0000","last_fullsized":"2024-03-29T04:18:57.632529+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.621716+0000","last_change":"2024-03-29T04:18:57.621716+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.621716+0000","last_undegraded":"2024-03-29T04:18:57.621716+0000","last_fullsized":"2024-03-29T04:18:57.621716+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.636959+0000","last_change":"2024-03-29T04:18:57.636959+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.636959+0000","last_undegraded":"2024-03-29T04:18:57.636959+0000","last_fullsized":"2024-03-29T04:18:57.636959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.651723+0000","last_change":"2024-03-29T04:18:57.651723+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.651723+0000","last_undegraded":"2024-03-29T04:18:57.651723+0000","last_fullsized":"2024-03-29T04:18:57.651723+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.636989+0000","last_change":"2024-03-29T04:18:57.636989+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.636989+0000","last_undegraded":"2024-03-29T04:18:57.636989+0000","last_fullsized":"2024-03-29T04:18:57.636989+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.638252+0000","last_change":"2024-03-29T04:18:57.638252+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.638252+0000","last_undegraded":"2024-03-29T04:18:57.638252+0000","last_fullsized":"2024-03-29T04:18:57.638252+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.632094+0000","last_change":"2024-03-29T04:18:57.632094+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.632094+0000","last_undegraded":"2024-03-29T04:18:57.632094+0000","last_fullsized":"2024-03-29T04:18:57.632094+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.623561+0000","last_change":"2024-03-29T04:18:57.623561+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.623561+0000","last_undegraded":"2024-03-29T04:18:57.623561+0000","last_fullsized":"2024-03-29T04:18:57.623561+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.627332+0000","last_change":"2024-03-29T04:18:57.627332+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.627332+0000","last_undegraded":"2024-03-29T04:18:57.627332+0000","last_fullsized":"2024-03-29T04:18:57.627332+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.622588+0000","last_change":"2024-03-29T04:18:57.622588+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.622588+0000","last_undegraded":"2024-03-29T04:18:57.622588+0000","last_fullsized":"2024-03-29T04:18:57.622588+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.635894+0000","last_change":"2024-03-29T04:18:57.635894+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.635894+0000","last_undegraded":"2024-03-29T04:18:57.635894+0000","last_fullsized":"2024-03-29T04:18:57.635894+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.643079+0000","last_change":"2024-03-29T04:18:57.643079+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.643079+0000","last_undegraded":"2024-03-29T04:18:57.643079+0000","last_fullsized":"2024-03-29T04:18:57.643079+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.635697+0000","last_change":"2024-03-29T04:18:57.635697+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.635697+0000","last_undegraded":"2024-03-29T04:18:57.635697+0000","last_fullsized":"2024-03-29T04:18:57.635697+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.627435+0000","last_change":"2024-03-29T04:18:57.627435+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.627435+0000","last_undegraded":"2024-03-29T04:18:57.627435+0000","last_fullsized":"2024-03-29T04:18:57.627435+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.624839+0000","last_change":"2024-03-29T04:18:57.624839+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.624839+0000","last_undegraded":"2024-03-29T04:18:57.624839+0000","last_fullsized":"2024-03-29T04:18:57.624839+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.627398+0000","last_change":"2024-03-29T04:18:57.627398+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.627398+0000","last_undegraded":"2024-03-29T04:18:57.627398+0000","last_fullsized":"2024-03-29T04:18:57.627398+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.623666+0000","last_change":"2024-03-29T04:18:57.623666+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.623666+0000","last_undegraded":"2024-03-29T04:18:57.623666+0000","last_fullsized":"2024-03-29T04:18:57.623666+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.630493+0000","last_change":"2024-03-29T04:18:57.630493+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.630493+0000","last_undegraded":"2024-03-29T04:18:57.630493+0000","last_fullsized":"2024-03-29T04:18:57.630493+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.625750+0000","last_change":"2024-03-29T04:18:57.625750+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.625750+0000","last_undegraded":"2024-03-29T04:18:57.625750+0000","last_fullsized":"2024-03-29T04:18:57.625750+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.622621+0000","last_change":"2024-03-29T04:18:57.622621+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.622621+0000","last_undegraded":"2024-03-29T04:18:57.622621+0000","last_fullsized":"2024-03-29T04:18:57.622621+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.631780+0000","last_change":"2024-03-29T04:18:57.631780+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.631780+0000","last_undegraded":"2024-03-29T04:18:57.631780+0000","last_fullsized":"2024-03-29T04:18:57.631780+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.627065+0000","last_change":"2024-03-29T04:18:57.627065+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.627065+0000","last_undegraded":"2024-03-29T04:18:57.627065+0000","last_fullsized":"2024-03-29T04:18:57.627065+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.639988+0000","last_change":"2024-03-29T04:18:57.639988+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.639988+0000","last_undegraded":"2024-03-29T04:18:57.639988+0000","last_fullsized":"2024-03-29T04:18:57.639988+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.656457+0000","last_change":"2024-03-29T04:18:57.652890+0000","last_active":"2024-03-29T04:18:57.656457+0000","last_peered":"2024-03-29T04:18:57.656457+0000","last_clean":"2024-03-29T04:18:57.656457+0000","last_became_active":"2024-03-29T04:18:57.652244+0000","last_became_peered":"2024-03-29T04:18:57.652244+0000","last_unstale":"2024-03-29T04:18:57.656457+0000","last_undegraded":"2024-03-29T04:18:57.656457+0000","last_fullsized":"2024-03-29T04:18:57.656457+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:30:52.040122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.630597+0000","last_change":"2024-03-29T04:18:57.630597+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.630597+0000","last_undegraded":"2024-03-29T04:18:57.630597+0000","last_fullsized":"2024-03-29T04:18:57.630597+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.647365+0000","last_change":"2024-03-29T04:18:57.646783+0000","last_active":"2024-03-29T04:18:57.647365+0000","last_peered":"2024-03-29T04:18:57.647365+0000","last_clean":"2024-03-29T04:18:57.647365+0000","last_became_active":"2024-03-29T04:18:57.646495+0000","last_became_peered":"2024-03-29T04:18:57.646495+0000","last_unstale":"2024-03-29T04:18:57.647365+0000","last_undegraded":"2024-03-29T04:18:57.647365+0000","last_fullsized":"2024-03-29T04:18:57.647365+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:56:06.055281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.637914+0000","last_change":"2024-03-29T04:18:57.637914+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.637914+0000","last_undegraded":"2024-03-29T04:18:57.637914+0000","last_fullsized":"2024-03-29T04:18:57.637914+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.654763+0000","last_change":"2024-03-29T04:18:57.654191+0000","last_active":"2024-03-29T04:18:57.654763+0000","last_peered":"2024-03-29T04:18:57.654763+0000","last_clean":"2024-03-29T04:18:57.654763+0000","last_became_active":"2024-03-29T04:18:57.653911+0000","last_became_peered":"2024-03-29T04:18:57.653911+0000","last_unstale":"2024-03-29T04:18:57.654763+0000","last_undegraded":"2024-03-29T04:18:57.654763+0000","last_fullsized":"2024-03-29T04:18:57.654763+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:57.166873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.624736+0000","last_change":"2024-03-29T04:18:57.624736+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.624736+0000","last_undegraded":"2024-03-29T04:18:57.624736+0000","last_fullsized":"2024-03-29T04:18:57.624736+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.662129+0000","last_change":"2024-03-29T04:18:57.651021+0000","last_active":"2024-03-29T04:18:57.662129+0000","last_peered":"2024-03-29T04:18:57.662129+0000","last_clean":"2024-03-29T04:18:57.662129+0000","last_became_active":"2024-03-29T04:18:57.644336+0000","last_became_peered":"2024-03-29T04:18:57.644336+0000","last_unstale":"2024-03-29T04:18:57.662129+0000","last_undegraded":"2024-03-29T04:18:57.662129+0000","last_fullsized":"2024-03-29T04:18:57.662129+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:56:06.055281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.623072+0000","last_change":"2024-03-29T04:18:57.623072+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.623072+0000","last_undegraded":"2024-03-29T04:18:57.623072+0000","last_fullsized":"2024-03-29T04:18:57.623072+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.662014+0000","last_change":"2024-03-29T04:18:57.650116+0000","last_active":"2024-03-29T04:18:57.662014+0000","last_peered":"2024-03-29T04:18:57.662014+0000","last_clean":"2024-03-29T04:18:57.662014+0000","last_became_active":"2024-03-29T04:18:57.638212+0000","last_became_peered":"2024-03-29T04:18:57.638212+0000","last_unstale":"2024-03-29T04:18:57.662014+0000","last_undegraded":"2024-03-29T04:18:57.662014+0000","last_fullsized":"2024-03-29T04:18:57.662014+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:57.166873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.632972+0000","last_change":"2024-03-29T04:18:57.632972+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.632972+0000","last_undegraded":"2024-03-29T04:18:57.632972+0000","last_fullsized":"2024-03-29T04:18:57.632972+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.666853+0000","last_change":"2024-03-29T04:18:57.665159+0000","last_active":"2024-03-29T04:18:57.666853+0000","last_peered":"2024-03-29T04:18:57.666853+0000","last_clean":"2024-03-29T04:18:57.666853+0000","last_became_active":"2024-03-29T04:18:57.664442+0000","last_became_peered":"2024-03-29T04:18:57.664442+0000","last_unstale":"2024-03-29T04:18:57.666853+0000","last_undegraded":"2024-03-29T04:18:57.666853+0000","last_fullsized":"2024-03-29T04:18:57.666853+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:57.166873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.641871+0000","last_change":"2024-03-29T04:18:57.641871+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.641871+0000","last_undegraded":"2024-03-29T04:18:57.641871+0000","last_fullsized":"2024-03-29T04:18:57.641871+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.663425+0000","last_change":"2024-03-29T04:18:57.662680+0000","last_active":"2024-03-29T04:18:57.663425+0000","last_peered":"2024-03-29T04:18:57.663425+0000","last_clean":"2024-03-29T04:18:57.663425+0000","last_became_active":"2024-03-29T04:18:57.662148+0000","last_became_peered":"2024-03-29T04:18:57.662148+0000","last_unstale":"2024-03-29T04:18:57.663425+0000","last_undegraded":"2024-03-29T04:18:57.663425+0000","last_fullsized":"2024-03-29T04:18:57.663425+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:08:44.644431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.626138+0000","last_change":"2024-03-29T04:18:57.626138+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.626138+0000","last_undegraded":"2024-03-29T04:18:57.626138+0000","last_fullsized":"2024-03-29T04:18:57.626138+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.662773+0000","last_change":"2024-03-29T04:18:57.652332+0000","last_active":"2024-03-29T04:18:57.662773+0000","last_peered":"2024-03-29T04:18:57.662773+0000","last_clean":"2024-03-29T04:18:57.662773+0000","last_became_active":"2024-03-29T04:18:57.651656+0000","last_became_peered":"2024-03-29T04:18:57.651656+0000","last_unstale":"2024-03-29T04:18:57.662773+0000","last_undegraded":"2024-03-29T04:18:57.662773+0000","last_fullsized":"2024-03-29T04:18:57.662773+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:08:44.644431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.627438+0000","last_change":"2024-03-29T04:18:57.627438+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.627438+0000","last_undegraded":"2024-03-29T04:18:57.627438+0000","last_fullsized":"2024-03-29T04:18:57.627438+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.635115+0000","last_change":"2024-03-29T04:18:57.633865+0000","last_active":"2024-03-29T04:18:57.635115+0000","last_peered":"2024-03-29T04:18:57.635115+0000","last_clean":"2024-03-29T04:18:57.635115+0000","last_became_active":"2024-03-29T04:18:57.633449+0000","last_became_peered":"2024-03-29T04:18:57.633449+0000","last_unstale":"2024-03-29T04:18:57.635115+0000","last_undegraded":"2024-03-29T04:18:57.635115+0000","last_fullsized":"2024-03-29T04:18:57.635115+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:07:09.642311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.634764+0000","last_change":"2024-03-29T04:18:57.634764+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.634764+0000","last_undegraded":"2024-03-29T04:18:57.634764+0000","last_fullsized":"2024-03-29T04:18:57.634764+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.669540+0000","last_change":"2024-03-29T04:18:57.665734+0000","last_active":"2024-03-29T04:18:57.669540+0000","last_peered":"2024-03-29T04:18:57.669540+0000","last_clean":"2024-03-29T04:18:57.669540+0000","last_became_active":"2024-03-29T04:18:57.662309+0000","last_became_peered":"2024-03-29T04:18:57.662309+0000","last_unstale":"2024-03-29T04:18:57.669540+0000","last_undegraded":"2024-03-29T04:18:57.669540+0000","last_fullsized":"2024-03-29T04:18:57.669540+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:32.935677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.621183+0000","last_change":"2024-03-29T04:18:57.621183+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.621183+0000","last_undegraded":"2024-03-29T04:18:57.621183+0000","last_fullsized":"2024-03-29T04:18:57.621183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.666094+0000","last_change":"2024-03-29T04:18:57.660878+0000","last_active":"2024-03-29T04:18:57.666094+0000","last_peered":"2024-03-29T04:18:57.666094+0000","last_clean":"2024-03-29T04:18:57.666094+0000","last_became_active":"2024-03-29T04:18:57.659621+0000","last_became_peered":"2024-03-29T04:18:57.659621+0000","last_unstale":"2024-03-29T04:18:57.666094+0000","last_undegraded":"2024-03-29T04:18:57.666094+0000","last_fullsized":"2024-03-29T04:18:57.666094+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:08:44.644431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.634774+0000","last_change":"2024-03-29T04:18:57.634774+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.634774+0000","last_undegraded":"2024-03-29T04:18:57.634774+0000","last_fullsized":"2024-03-29T04:18:57.634774+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.650916+0000","last_change":"2024-03-29T04:18:57.650361+0000","last_active":"2024-03-29T04:18:57.650916+0000","last_peered":"2024-03-29T04:18:57.650916+0000","last_clean":"2024-03-29T04:18:57.650916+0000","last_became_active":"2024-03-29T04:18:57.650006+0000","last_became_peered":"2024-03-29T04:18:57.650006+0000","last_unstale":"2024-03-29T04:18:57.650916+0000","last_undegraded":"2024-03-29T04:18:57.650916+0000","last_fullsized":"2024-03-29T04:18:57.650916+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:34.306196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.622013+0000","last_change":"2024-03-29T04:18:57.622013+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.622013+0000","last_undegraded":"2024-03-29T04:18:57.622013+0000","last_fullsized":"2024-03-29T04:18:57.622013+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.634732+0000","last_change":"2024-03-29T04:18:57.633188+0000","last_active":"2024-03-29T04:18:57.634732+0000","last_peered":"2024-03-29T04:18:57.634732+0000","last_clean":"2024-03-29T04:18:57.634732+0000","last_became_active":"2024-03-29T04:18:57.632810+0000","last_became_peered":"2024-03-29T04:18:57.632810+0000","last_unstale":"2024-03-29T04:18:57.634732+0000","last_undegraded":"2024-03-29T04:18:57.634732+0000","last_fullsized":"2024-03-29T04:18:57.634732+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:58:15.229696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.624837+0000","last_change":"2024-03-29T04:18:57.624837+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.624837+0000","last_undegraded":"2024-03-29T04:18:57.624837+0000","last_fullsized":"2024-03-29T04:18:57.624837+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.667382+0000","last_change":"2024-03-29T04:18:57.666809+0000","last_active":"2024-03-29T04:18:57.667382+0000","last_peered":"2024-03-29T04:18:57.667382+0000","last_clean":"2024-03-29T04:18:57.667382+0000","last_became_active":"2024-03-29T04:18:57.666382+0000","last_became_peered":"2024-03-29T04:18:57.666382+0000","last_unstale":"2024-03-29T04:18:57.667382+0000","last_undegraded":"2024-03-29T04:18:57.667382+0000","last_fullsized":"2024-03-29T04:18:57.667382+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:32.935677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.630791+0000","last_change":"2024-03-29T04:18:57.630791+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.630791+0000","last_undegraded":"2024-03-29T04:18:57.630791+0000","last_fullsized":"2024-03-29T04:18:57.630791+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.655700+0000","last_change":"2024-03-29T04:18:57.653609+0000","last_active":"2024-03-29T04:18:57.655700+0000","last_peered":"2024-03-29T04:18:57.655700+0000","last_clean":"2024-03-29T04:18:57.655700+0000","last_became_active":"2024-03-29T04:18:57.653049+0000","last_became_peered":"2024-03-29T04:18:57.653049+0000","last_unstale":"2024-03-29T04:18:57.655700+0000","last_undegraded":"2024-03-29T04:18:57.655700+0000","last_fullsized":"2024-03-29T04:18:57.655700+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:14:26.382626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.625716+0000","last_change":"2024-03-29T04:18:57.625716+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.625716+0000","last_undegraded":"2024-03-29T04:18:57.625716+0000","last_fullsized":"2024-03-29T04:18:57.625716+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.667279+0000","last_change":"2024-03-29T04:18:57.664980+0000","last_active":"2024-03-29T04:18:57.667279+0000","last_peered":"2024-03-29T04:18:57.667279+0000","last_clean":"2024-03-29T04:18:57.667279+0000","last_became_active":"2024-03-29T04:18:57.664377+0000","last_became_peered":"2024-03-29T04:18:57.664377+0000","last_unstale":"2024-03-29T04:18:57.667279+0000","last_undegraded":"2024-03-29T04:18:57.667279+0000","last_fullsized":"2024-03-29T04:18:57.667279+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:37.267037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.624394+0000","last_change":"2024-03-29T04:18:57.624394+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.624394+0000","last_undegraded":"2024-03-29T04:18:57.624394+0000","last_fullsized":"2024-03-29T04:18:57.624394+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.669561+0000","last_change":"2024-03-29T04:18:57.666734+0000","last_active":"2024-03-29T04:18:57.669561+0000","last_peered":"2024-03-29T04:18:57.669561+0000","last_clean":"2024-03-29T04:18:57.669561+0000","last_became_active":"2024-03-29T04:18:57.666091+0000","last_became_peered":"2024-03-29T04:18:57.666091+0000","last_unstale":"2024-03-29T04:18:57.669561+0000","last_undegraded":"2024-03-29T04:18:57.669561+0000","last_fullsized":"2024-03-29T04:18:57.669561+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:51.553019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.631448+0000","last_change":"2024-03-29T04:18:57.631448+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.631448+0000","last_undegraded":"2024-03-29T04:18:57.631448+0000","last_fullsized":"2024-03-29T04:18:57.631448+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.670139+0000","last_change":"2024-03-29T04:18:57.666956+0000","last_active":"2024-03-29T04:18:57.670139+0000","last_peered":"2024-03-29T04:18:57.670139+0000","last_clean":"2024-03-29T04:18:57.670139+0000","last_became_active":"2024-03-29T04:18:57.666383+0000","last_became_peered":"2024-03-29T04:18:57.666383+0000","last_unstale":"2024-03-29T04:18:57.670139+0000","last_undegraded":"2024-03-29T04:18:57.670139+0000","last_fullsized":"2024-03-29T04:18:57.670139+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:32.935677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.639038+0000","last_change":"2024-03-29T04:18:57.639038+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.639038+0000","last_undegraded":"2024-03-29T04:18:57.639038+0000","last_fullsized":"2024-03-29T04:18:57.639038+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.650794+0000","last_change":"2024-03-29T04:18:57.649911+0000","last_active":"2024-03-29T04:18:57.650794+0000","last_peered":"2024-03-29T04:18:57.650794+0000","last_clean":"2024-03-29T04:18:57.650794+0000","last_became_active":"2024-03-29T04:18:57.649663+0000","last_became_peered":"2024-03-29T04:18:57.649663+0000","last_unstale":"2024-03-29T04:18:57.650794+0000","last_undegraded":"2024-03-29T04:18:57.650794+0000","last_fullsized":"2024-03-29T04:18:57.650794+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:05:33.397788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.627310+0000","last_change":"2024-03-29T04:18:57.627310+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.627310+0000","last_undegraded":"2024-03-29T04:18:57.627310+0000","last_fullsized":"2024-03-29T04:18:57.627310+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.655035+0000","last_change":"2024-03-29T04:18:57.654452+0000","last_active":"2024-03-29T04:18:57.655035+0000","last_peered":"2024-03-29T04:18:57.655035+0000","last_clean":"2024-03-29T04:18:57.655035+0000","last_became_active":"2024-03-29T04:18:57.654084+0000","last_became_peered":"2024-03-29T04:18:57.654084+0000","last_unstale":"2024-03-29T04:18:57.655035+0000","last_undegraded":"2024-03-29T04:18:57.655035+0000","last_fullsized":"2024-03-29T04:18:57.655035+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:56:06.055281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.632889+0000","last_change":"2024-03-29T04:18:57.632889+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.632889+0000","last_undegraded":"2024-03-29T04:18:57.632889+0000","last_fullsized":"2024-03-29T04:18:57.632889+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.635307+0000","last_change":"2024-03-29T04:18:57.633652+0000","last_active":"2024-03-29T04:18:57.635307+0000","last_peered":"2024-03-29T04:18:57.635307+0000","last_clean":"2024-03-29T04:18:57.635307+0000","last_became_active":"2024-03-29T04:18:57.633156+0000","last_became_peered":"2024-03-29T04:18:57.633156+0000","last_unstale":"2024-03-29T04:18:57.635307+0000","last_undegraded":"2024-03-29T04:18:57.635307+0000","last_fullsized":"2024-03-29T04:18:57.635307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:29:47.004303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.622136+0000","last_change":"2024-03-29T04:18:57.622136+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.622136+0000","last_undegraded":"2024-03-29T04:18:57.622136+0000","last_fullsized":"2024-03-29T04:18:57.622136+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.664068+0000","last_change":"2024-03-29T04:18:57.663600+0000","last_active":"2024-03-29T04:18:57.664068+0000","last_peered":"2024-03-29T04:18:57.664068+0000","last_clean":"2024-03-29T04:18:57.664068+0000","last_became_active":"2024-03-29T04:18:57.663253+0000","last_became_peered":"2024-03-29T04:18:57.663253+0000","last_unstale":"2024-03-29T04:18:57.664068+0000","last_undegraded":"2024-03-29T04:18:57.664068+0000","last_fullsized":"2024-03-29T04:18:57.664068+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:02:24.387380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.649106+0000","last_change":"2024-03-29T04:18:57.649106+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.649106+0000","last_undegraded":"2024-03-29T04:18:57.649106+0000","last_fullsized":"2024-03-29T04:18:57.649106+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.655742+0000","last_change":"2024-03-29T04:18:57.647273+0000","last_active":"2024-03-29T04:18:57.655742+0000","last_peered":"2024-03-29T04:18:57.655742+0000","last_clean":"2024-03-29T04:18:57.655742+0000","last_became_active":"2024-03-29T04:18:57.646106+0000","last_became_peered":"2024-03-29T04:18:57.646106+0000","last_unstale":"2024-03-29T04:18:57.655742+0000","last_undegraded":"2024-03-29T04:18:57.655742+0000","last_fullsized":"2024-03-29T04:18:57.655742+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:32:54.500069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.625166+0000","last_change":"2024-03-29T04:18:57.625166+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.625166+0000","last_undegraded":"2024-03-29T04:18:57.625166+0000","last_fullsized":"2024-03-29T04:18:57.625166+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.634816+0000","last_change":"2024-03-29T04:18:57.633301+0000","last_active":"2024-03-29T04:18:57.634816+0000","last_peered":"2024-03-29T04:18:57.634816+0000","last_clean":"2024-03-29T04:18:57.634816+0000","last_became_active":"2024-03-29T04:18:57.633009+0000","last_became_peered":"2024-03-29T04:18:57.633009+0000","last_unstale":"2024-03-29T04:18:57.634816+0000","last_undegraded":"2024-03-29T04:18:57.634816+0000","last_fullsized":"2024-03-29T04:18:57.634816+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:08:15.678077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.631700+0000","last_change":"2024-03-29T04:18:57.631700+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.631700+0000","last_undegraded":"2024-03-29T04:18:57.631700+0000","last_fullsized":"2024-03-29T04:18:57.631700+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.667043+0000","last_change":"2024-03-29T04:18:57.660454+0000","last_active":"2024-03-29T04:18:57.667043+0000","last_peered":"2024-03-29T04:18:57.667043+0000","last_clean":"2024-03-29T04:18:57.667043+0000","last_became_active":"2024-03-29T04:18:57.650654+0000","last_became_peered":"2024-03-29T04:18:57.650654+0000","last_unstale":"2024-03-29T04:18:57.667043+0000","last_undegraded":"2024-03-29T04:18:57.667043+0000","last_fullsized":"2024-03-29T04:18:57.667043+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:56:06.055281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.633061+0000","last_change":"2024-03-29T04:18:57.633061+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.633061+0000","last_undegraded":"2024-03-29T04:18:57.633061+0000","last_fullsized":"2024-03-29T04:18:57.633061+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.663471+0000","last_change":"2024-03-29T04:18:57.662501+0000","last_active":"2024-03-29T04:18:57.663471+0000","last_peered":"2024-03-29T04:18:57.663471+0000","last_clean":"2024-03-29T04:18:57.663471+0000","last_became_active":"2024-03-29T04:18:57.662145+0000","last_became_peered":"2024-03-29T04:18:57.662145+0000","last_unstale":"2024-03-29T04:18:57.663471+0000","last_undegraded":"2024-03-29T04:18:57.663471+0000","last_fullsized":"2024-03-29T04:18:57.663471+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:06:54.580779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.624796+0000","last_change":"2024-03-29T04:18:57.624796+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.624796+0000","last_undegraded":"2024-03-29T04:18:57.624796+0000","last_fullsized":"2024-03-29T04:18:57.624796+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.636288+0000","last_change":"2024-03-29T04:18:57.635473+0000","last_active":"2024-03-29T04:18:57.636288+0000","last_peered":"2024-03-29T04:18:57.636288+0000","last_clean":"2024-03-29T04:18:57.636288+0000","last_became_active":"2024-03-29T04:18:57.635129+0000","last_became_peered":"2024-03-29T04:18:57.635129+0000","last_unstale":"2024-03-29T04:18:57.636288+0000","last_undegraded":"2024-03-29T04:18:57.636288+0000","last_fullsized":"2024-03-29T04:18:57.636288+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:37.267037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.621515+0000","last_change":"2024-03-29T04:18:57.621515+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.621515+0000","last_undegraded":"2024-03-29T04:18:57.621515+0000","last_fullsized":"2024-03-29T04:18:57.621515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.642130+0000","last_change":"2024-03-29T04:18:57.641648+0000","last_active":"2024-03-29T04:18:57.642130+0000","last_peered":"2024-03-29T04:18:57.642130+0000","last_clean":"2024-03-29T04:18:57.642130+0000","last_became_active":"2024-03-29T04:18:57.641274+0000","last_became_peered":"2024-03-29T04:18:57.641274+0000","last_unstale":"2024-03-29T04:18:57.642130+0000","last_undegraded":"2024-03-29T04:18:57.642130+0000","last_fullsized":"2024-03-29T04:18:57.642130+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:57.166873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.628402+0000","last_change":"2024-03-29T04:18:57.628402+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.628402+0000","last_undegraded":"2024-03-29T04:18:57.628402+0000","last_fullsized":"2024-03-29T04:18:57.628402+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.642242+0000","last_change":"2024-03-29T04:18:57.636067+0000","last_active":"2024-03-29T04:18:57.642242+0000","last_peered":"2024-03-29T04:18:57.642242+0000","last_clean":"2024-03-29T04:18:57.642242+0000","last_became_active":"2024-03-29T04:18:57.635270+0000","last_became_peered":"2024-03-29T04:18:57.635270+0000","last_unstale":"2024-03-29T04:18:57.642242+0000","last_undegraded":"2024-03-29T04:18:57.642242+0000","last_fullsized":"2024-03-29T04:18:57.642242+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:45.160719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.644151+0000","last_change":"2024-03-29T04:18:57.644151+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.644151+0000","last_undegraded":"2024-03-29T04:18:57.644151+0000","last_fullsized":"2024-03-29T04:18:57.644151+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.661753+0000","last_change":"2024-03-29T04:18:57.643649+0000","last_active":"2024-03-29T04:18:57.661753+0000","last_peered":"2024-03-29T04:18:57.661753+0000","last_clean":"2024-03-29T04:18:57.661753+0000","last_became_active":"2024-03-29T04:18:57.642751+0000","last_became_peered":"2024-03-29T04:18:57.642751+0000","last_unstale":"2024-03-29T04:18:57.661753+0000","last_undegraded":"2024-03-29T04:18:57.661753+0000","last_fullsized":"2024-03-29T04:18:57.661753+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:57.166873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.630826+0000","last_change":"2024-03-29T04:18:57.630826+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.630826+0000","last_undegraded":"2024-03-29T04:18:57.630826+0000","last_fullsized":"2024-03-29T04:18:57.630826+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.634944+0000","last_change":"2024-03-29T04:18:57.633269+0000","last_active":"2024-03-29T04:18:57.634944+0000","last_peered":"2024-03-29T04:18:57.634944+0000","last_clean":"2024-03-29T04:18:57.634944+0000","last_became_active":"2024-03-29T04:18:57.632982+0000","last_became_peered":"2024-03-29T04:18:57.632982+0000","last_unstale":"2024-03-29T04:18:57.634944+0000","last_undegraded":"2024-03-29T04:18:57.634944+0000","last_fullsized":"2024-03-29T04:18:57.634944+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:56:45.800534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.647972+0000","last_change":"2024-03-29T04:18:57.647972+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.647972+0000","last_undegraded":"2024-03-29T04:18:57.647972+0000","last_fullsized":"2024-03-29T04:18:57.647972+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.658040+0000","last_change":"2024-03-29T04:18:57.655959+0000","last_active":"2024-03-29T04:18:57.658040+0000","last_peered":"2024-03-29T04:18:57.658040+0000","last_clean":"2024-03-29T04:18:57.658040+0000","last_became_active":"2024-03-29T04:18:57.651470+0000","last_became_peered":"2024-03-29T04:18:57.651470+0000","last_unstale":"2024-03-29T04:18:57.658040+0000","last_undegraded":"2024-03-29T04:18:57.658040+0000","last_fullsized":"2024-03-29T04:18:57.658040+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:18:30.179360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.639032+0000","last_change":"2024-03-29T04:17:25.361826+0000","last_active":"2024-03-29T04:18:57.639032+0000","last_peered":"2024-03-29T04:18:57.639032+0000","last_clean":"2024-03-29T04:18:57.639032+0000","last_became_active":"2024-03-29T04:17:25.361259+0000","last_became_peered":"2024-03-29T04:17:25.361259+0000","last_unstale":"2024-03-29T04:18:57.639032+0000","last_undegraded":"2024-03-29T04:18:57.639032+0000","last_fullsized":"2024-03-29T04:18:57.639032+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_clean_scrub_stamp":"2024-03-29T04:17:24.334639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:40.027873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1019904,"data_stored":771678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":181310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":438272,"data_stored":187977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":181310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":438272,"data_stored":187977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":438272,"data_stored":187977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":425984,"data_stored":181310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1019904,"data_stored":771678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:19:03.083 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:19:03.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.263+0000 7fce64b3f640 1 Processor -- start 2024-03-29T04:19:03.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.263+0000 7fce64b3f640 1 -- start start 2024-03-29T04:19:03.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.267+0000 7fce64b3f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce600aff40 0x7fce6015f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:03.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.267+0000 7fce64b3f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fce6006e690 0x7fce6006b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:03.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.267+0000 7fce64b3f640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce60158d90 con 0x7fce6006f730 2024-03-29T04:19:03.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.267+0000 7fce64b3f640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fce6006ee40 con 0x7fce6006e690 2024-03-29T04:19:03.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.267+0000 7fce64b3f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fce60161ea0 con 0x7fce600aff40 2024-03-29T04:19:03.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.267+0000 7fce5ed9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fce6006e690 0x7fce6006b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:03.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.267+0000 7fce5ed9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fce6006e690 0x7fce6006b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44388/0 (socket says 172.21.15.22:44388) 2024-03-29T04:19:03.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.267+0000 7fce5ed9d640 1 -- 172.21.15.22:0/303124676 learned_addr learned my addr 172.21.15.22:0/303124676 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:03.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.267+0000 7fce5e59c640 1 --2- 172.21.15.22:0/303124676 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce600aff40 0x7fce6015f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:03.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.267+0000 7fce5ed9d640 1 -- 172.21.15.22:0/303124676 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce600aff40 msgr2=0x7fce6015f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:03.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.267+0000 7fce5ed9d640 1 --2- 172.21.15.22:0/303124676 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce600aff40 0x7fce6015f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.267+0000 7fce5ed9d640 1 -- 172.21.15.22:0/303124676 >> v1:172.21.15.151:6789/0 conn(0x7fce6006f730 legacy=0x7fce6006df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:03.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.267+0000 7fce5ed9d640 1 -- 172.21.15.22:0/303124676 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce601620f0 con 0x7fce6006e690 2024-03-29T04:19:03.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.267+0000 7fce5e59c640 1 --2- 172.21.15.22:0/303124676 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce600aff40 0x7fce6015f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:19:03.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.267+0000 7fce5ed9d640 1 --2- 172.21.15.22:0/303124676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce6006e690 0x7fce6006b660 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fce58009e50 tx=0x7fce5801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c42ef08cbabc6c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:03.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.267+0000 7fce5dd9b640 1 -- 172.21.15.22:0/303124676 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce580090d0 con 0x7fce6006e690 2024-03-29T04:19:03.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.267+0000 7fce5dd9b640 1 -- 172.21.15.22:0/303124676 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce580ae070 con 0x7fce6006e690 2024-03-29T04:19:03.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.267+0000 7fce5dd9b640 1 -- 172.21.15.22:0/303124676 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58009270 con 0x7fce6006e690 2024-03-29T04:19:03.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce64b3f640 1 -- 172.21.15.22:0/303124676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce6006e690 msgr2=0x7fce6006b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:03.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce64b3f640 1 --2- 172.21.15.22:0/303124676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce6006e690 0x7fce6006b660 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fce58009e50 tx=0x7fce5801c780 comp rx=0 tx=0).stop 2024-03-29T04:19:03.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce64b3f640 1 -- 172.21.15.22:0/303124676 shutdown_connections 2024-03-29T04:19:03.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce64b3f640 1 --2- 172.21.15.22:0/303124676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce6006e690 0x7fce6006b660 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce64b3f640 1 --2- 172.21.15.22:0/303124676 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce600aff40 0x7fce6015f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce64b3f640 1 -- 172.21.15.22:0/303124676 >> 172.21.15.22:0/303124676 conn(0x7fce600b2b30 msgr2=0x7fce600d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:03.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce64b3f640 1 -- 172.21.15.22:0/303124676 shutdown_connections 2024-03-29T04:19:03.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce64b3f640 1 -- 172.21.15.22:0/303124676 wait complete. 2024-03-29T04:19:03.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce64b3f640 1 Processor -- start 2024-03-29T04:19:03.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce64b3f640 1 -- start start 2024-03-29T04:19:03.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce64b3f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce6006e690 0x7fce6015dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:03.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce64b3f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce6006f730 0x7fce6014d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:03.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce64b3f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce600aff40 0x7fce600b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:03.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce64b3f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fce6006c780 con 0x7fce600aff40 2024-03-29T04:19:03.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce64b3f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce6006c600 con 0x7fce6006f730 2024-03-29T04:19:03.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce64b3f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fce6016b240 con 0x7fce6006e690 2024-03-29T04:19:03.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce5ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce600aff40 0x7fce600b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:03.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce5ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce600aff40 0x7fce600b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44396/0 (socket says 172.21.15.22:44396) 2024-03-29T04:19:03.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce5ed9d640 1 -- 172.21.15.22:0/2448486182 learned_addr learned my addr 172.21.15.22:0/2448486182 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:03.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce55d9b640 1 --2- 172.21.15.22:0/2448486182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce6006f730 0x7fce6014d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:03.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce5e59c640 1 --2- 172.21.15.22:0/2448486182 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce6006e690 0x7fce6015dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:03.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce5ed9d640 1 -- 172.21.15.22:0/2448486182 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce6006e690 msgr2=0x7fce6015dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:03.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce5ed9d640 1 --2- 172.21.15.22:0/2448486182 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce6006e690 0x7fce6015dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce5ed9d640 1 -- 172.21.15.22:0/2448486182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce6006f730 msgr2=0x7fce6014d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:03.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce5ed9d640 1 --2- 172.21.15.22:0/2448486182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce6006f730 0x7fce6014d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce5ed9d640 1 -- 172.21.15.22:0/2448486182 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce6014f640 con 0x7fce600aff40 2024-03-29T04:19:03.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce5e59c640 1 --2- 172.21.15.22:0/2448486182 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce6006e690 0x7fce6015dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:19:03.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce5ed9d640 1 --2- 172.21.15.22:0/2448486182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce600aff40 0x7fce600b0320 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fce5801c5b0 tx=0x7fce58019520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:03.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce577fe640 1 -- 172.21.15.22:0/2448486182 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce580b8070 con 0x7fce600aff40 2024-03-29T04:19:03.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce577fe640 1 -- 172.21.15.22:0/2448486182 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce580a98b0 con 0x7fce600aff40 2024-03-29T04:19:03.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce64b3f640 1 -- 172.21.15.22:0/2448486182 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce6014eca0 con 0x7fce600aff40 2024-03-29T04:19:03.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce577fe640 1 -- 172.21.15.22:0/2448486182 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce580a9ca0 con 0x7fce600aff40 2024-03-29T04:19:03.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.271+0000 7fce64b3f640 1 -- 172.21.15.22:0/2448486182 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce600adb30 con 0x7fce600aff40 2024-03-29T04:19:03.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.275+0000 7fce64b3f640 1 -- 172.21.15.22:0/2448486182 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce60068890 con 0x7fce600aff40 2024-03-29T04:19:03.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.275+0000 7fce577fe640 1 -- 172.21.15.22:0/2448486182 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fce580c5440 con 0x7fce600aff40 2024-03-29T04:19:03.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.275+0000 7fce577fe640 1 --2- 172.21.15.22:0/2448486182 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fce40067fc0 0x7fce4006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:03.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.275+0000 7fce577fe640 1 -- 172.21.15.22:0/2448486182 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fce58120760 con 0x7fce600aff40 2024-03-29T04:19:03.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.275+0000 7fce5e59c640 1 --2- 172.21.15.22:0/2448486182 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fce40067fc0 0x7fce4006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:03.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.275+0000 7fce5e59c640 1 --2- 172.21.15.22:0/2448486182 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fce40067fc0 0x7fce4006a480 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fce50006fd0 tx=0x7fce50008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:03.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.283+0000 7fce577fe640 1 -- 172.21.15.22:0/2448486182 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fce6014f640 con 0x7fce600aff40 2024-03-29T04:19:03.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.475+0000 7fce64b3f640 1 -- 172.21.15.22:0/2448486182 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fce600d1470 con 0x7fce40067fc0 2024-03-29T04:19:03.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.495+0000 7fce577fe640 1 -- 172.21.15.22:0/2448486182 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246633 (secure 0 0 0) 0x7fce600d1470 con 0x7fce40067fc0 2024-03-29T04:19:03.478 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:19:03.479 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-29T04:19:03.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.499+0000 7fce64b3f640 1 -- 172.21.15.22:0/2448486182 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fce40067fc0 msgr2=0x7fce4006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:03.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.499+0000 7fce64b3f640 1 --2- 172.21.15.22:0/2448486182 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fce40067fc0 0x7fce4006a480 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fce50006fd0 tx=0x7fce50008040 comp rx=0 tx=0).stop 2024-03-29T04:19:03.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.499+0000 7fce64b3f640 1 -- 172.21.15.22:0/2448486182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce600aff40 msgr2=0x7fce600b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:03.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.499+0000 7fce64b3f640 1 --2- 172.21.15.22:0/2448486182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce600aff40 0x7fce600b0320 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fce5801c5b0 tx=0x7fce58019520 comp rx=0 tx=0).stop 2024-03-29T04:19:03.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.499+0000 7fce64b3f640 1 -- 172.21.15.22:0/2448486182 shutdown_connections 2024-03-29T04:19:03.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.499+0000 7fce64b3f640 1 --2- 172.21.15.22:0/2448486182 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fce40067fc0 0x7fce4006a480 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.499+0000 7fce64b3f640 1 --2- 172.21.15.22:0/2448486182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce600aff40 0x7fce600b0320 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.499+0000 7fce64b3f640 1 --2- 172.21.15.22:0/2448486182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce6006f730 0x7fce6014d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.499+0000 7fce64b3f640 1 --2- 172.21.15.22:0/2448486182 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fce6006e690 0x7fce6015dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.499+0000 7fce64b3f640 1 -- 172.21.15.22:0/2448486182 >> 172.21.15.22:0/2448486182 conn(0x7fce600b2b30 msgr2=0x7fce6015e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:03.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.499+0000 7fce64b3f640 1 -- 172.21.15.22:0/2448486182 shutdown_connections 2024-03-29T04:19:03.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.499+0000 7fce64b3f640 1 -- 172.21.15.22:0/2448486182 wait complete. 2024-03-29T04:19:03.495 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-03-29T04:19:01.721733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241488,"kb_used_data":4524,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733232,"statfs":{"total":773094113280,"available":772846829568,"internally_reserved":0,"allocated":4632576,"data_stored":2651217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.108925"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.625563+0000","last_change":"2024-03-29T04:18:57.625563+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.625563+0000","last_undegraded":"2024-03-29T04:18:57.625563+0000","last_fullsized":"2024-03-29T04:18:57.625563+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.625314+0000","last_change":"2024-03-29T04:18:57.625314+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.625314+0000","last_undegraded":"2024-03-29T04:18:57.625314+0000","last_fullsized":"2024-03-29T04:18:57.625314+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.622173+0000","last_change":"2024-03-29T04:18:57.622173+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.622173+0000","last_undegraded":"2024-03-29T04:18:57.622173+0000","last_fullsized":"2024-03-29T04:18:57.622173+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.628289+0000","last_change":"2024-03-29T04:18:57.628289+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.628289+0000","last_undegraded":"2024-03-29T04:18:57.628289+0000","last_fullsized":"2024-03-29T04:18:57.628289+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.627884+0000","last_change":"2024-03-29T04:18:57.627884+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.627884+0000","last_undegraded":"2024-03-29T04:18:57.627884+0000","last_fullsized":"2024-03-29T04:18:57.627884+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.629072+0000","last_change":"2024-03-29T04:18:57.629072+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.629072+0000","last_undegraded":"2024-03-29T04:18:57.629072+0000","last_fullsized":"2024-03-29T04:18:57.629072+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.638167+0000","last_change":"2024-03-29T04:18:57.638167+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.638167+0000","last_undegraded":"2024-03-29T04:18:57.638167+0000","last_fullsized":"2024-03-29T04:18:57.638167+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.624752+0000","last_change":"2024-03-29T04:18:57.624752+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.624752+0000","last_undegraded":"2024-03-29T04:18:57.624752+0000","last_fullsized":"2024-03-29T04:18:57.624752+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.621806+0000","last_change":"2024-03-29T04:18:57.621806+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.621806+0000","last_undegraded":"2024-03-29T04:18:57.621806+0000","last_fullsized":"2024-03-29T04:18:57.621806+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.635701+0000","last_change":"2024-03-29T04:18:57.635701+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.635701+0000","last_undegraded":"2024-03-29T04:18:57.635701+0000","last_fullsized":"2024-03-29T04:18:57.635701+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.632529+0000","last_change":"2024-03-29T04:18:57.632529+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.632529+0000","last_undegraded":"2024-03-29T04:18:57.632529+0000","last_fullsized":"2024-03-29T04:18:57.632529+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.621716+0000","last_change":"2024-03-29T04:18:57.621716+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.621716+0000","last_undegraded":"2024-03-29T04:18:57.621716+0000","last_fullsized":"2024-03-29T04:18:57.621716+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.636959+0000","last_change":"2024-03-29T04:18:57.636959+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.636959+0000","last_undegraded":"2024-03-29T04:18:57.636959+0000","last_fullsized":"2024-03-29T04:18:57.636959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.651723+0000","last_change":"2024-03-29T04:18:57.651723+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.651723+0000","last_undegraded":"2024-03-29T04:18:57.651723+0000","last_fullsized":"2024-03-29T04:18:57.651723+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.636989+0000","last_change":"2024-03-29T04:18:57.636989+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.636989+0000","last_undegraded":"2024-03-29T04:18:57.636989+0000","last_fullsized":"2024-03-29T04:18:57.636989+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.638252+0000","last_change":"2024-03-29T04:18:57.638252+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.638252+0000","last_undegraded":"2024-03-29T04:18:57.638252+0000","last_fullsized":"2024-03-29T04:18:57.638252+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.632094+0000","last_change":"2024-03-29T04:18:57.632094+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.632094+0000","last_undegraded":"2024-03-29T04:18:57.632094+0000","last_fullsized":"2024-03-29T04:18:57.632094+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.623561+0000","last_change":"2024-03-29T04:18:57.623561+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.623561+0000","last_undegraded":"2024-03-29T04:18:57.623561+0000","last_fullsized":"2024-03-29T04:18:57.623561+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.627332+0000","last_change":"2024-03-29T04:18:57.627332+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.627332+0000","last_undegraded":"2024-03-29T04:18:57.627332+0000","last_fullsized":"2024-03-29T04:18:57.627332+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.622588+0000","last_change":"2024-03-29T04:18:57.622588+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.622588+0000","last_undegraded":"2024-03-29T04:18:57.622588+0000","last_fullsized":"2024-03-29T04:18:57.622588+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.635894+0000","last_change":"2024-03-29T04:18:57.635894+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.635894+0000","last_undegraded":"2024-03-29T04:18:57.635894+0000","last_fullsized":"2024-03-29T04:18:57.635894+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.643079+0000","last_change":"2024-03-29T04:18:57.643079+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.643079+0000","last_undegraded":"2024-03-29T04:18:57.643079+0000","last_fullsized":"2024-03-29T04:18:57.643079+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.635697+0000","last_change":"2024-03-29T04:18:57.635697+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.635697+0000","last_undegraded":"2024-03-29T04:18:57.635697+0000","last_fullsized":"2024-03-29T04:18:57.635697+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.627435+0000","last_change":"2024-03-29T04:18:57.627435+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.627435+0000","last_undegraded":"2024-03-29T04:18:57.627435+0000","last_fullsized":"2024-03-29T04:18:57.627435+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.624839+0000","last_change":"2024-03-29T04:18:57.624839+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.624839+0000","last_undegraded":"2024-03-29T04:18:57.624839+0000","last_fullsized":"2024-03-29T04:18:57.624839+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.627398+0000","last_change":"2024-03-29T04:18:57.627398+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.627398+0000","last_undegraded":"2024-03-29T04:18:57.627398+0000","last_fullsized":"2024-03-29T04:18:57.627398+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.623666+0000","last_change":"2024-03-29T04:18:57.623666+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.623666+0000","last_undegraded":"2024-03-29T04:18:57.623666+0000","last_fullsized":"2024-03-29T04:18:57.623666+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.630493+0000","last_change":"2024-03-29T04:18:57.630493+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.630493+0000","last_undegraded":"2024-03-29T04:18:57.630493+0000","last_fullsized":"2024-03-29T04:18:57.630493+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.625750+0000","last_change":"2024-03-29T04:18:57.625750+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.625750+0000","last_undegraded":"2024-03-29T04:18:57.625750+0000","last_fullsized":"2024-03-29T04:18:57.625750+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.622621+0000","last_change":"2024-03-29T04:18:57.622621+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.622621+0000","last_undegraded":"2024-03-29T04:18:57.622621+0000","last_fullsized":"2024-03-29T04:18:57.622621+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.631780+0000","last_change":"2024-03-29T04:18:57.631780+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.631780+0000","last_undegraded":"2024-03-29T04:18:57.631780+0000","last_fullsized":"2024-03-29T04:18:57.631780+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.627065+0000","last_change":"2024-03-29T04:18:57.627065+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.627065+0000","last_undegraded":"2024-03-29T04:18:57.627065+0000","last_fullsized":"2024-03-29T04:18:57.627065+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.639988+0000","last_change":"2024-03-29T04:18:57.639988+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.639988+0000","last_undegraded":"2024-03-29T04:18:57.639988+0000","last_fullsized":"2024-03-29T04:18:57.639988+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.656457+0000","last_change":"2024-03-29T04:18:57.652890+0000","last_active":"2024-03-29T04:18:57.656457+0000","last_peered":"2024-03-29T04:18:57.656457+0000","last_clean":"2024-03-29T04:18:57.656457+0000","last_became_active":"2024-03-29T04:18:57.652244+0000","last_became_peered":"2024-03-29T04:18:57.652244+0000","last_unstale":"2024-03-29T04:18:57.656457+0000","last_undegraded":"2024-03-29T04:18:57.656457+0000","last_fullsized":"2024-03-29T04:18:57.656457+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:30:52.040122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.630597+0000","last_change":"2024-03-29T04:18:57.630597+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.630597+0000","last_undegraded":"2024-03-29T04:18:57.630597+0000","last_fullsized":"2024-03-29T04:18:57.630597+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.647365+0000","last_change":"2024-03-29T04:18:57.646783+0000","last_active":"2024-03-29T04:18:57.647365+0000","last_peered":"2024-03-29T04:18:57.647365+0000","last_clean":"2024-03-29T04:18:57.647365+0000","last_became_active":"2024-03-29T04:18:57.646495+0000","last_became_peered":"2024-03-29T04:18:57.646495+0000","last_unstale":"2024-03-29T04:18:57.647365+0000","last_undegraded":"2024-03-29T04:18:57.647365+0000","last_fullsized":"2024-03-29T04:18:57.647365+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:56:06.055281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.637914+0000","last_change":"2024-03-29T04:18:57.637914+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.637914+0000","last_undegraded":"2024-03-29T04:18:57.637914+0000","last_fullsized":"2024-03-29T04:18:57.637914+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.654763+0000","last_change":"2024-03-29T04:18:57.654191+0000","last_active":"2024-03-29T04:18:57.654763+0000","last_peered":"2024-03-29T04:18:57.654763+0000","last_clean":"2024-03-29T04:18:57.654763+0000","last_became_active":"2024-03-29T04:18:57.653911+0000","last_became_peered":"2024-03-29T04:18:57.653911+0000","last_unstale":"2024-03-29T04:18:57.654763+0000","last_undegraded":"2024-03-29T04:18:57.654763+0000","last_fullsized":"2024-03-29T04:18:57.654763+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:57.166873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.624736+0000","last_change":"2024-03-29T04:18:57.624736+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.624736+0000","last_undegraded":"2024-03-29T04:18:57.624736+0000","last_fullsized":"2024-03-29T04:18:57.624736+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.662129+0000","last_change":"2024-03-29T04:18:57.651021+0000","last_active":"2024-03-29T04:18:57.662129+0000","last_peered":"2024-03-29T04:18:57.662129+0000","last_clean":"2024-03-29T04:18:57.662129+0000","last_became_active":"2024-03-29T04:18:57.644336+0000","last_became_peered":"2024-03-29T04:18:57.644336+0000","last_unstale":"2024-03-29T04:18:57.662129+0000","last_undegraded":"2024-03-29T04:18:57.662129+0000","last_fullsized":"2024-03-29T04:18:57.662129+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:56:06.055281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.623072+0000","last_change":"2024-03-29T04:18:57.623072+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.623072+0000","last_undegraded":"2024-03-29T04:18:57.623072+0000","last_fullsized":"2024-03-29T04:18:57.623072+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.662014+0000","last_change":"2024-03-29T04:18:57.650116+0000","last_active":"2024-03-29T04:18:57.662014+0000","last_peered":"2024-03-29T04:18:57.662014+0000","last_clean":"2024-03-29T04:18:57.662014+0000","last_became_active":"2024-03-29T04:18:57.638212+0000","last_became_peered":"2024-03-29T04:18:57.638212+0000","last_unstale":"2024-03-29T04:18:57.662014+0000","last_undegraded":"2024-03-29T04:18:57.662014+0000","last_fullsized":"2024-03-29T04:18:57.662014+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:57.166873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.632972+0000","last_change":"2024-03-29T04:18:57.632972+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.632972+0000","last_undegraded":"2024-03-29T04:18:57.632972+0000","last_fullsized":"2024-03-29T04:18:57.632972+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.666853+0000","last_change":"2024-03-29T04:18:57.665159+0000","last_active":"2024-03-29T04:18:57.666853+0000","last_peered":"2024-03-29T04:18:57.666853+0000","last_clean":"2024-03-29T04:18:57.666853+0000","last_became_active":"2024-03-29T04:18:57.664442+0000","last_became_peered":"2024-03-29T04:18:57.664442+0000","last_unstale":"2024-03-29T04:18:57.666853+0000","last_undegraded":"2024-03-29T04:18:57.666853+0000","last_fullsized":"2024-03-29T04:18:57.666853+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:57.166873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.641871+0000","last_change":"2024-03-29T04:18:57.641871+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.641871+0000","last_undegraded":"2024-03-29T04:18:57.641871+0000","last_fullsized":"2024-03-29T04:18:57.641871+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.663425+0000","last_change":"2024-03-29T04:18:57.662680+0000","last_active":"2024-03-29T04:18:57.663425+0000","last_peered":"2024-03-29T04:18:57.663425+0000","last_clean":"2024-03-29T04:18:57.663425+0000","last_became_active":"2024-03-29T04:18:57.662148+0000","last_became_peered":"2024-03-29T04:18:57.662148+0000","last_unstale":"2024-03-29T04:18:57.663425+0000","last_undegraded":"2024-03-29T04:18:57.663425+0000","last_fullsized":"2024-03-29T04:18:57.663425+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:08:44.644431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.626138+0000","last_change":"2024-03-29T04:18:57.626138+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.626138+0000","last_undegraded":"2024-03-29T04:18:57.626138+0000","last_fullsized":"2024-03-29T04:18:57.626138+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.662773+0000","last_change":"2024-03-29T04:18:57.652332+0000","last_active":"2024-03-29T04:18:57.662773+0000","last_peered":"2024-03-29T04:18:57.662773+0000","last_clean":"2024-03-29T04:18:57.662773+0000","last_became_active":"2024-03-29T04:18:57.651656+0000","last_became_peered":"2024-03-29T04:18:57.651656+0000","last_unstale":"2024-03-29T04:18:57.662773+0000","last_undegraded":"2024-03-29T04:18:57.662773+0000","last_fullsized":"2024-03-29T04:18:57.662773+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:08:44.644431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.627438+0000","last_change":"2024-03-29T04:18:57.627438+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.627438+0000","last_undegraded":"2024-03-29T04:18:57.627438+0000","last_fullsized":"2024-03-29T04:18:57.627438+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.635115+0000","last_change":"2024-03-29T04:18:57.633865+0000","last_active":"2024-03-29T04:18:57.635115+0000","last_peered":"2024-03-29T04:18:57.635115+0000","last_clean":"2024-03-29T04:18:57.635115+0000","last_became_active":"2024-03-29T04:18:57.633449+0000","last_became_peered":"2024-03-29T04:18:57.633449+0000","last_unstale":"2024-03-29T04:18:57.635115+0000","last_undegraded":"2024-03-29T04:18:57.635115+0000","last_fullsized":"2024-03-29T04:18:57.635115+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:07:09.642311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.634764+0000","last_change":"2024-03-29T04:18:57.634764+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.634764+0000","last_undegraded":"2024-03-29T04:18:57.634764+0000","last_fullsized":"2024-03-29T04:18:57.634764+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.669540+0000","last_change":"2024-03-29T04:18:57.665734+0000","last_active":"2024-03-29T04:18:57.669540+0000","last_peered":"2024-03-29T04:18:57.669540+0000","last_clean":"2024-03-29T04:18:57.669540+0000","last_became_active":"2024-03-29T04:18:57.662309+0000","last_became_peered":"2024-03-29T04:18:57.662309+0000","last_unstale":"2024-03-29T04:18:57.669540+0000","last_undegraded":"2024-03-29T04:18:57.669540+0000","last_fullsized":"2024-03-29T04:18:57.669540+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:32.935677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.621183+0000","last_change":"2024-03-29T04:18:57.621183+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.621183+0000","last_undegraded":"2024-03-29T04:18:57.621183+0000","last_fullsized":"2024-03-29T04:18:57.621183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.666094+0000","last_change":"2024-03-29T04:18:57.660878+0000","last_active":"2024-03-29T04:18:57.666094+0000","last_peered":"2024-03-29T04:18:57.666094+0000","last_clean":"2024-03-29T04:18:57.666094+0000","last_became_active":"2024-03-29T04:18:57.659621+0000","last_became_peered":"2024-03-29T04:18:57.659621+0000","last_unstale":"2024-03-29T04:18:57.666094+0000","last_undegraded":"2024-03-29T04:18:57.666094+0000","last_fullsized":"2024-03-29T04:18:57.666094+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:08:44.644431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.634774+0000","last_change":"2024-03-29T04:18:57.634774+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.634774+0000","last_undegraded":"2024-03-29T04:18:57.634774+0000","last_fullsized":"2024-03-29T04:18:57.634774+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.650916+0000","last_change":"2024-03-29T04:18:57.650361+0000","last_active":"2024-03-29T04:18:57.650916+0000","last_peered":"2024-03-29T04:18:57.650916+0000","last_clean":"2024-03-29T04:18:57.650916+0000","last_became_active":"2024-03-29T04:18:57.650006+0000","last_became_peered":"2024-03-29T04:18:57.650006+0000","last_unstale":"2024-03-29T04:18:57.650916+0000","last_undegraded":"2024-03-29T04:18:57.650916+0000","last_fullsized":"2024-03-29T04:18:57.650916+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:34.306196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.622013+0000","last_change":"2024-03-29T04:18:57.622013+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.622013+0000","last_undegraded":"2024-03-29T04:18:57.622013+0000","last_fullsized":"2024-03-29T04:18:57.622013+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.634732+0000","last_change":"2024-03-29T04:18:57.633188+0000","last_active":"2024-03-29T04:18:57.634732+0000","last_peered":"2024-03-29T04:18:57.634732+0000","last_clean":"2024-03-29T04:18:57.634732+0000","last_became_active":"2024-03-29T04:18:57.632810+0000","last_became_peered":"2024-03-29T04:18:57.632810+0000","last_unstale":"2024-03-29T04:18:57.634732+0000","last_undegraded":"2024-03-29T04:18:57.634732+0000","last_fullsized":"2024-03-29T04:18:57.634732+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:58:15.229696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.624837+0000","last_change":"2024-03-29T04:18:57.624837+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.624837+0000","last_undegraded":"2024-03-29T04:18:57.624837+0000","last_fullsized":"2024-03-29T04:18:57.624837+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.667382+0000","last_change":"2024-03-29T04:18:57.666809+0000","last_active":"2024-03-29T04:18:57.667382+0000","last_peered":"2024-03-29T04:18:57.667382+0000","last_clean":"2024-03-29T04:18:57.667382+0000","last_became_active":"2024-03-29T04:18:57.666382+0000","last_became_peered":"2024-03-29T04:18:57.666382+0000","last_unstale":"2024-03-29T04:18:57.667382+0000","last_undegraded":"2024-03-29T04:18:57.667382+0000","last_fullsized":"2024-03-29T04:18:57.667382+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:32.935677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.630791+0000","last_change":"2024-03-29T04:18:57.630791+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.630791+0000","last_undegraded":"2024-03-29T04:18:57.630791+0000","last_fullsized":"2024-03-29T04:18:57.630791+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.655700+0000","last_change":"2024-03-29T04:18:57.653609+0000","last_active":"2024-03-29T04:18:57.655700+0000","last_peered":"2024-03-29T04:18:57.655700+0000","last_clean":"2024-03-29T04:18:57.655700+0000","last_became_active":"2024-03-29T04:18:57.653049+0000","last_became_peered":"2024-03-29T04:18:57.653049+0000","last_unstale":"2024-03-29T04:18:57.655700+0000","last_undegraded":"2024-03-29T04:18:57.655700+0000","last_fullsized":"2024-03-29T04:18:57.655700+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:14:26.382626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.625716+0000","last_change":"2024-03-29T04:18:57.625716+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.625716+0000","last_undegraded":"2024-03-29T04:18:57.625716+0000","last_fullsized":"2024-03-29T04:18:57.625716+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.667279+0000","last_change":"2024-03-29T04:18:57.664980+0000","last_active":"2024-03-29T04:18:57.667279+0000","last_peered":"2024-03-29T04:18:57.667279+0000","last_clean":"2024-03-29T04:18:57.667279+0000","last_became_active":"2024-03-29T04:18:57.664377+0000","last_became_peered":"2024-03-29T04:18:57.664377+0000","last_unstale":"2024-03-29T04:18:57.667279+0000","last_undegraded":"2024-03-29T04:18:57.667279+0000","last_fullsized":"2024-03-29T04:18:57.667279+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:37.267037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.624394+0000","last_change":"2024-03-29T04:18:57.624394+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.624394+0000","last_undegraded":"2024-03-29T04:18:57.624394+0000","last_fullsized":"2024-03-29T04:18:57.624394+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.669561+0000","last_change":"2024-03-29T04:18:57.666734+0000","last_active":"2024-03-29T04:18:57.669561+0000","last_peered":"2024-03-29T04:18:57.669561+0000","last_clean":"2024-03-29T04:18:57.669561+0000","last_became_active":"2024-03-29T04:18:57.666091+0000","last_became_peered":"2024-03-29T04:18:57.666091+0000","last_unstale":"2024-03-29T04:18:57.669561+0000","last_undegraded":"2024-03-29T04:18:57.669561+0000","last_fullsized":"2024-03-29T04:18:57.669561+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:51.553019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.631448+0000","last_change":"2024-03-29T04:18:57.631448+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.631448+0000","last_undegraded":"2024-03-29T04:18:57.631448+0000","last_fullsized":"2024-03-29T04:18:57.631448+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.670139+0000","last_change":"2024-03-29T04:18:57.666956+0000","last_active":"2024-03-29T04:18:57.670139+0000","last_peered":"2024-03-29T04:18:57.670139+0000","last_clean":"2024-03-29T04:18:57.670139+0000","last_became_active":"2024-03-29T04:18:57.666383+0000","last_became_peered":"2024-03-29T04:18:57.666383+0000","last_unstale":"2024-03-29T04:18:57.670139+0000","last_undegraded":"2024-03-29T04:18:57.670139+0000","last_fullsized":"2024-03-29T04:18:57.670139+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:32.935677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.639038+0000","last_change":"2024-03-29T04:18:57.639038+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.639038+0000","last_undegraded":"2024-03-29T04:18:57.639038+0000","last_fullsized":"2024-03-29T04:18:57.639038+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.650794+0000","last_change":"2024-03-29T04:18:57.649911+0000","last_active":"2024-03-29T04:18:57.650794+0000","last_peered":"2024-03-29T04:18:57.650794+0000","last_clean":"2024-03-29T04:18:57.650794+0000","last_became_active":"2024-03-29T04:18:57.649663+0000","last_became_peered":"2024-03-29T04:18:57.649663+0000","last_unstale":"2024-03-29T04:18:57.650794+0000","last_undegraded":"2024-03-29T04:18:57.650794+0000","last_fullsized":"2024-03-29T04:18:57.650794+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:05:33.397788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.627310+0000","last_change":"2024-03-29T04:18:57.627310+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.627310+0000","last_undegraded":"2024-03-29T04:18:57.627310+0000","last_fullsized":"2024-03-29T04:18:57.627310+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.655035+0000","last_change":"2024-03-29T04:18:57.654452+0000","last_active":"2024-03-29T04:18:57.655035+0000","last_peered":"2024-03-29T04:18:57.655035+0000","last_clean":"2024-03-29T04:18:57.655035+0000","last_became_active":"2024-03-29T04:18:57.654084+0000","last_became_peered":"2024-03-29T04:18:57.654084+0000","last_unstale":"2024-03-29T04:18:57.655035+0000","last_undegraded":"2024-03-29T04:18:57.655035+0000","last_fullsized":"2024-03-29T04:18:57.655035+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:56:06.055281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.632889+0000","last_change":"2024-03-29T04:18:57.632889+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.632889+0000","last_undegraded":"2024-03-29T04:18:57.632889+0000","last_fullsized":"2024-03-29T04:18:57.632889+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.635307+0000","last_change":"2024-03-29T04:18:57.633652+0000","last_active":"2024-03-29T04:18:57.635307+0000","last_peered":"2024-03-29T04:18:57.635307+0000","last_clean":"2024-03-29T04:18:57.635307+0000","last_became_active":"2024-03-29T04:18:57.633156+0000","last_became_peered":"2024-03-29T04:18:57.633156+0000","last_unstale":"2024-03-29T04:18:57.635307+0000","last_undegraded":"2024-03-29T04:18:57.635307+0000","last_fullsized":"2024-03-29T04:18:57.635307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:29:47.004303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.622136+0000","last_change":"2024-03-29T04:18:57.622136+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.622136+0000","last_undegraded":"2024-03-29T04:18:57.622136+0000","last_fullsized":"2024-03-29T04:18:57.622136+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.664068+0000","last_change":"2024-03-29T04:18:57.663600+0000","last_active":"2024-03-29T04:18:57.664068+0000","last_peered":"2024-03-29T04:18:57.664068+0000","last_clean":"2024-03-29T04:18:57.664068+0000","last_became_active":"2024-03-29T04:18:57.663253+0000","last_became_peered":"2024-03-29T04:18:57.663253+0000","last_unstale":"2024-03-29T04:18:57.664068+0000","last_undegraded":"2024-03-29T04:18:57.664068+0000","last_fullsized":"2024-03-29T04:18:57.664068+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:02:24.387380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.649106+0000","last_change":"2024-03-29T04:18:57.649106+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.649106+0000","last_undegraded":"2024-03-29T04:18:57.649106+0000","last_fullsized":"2024-03-29T04:18:57.649106+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.655742+0000","last_change":"2024-03-29T04:18:57.647273+0000","last_active":"2024-03-29T04:18:57.655742+0000","last_peered":"2024-03-29T04:18:57.655742+0000","last_clean":"2024-03-29T04:18:57.655742+0000","last_became_active":"2024-03-29T04:18:57.646106+0000","last_became_peered":"2024-03-29T04:18:57.646106+0000","last_unstale":"2024-03-29T04:18:57.655742+0000","last_undegraded":"2024-03-29T04:18:57.655742+0000","last_fullsized":"2024-03-29T04:18:57.655742+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:32:54.500069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.625166+0000","last_change":"2024-03-29T04:18:57.625166+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.625166+0000","last_undegraded":"2024-03-29T04:18:57.625166+0000","last_fullsized":"2024-03-29T04:18:57.625166+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.634816+0000","last_change":"2024-03-29T04:18:57.633301+0000","last_active":"2024-03-29T04:18:57.634816+0000","last_peered":"2024-03-29T04:18:57.634816+0000","last_clean":"2024-03-29T04:18:57.634816+0000","last_became_active":"2024-03-29T04:18:57.633009+0000","last_became_peered":"2024-03-29T04:18:57.633009+0000","last_unstale":"2024-03-29T04:18:57.634816+0000","last_undegraded":"2024-03-29T04:18:57.634816+0000","last_fullsized":"2024-03-29T04:18:57.634816+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:08:15.678077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.631700+0000","last_change":"2024-03-29T04:18:57.631700+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.631700+0000","last_undegraded":"2024-03-29T04:18:57.631700+0000","last_fullsized":"2024-03-29T04:18:57.631700+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.667043+0000","last_change":"2024-03-29T04:18:57.660454+0000","last_active":"2024-03-29T04:18:57.667043+0000","last_peered":"2024-03-29T04:18:57.667043+0000","last_clean":"2024-03-29T04:18:57.667043+0000","last_became_active":"2024-03-29T04:18:57.650654+0000","last_became_peered":"2024-03-29T04:18:57.650654+0000","last_unstale":"2024-03-29T04:18:57.667043+0000","last_undegraded":"2024-03-29T04:18:57.667043+0000","last_fullsized":"2024-03-29T04:18:57.667043+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:56:06.055281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.633061+0000","last_change":"2024-03-29T04:18:57.633061+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.633061+0000","last_undegraded":"2024-03-29T04:18:57.633061+0000","last_fullsized":"2024-03-29T04:18:57.633061+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.663471+0000","last_change":"2024-03-29T04:18:57.662501+0000","last_active":"2024-03-29T04:18:57.663471+0000","last_peered":"2024-03-29T04:18:57.663471+0000","last_clean":"2024-03-29T04:18:57.663471+0000","last_became_active":"2024-03-29T04:18:57.662145+0000","last_became_peered":"2024-03-29T04:18:57.662145+0000","last_unstale":"2024-03-29T04:18:57.663471+0000","last_undegraded":"2024-03-29T04:18:57.663471+0000","last_fullsized":"2024-03-29T04:18:57.663471+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:06:54.580779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.624796+0000","last_change":"2024-03-29T04:18:57.624796+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.624796+0000","last_undegraded":"2024-03-29T04:18:57.624796+0000","last_fullsized":"2024-03-29T04:18:57.624796+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.636288+0000","last_change":"2024-03-29T04:18:57.635473+0000","last_active":"2024-03-29T04:18:57.636288+0000","last_peered":"2024-03-29T04:18:57.636288+0000","last_clean":"2024-03-29T04:18:57.636288+0000","last_became_active":"2024-03-29T04:18:57.635129+0000","last_became_peered":"2024-03-29T04:18:57.635129+0000","last_unstale":"2024-03-29T04:18:57.636288+0000","last_undegraded":"2024-03-29T04:18:57.636288+0000","last_fullsized":"2024-03-29T04:18:57.636288+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:37.267037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.621515+0000","last_change":"2024-03-29T04:18:57.621515+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.621515+0000","last_undegraded":"2024-03-29T04:18:57.621515+0000","last_fullsized":"2024-03-29T04:18:57.621515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.642130+0000","last_change":"2024-03-29T04:18:57.641648+0000","last_active":"2024-03-29T04:18:57.642130+0000","last_peered":"2024-03-29T04:18:57.642130+0000","last_clean":"2024-03-29T04:18:57.642130+0000","last_became_active":"2024-03-29T04:18:57.641274+0000","last_became_peered":"2024-03-29T04:18:57.641274+0000","last_unstale":"2024-03-29T04:18:57.642130+0000","last_undegraded":"2024-03-29T04:18:57.642130+0000","last_fullsized":"2024-03-29T04:18:57.642130+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:57.166873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.628402+0000","last_change":"2024-03-29T04:18:57.628402+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.628402+0000","last_undegraded":"2024-03-29T04:18:57.628402+0000","last_fullsized":"2024-03-29T04:18:57.628402+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.642242+0000","last_change":"2024-03-29T04:18:57.636067+0000","last_active":"2024-03-29T04:18:57.642242+0000","last_peered":"2024-03-29T04:18:57.642242+0000","last_clean":"2024-03-29T04:18:57.642242+0000","last_became_active":"2024-03-29T04:18:57.635270+0000","last_became_peered":"2024-03-29T04:18:57.635270+0000","last_unstale":"2024-03-29T04:18:57.642242+0000","last_undegraded":"2024-03-29T04:18:57.642242+0000","last_fullsized":"2024-03-29T04:18:57.642242+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:45.160719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.644151+0000","last_change":"2024-03-29T04:18:57.644151+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.644151+0000","last_undegraded":"2024-03-29T04:18:57.644151+0000","last_fullsized":"2024-03-29T04:18:57.644151+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.661753+0000","last_change":"2024-03-29T04:18:57.643649+0000","last_active":"2024-03-29T04:18:57.661753+0000","last_peered":"2024-03-29T04:18:57.661753+0000","last_clean":"2024-03-29T04:18:57.661753+0000","last_became_active":"2024-03-29T04:18:57.642751+0000","last_became_peered":"2024-03-29T04:18:57.642751+0000","last_unstale":"2024-03-29T04:18:57.661753+0000","last_undegraded":"2024-03-29T04:18:57.661753+0000","last_fullsized":"2024-03-29T04:18:57.661753+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:57.166873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.630826+0000","last_change":"2024-03-29T04:18:57.630826+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.630826+0000","last_undegraded":"2024-03-29T04:18:57.630826+0000","last_fullsized":"2024-03-29T04:18:57.630826+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.634944+0000","last_change":"2024-03-29T04:18:57.633269+0000","last_active":"2024-03-29T04:18:57.634944+0000","last_peered":"2024-03-29T04:18:57.634944+0000","last_clean":"2024-03-29T04:18:57.634944+0000","last_became_active":"2024-03-29T04:18:57.632982+0000","last_became_peered":"2024-03-29T04:18:57.632982+0000","last_unstale":"2024-03-29T04:18:57.634944+0000","last_undegraded":"2024-03-29T04:18:57.634944+0000","last_fullsized":"2024-03-29T04:18:57.634944+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:56:45.800534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-29T04:18:57.647972+0000","last_change":"2024-03-29T04:18:57.647972+0000","last_active":"2024-03-29T04:18:57.609853+0000","last_peered":"2024-03-29T04:18:57.609853+0000","last_clean":"2024-03-29T04:18:57.609853+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:18:57.647972+0000","last_undegraded":"2024-03-29T04:18:57.647972+0000","last_fullsized":"2024-03-29T04:18:57.647972+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.658040+0000","last_change":"2024-03-29T04:18:57.655959+0000","last_active":"2024-03-29T04:18:57.658040+0000","last_peered":"2024-03-29T04:18:57.658040+0000","last_clean":"2024-03-29T04:18:57.658040+0000","last_became_active":"2024-03-29T04:18:57.651470+0000","last_became_peered":"2024-03-29T04:18:57.651470+0000","last_unstale":"2024-03-29T04:18:57.658040+0000","last_undegraded":"2024-03-29T04:18:57.658040+0000","last_fullsized":"2024-03-29T04:18:57.658040+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:18:30.179360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:18:57.639032+0000","last_change":"2024-03-29T04:17:25.361826+0000","last_active":"2024-03-29T04:18:57.639032+0000","last_peered":"2024-03-29T04:18:57.639032+0000","last_clean":"2024-03-29T04:18:57.639032+0000","last_became_active":"2024-03-29T04:17:25.361259+0000","last_became_peered":"2024-03-29T04:17:25.361259+0000","last_unstale":"2024-03-29T04:18:57.639032+0000","last_undegraded":"2024-03-29T04:18:57.639032+0000","last_fullsized":"2024-03-29T04:18:57.639032+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_clean_scrub_stamp":"2024-03-29T04:17:24.334639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:40.027873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1019904,"data_stored":771678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":181310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":438272,"data_stored":187977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":181310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":438272,"data_stored":187977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":438272,"data_stored":187977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":425984,"data_stored":181310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1019904,"data_stored":771678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:19:03.498 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2024-03-29T04:19:03.498 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.625563+0000', 'last_change': '2024-03-29T04:18:57.625563+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.625563+0000', 'last_undegraded': '2024-03-29T04:18:57.625563+0000', 'last_fullsized': '2024-03-29T04:18:57.625563+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-29T04:19:03.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2024-03-29T04:19:03.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.625314+0000', 'last_change': '2024-03-29T04:18:57.625314+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.625314+0000', 'last_undegraded': '2024-03-29T04:18:57.625314+0000', 'last_fullsized': '2024-03-29T04:18:57.625314+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-29T04:19:03.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2024-03-29T04:19:03.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.622173+0000', 'last_change': '2024-03-29T04:18:57.622173+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.622173+0000', 'last_undegraded': '2024-03-29T04:18:57.622173+0000', 'last_fullsized': '2024-03-29T04:18:57.622173+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-29T04:19:03.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2024-03-29T04:19:03.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.628289+0000', 'last_change': '2024-03-29T04:18:57.628289+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.628289+0000', 'last_undegraded': '2024-03-29T04:18:57.628289+0000', 'last_fullsized': '2024-03-29T04:18:57.628289+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-29T04:19:03.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2024-03-29T04:19:03.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.627884+0000', 'last_change': '2024-03-29T04:18:57.627884+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.627884+0000', 'last_undegraded': '2024-03-29T04:18:57.627884+0000', 'last_fullsized': '2024-03-29T04:18:57.627884+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-29T04:19:03.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2024-03-29T04:19:03.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.629072+0000', 'last_change': '2024-03-29T04:18:57.629072+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.629072+0000', 'last_undegraded': '2024-03-29T04:18:57.629072+0000', 'last_fullsized': '2024-03-29T04:18:57.629072+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-29T04:19:03.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2024-03-29T04:19:03.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.638167+0000', 'last_change': '2024-03-29T04:18:57.638167+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.638167+0000', 'last_undegraded': '2024-03-29T04:18:57.638167+0000', 'last_fullsized': '2024-03-29T04:18:57.638167+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:19:03.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2024-03-29T04:19:03.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.624752+0000', 'last_change': '2024-03-29T04:18:57.624752+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.624752+0000', 'last_undegraded': '2024-03-29T04:18:57.624752+0000', 'last_fullsized': '2024-03-29T04:18:57.624752+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-29T04:19:03.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2024-03-29T04:19:03.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.621806+0000', 'last_change': '2024-03-29T04:18:57.621806+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.621806+0000', 'last_undegraded': '2024-03-29T04:18:57.621806+0000', 'last_fullsized': '2024-03-29T04:18:57.621806+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-29T04:19:03.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2024-03-29T04:19:03.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.635701+0000', 'last_change': '2024-03-29T04:18:57.635701+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.635701+0000', 'last_undegraded': '2024-03-29T04:18:57.635701+0000', 'last_fullsized': '2024-03-29T04:18:57.635701+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:19:03.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2024-03-29T04:19:03.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.632529+0000', 'last_change': '2024-03-29T04:18:57.632529+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.632529+0000', 'last_undegraded': '2024-03-29T04:18:57.632529+0000', 'last_fullsized': '2024-03-29T04:18:57.632529+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:19:03.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2024-03-29T04:19:03.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.621716+0000', 'last_change': '2024-03-29T04:18:57.621716+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.621716+0000', 'last_undegraded': '2024-03-29T04:18:57.621716+0000', 'last_fullsized': '2024-03-29T04:18:57.621716+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-29T04:19:03.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2024-03-29T04:19:03.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.636959+0000', 'last_change': '2024-03-29T04:18:57.636959+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.636959+0000', 'last_undegraded': '2024-03-29T04:18:57.636959+0000', 'last_fullsized': '2024-03-29T04:18:57.636959+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:19:03.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2024-03-29T04:19:03.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.651723+0000', 'last_change': '2024-03-29T04:18:57.651723+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.651723+0000', 'last_undegraded': '2024-03-29T04:18:57.651723+0000', 'last_fullsized': '2024-03-29T04:18:57.651723+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T04:19:03.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2024-03-29T04:19:03.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.636989+0000', 'last_change': '2024-03-29T04:18:57.636989+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.636989+0000', 'last_undegraded': '2024-03-29T04:18:57.636989+0000', 'last_fullsized': '2024-03-29T04:18:57.636989+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:19:03.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2024-03-29T04:19:03.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.638252+0000', 'last_change': '2024-03-29T04:18:57.638252+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.638252+0000', 'last_undegraded': '2024-03-29T04:18:57.638252+0000', 'last_fullsized': '2024-03-29T04:18:57.638252+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T04:19:03.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2024-03-29T04:19:03.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.632094+0000', 'last_change': '2024-03-29T04:18:57.632094+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.632094+0000', 'last_undegraded': '2024-03-29T04:18:57.632094+0000', 'last_fullsized': '2024-03-29T04:18:57.632094+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T04:19:03.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2024-03-29T04:19:03.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.623561+0000', 'last_change': '2024-03-29T04:18:57.623561+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.623561+0000', 'last_undegraded': '2024-03-29T04:18:57.623561+0000', 'last_fullsized': '2024-03-29T04:18:57.623561+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-29T04:19:03.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2024-03-29T04:19:03.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.627332+0000', 'last_change': '2024-03-29T04:18:57.627332+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.627332+0000', 'last_undegraded': '2024-03-29T04:18:57.627332+0000', 'last_fullsized': '2024-03-29T04:18:57.627332+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-29T04:19:03.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2024-03-29T04:19:03.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.622588+0000', 'last_change': '2024-03-29T04:18:57.622588+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.622588+0000', 'last_undegraded': '2024-03-29T04:18:57.622588+0000', 'last_fullsized': '2024-03-29T04:18:57.622588+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-29T04:19:03.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2024-03-29T04:19:03.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.635894+0000', 'last_change': '2024-03-29T04:18:57.635894+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.635894+0000', 'last_undegraded': '2024-03-29T04:18:57.635894+0000', 'last_fullsized': '2024-03-29T04:18:57.635894+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:19:03.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2024-03-29T04:19:03.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.643079+0000', 'last_change': '2024-03-29T04:18:57.643079+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.643079+0000', 'last_undegraded': '2024-03-29T04:18:57.643079+0000', 'last_fullsized': '2024-03-29T04:18:57.643079+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T04:19:03.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2024-03-29T04:19:03.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.635697+0000', 'last_change': '2024-03-29T04:18:57.635697+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.635697+0000', 'last_undegraded': '2024-03-29T04:18:57.635697+0000', 'last_fullsized': '2024-03-29T04:18:57.635697+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T04:19:03.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2024-03-29T04:19:03.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.627435+0000', 'last_change': '2024-03-29T04:18:57.627435+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.627435+0000', 'last_undegraded': '2024-03-29T04:18:57.627435+0000', 'last_fullsized': '2024-03-29T04:18:57.627435+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-29T04:19:03.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2024-03-29T04:19:03.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.624839+0000', 'last_change': '2024-03-29T04:18:57.624839+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.624839+0000', 'last_undegraded': '2024-03-29T04:18:57.624839+0000', 'last_fullsized': '2024-03-29T04:18:57.624839+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-29T04:19:03.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2024-03-29T04:19:03.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.627398+0000', 'last_change': '2024-03-29T04:18:57.627398+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.627398+0000', 'last_undegraded': '2024-03-29T04:18:57.627398+0000', 'last_fullsized': '2024-03-29T04:18:57.627398+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-29T04:19:03.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2024-03-29T04:19:03.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.623666+0000', 'last_change': '2024-03-29T04:18:57.623666+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.623666+0000', 'last_undegraded': '2024-03-29T04:18:57.623666+0000', 'last_fullsized': '2024-03-29T04:18:57.623666+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-29T04:19:03.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2024-03-29T04:19:03.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.630493+0000', 'last_change': '2024-03-29T04:18:57.630493+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.630493+0000', 'last_undegraded': '2024-03-29T04:18:57.630493+0000', 'last_fullsized': '2024-03-29T04:18:57.630493+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:19:03.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2024-03-29T04:19:03.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.625750+0000', 'last_change': '2024-03-29T04:18:57.625750+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.625750+0000', 'last_undegraded': '2024-03-29T04:18:57.625750+0000', 'last_fullsized': '2024-03-29T04:18:57.625750+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-29T04:19:03.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2024-03-29T04:19:03.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.622621+0000', 'last_change': '2024-03-29T04:18:57.622621+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.622621+0000', 'last_undegraded': '2024-03-29T04:18:57.622621+0000', 'last_fullsized': '2024-03-29T04:18:57.622621+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-29T04:19:03.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2024-03-29T04:19:03.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.631780+0000', 'last_change': '2024-03-29T04:18:57.631780+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.631780+0000', 'last_undegraded': '2024-03-29T04:18:57.631780+0000', 'last_fullsized': '2024-03-29T04:18:57.631780+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T04:19:03.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2024-03-29T04:19:03.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.627065+0000', 'last_change': '2024-03-29T04:18:57.627065+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.627065+0000', 'last_undegraded': '2024-03-29T04:18:57.627065+0000', 'last_fullsized': '2024-03-29T04:18:57.627065+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-29T04:19:03.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-03-29T04:19:03.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.639988+0000', 'last_change': '2024-03-29T04:18:57.639988+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.639988+0000', 'last_undegraded': '2024-03-29T04:18:57.639988+0000', 'last_fullsized': '2024-03-29T04:18:57.639988+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T04:19:03.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-03-29T04:19:03.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.630597+0000', 'last_change': '2024-03-29T04:18:57.630597+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.630597+0000', 'last_undegraded': '2024-03-29T04:18:57.630597+0000', 'last_fullsized': '2024-03-29T04:18:57.630597+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-29T04:19:03.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-03-29T04:19:03.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.637914+0000', 'last_change': '2024-03-29T04:18:57.637914+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.637914+0000', 'last_undegraded': '2024-03-29T04:18:57.637914+0000', 'last_fullsized': '2024-03-29T04:18:57.637914+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T04:19:03.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-03-29T04:19:03.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.624736+0000', 'last_change': '2024-03-29T04:18:57.624736+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.624736+0000', 'last_undegraded': '2024-03-29T04:18:57.624736+0000', 'last_fullsized': '2024-03-29T04:18:57.624736+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-29T04:19:03.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2024-03-29T04:19:03.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.623072+0000', 'last_change': '2024-03-29T04:18:57.623072+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.623072+0000', 'last_undegraded': '2024-03-29T04:18:57.623072+0000', 'last_fullsized': '2024-03-29T04:18:57.623072+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-29T04:19:03.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-03-29T04:19:03.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.632972+0000', 'last_change': '2024-03-29T04:18:57.632972+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.632972+0000', 'last_undegraded': '2024-03-29T04:18:57.632972+0000', 'last_fullsized': '2024-03-29T04:18:57.632972+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:19:03.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-03-29T04:19:03.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.641871+0000', 'last_change': '2024-03-29T04:18:57.641871+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.641871+0000', 'last_undegraded': '2024-03-29T04:18:57.641871+0000', 'last_fullsized': '2024-03-29T04:18:57.641871+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T04:19:03.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-03-29T04:19:03.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.626138+0000', 'last_change': '2024-03-29T04:18:57.626138+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.626138+0000', 'last_undegraded': '2024-03-29T04:18:57.626138+0000', 'last_fullsized': '2024-03-29T04:18:57.626138+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-29T04:19:03.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-03-29T04:19:03.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.627438+0000', 'last_change': '2024-03-29T04:18:57.627438+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.627438+0000', 'last_undegraded': '2024-03-29T04:18:57.627438+0000', 'last_fullsized': '2024-03-29T04:18:57.627438+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-29T04:19:03.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-03-29T04:19:03.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.634764+0000', 'last_change': '2024-03-29T04:18:57.634764+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.634764+0000', 'last_undegraded': '2024-03-29T04:18:57.634764+0000', 'last_fullsized': '2024-03-29T04:18:57.634764+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T04:19:03.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2024-03-29T04:19:03.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.621183+0000', 'last_change': '2024-03-29T04:18:57.621183+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.621183+0000', 'last_undegraded': '2024-03-29T04:18:57.621183+0000', 'last_fullsized': '2024-03-29T04:18:57.621183+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-29T04:19:03.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-03-29T04:19:03.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.634774+0000', 'last_change': '2024-03-29T04:18:57.634774+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.634774+0000', 'last_undegraded': '2024-03-29T04:18:57.634774+0000', 'last_fullsized': '2024-03-29T04:18:57.634774+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T04:19:03.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2024-03-29T04:19:03.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.622013+0000', 'last_change': '2024-03-29T04:18:57.622013+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.622013+0000', 'last_undegraded': '2024-03-29T04:18:57.622013+0000', 'last_fullsized': '2024-03-29T04:18:57.622013+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-29T04:19:03.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-03-29T04:19:03.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.624837+0000', 'last_change': '2024-03-29T04:18:57.624837+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.624837+0000', 'last_undegraded': '2024-03-29T04:18:57.624837+0000', 'last_fullsized': '2024-03-29T04:18:57.624837+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-29T04:19:03.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-03-29T04:19:03.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.630791+0000', 'last_change': '2024-03-29T04:18:57.630791+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.630791+0000', 'last_undegraded': '2024-03-29T04:18:57.630791+0000', 'last_fullsized': '2024-03-29T04:18:57.630791+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T04:19:03.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-03-29T04:19:03.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.625716+0000', 'last_change': '2024-03-29T04:18:57.625716+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.625716+0000', 'last_undegraded': '2024-03-29T04:18:57.625716+0000', 'last_fullsized': '2024-03-29T04:18:57.625716+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-29T04:19:03.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-03-29T04:19:03.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.624394+0000', 'last_change': '2024-03-29T04:18:57.624394+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.624394+0000', 'last_undegraded': '2024-03-29T04:18:57.624394+0000', 'last_fullsized': '2024-03-29T04:18:57.624394+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-29T04:19:03.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-03-29T04:19:03.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.631448+0000', 'last_change': '2024-03-29T04:18:57.631448+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.631448+0000', 'last_undegraded': '2024-03-29T04:18:57.631448+0000', 'last_fullsized': '2024-03-29T04:18:57.631448+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:19:03.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-03-29T04:19:03.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.639038+0000', 'last_change': '2024-03-29T04:18:57.639038+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.639038+0000', 'last_undegraded': '2024-03-29T04:18:57.639038+0000', 'last_fullsized': '2024-03-29T04:18:57.639038+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T04:19:03.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-03-29T04:19:03.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.627310+0000', 'last_change': '2024-03-29T04:18:57.627310+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.627310+0000', 'last_undegraded': '2024-03-29T04:18:57.627310+0000', 'last_fullsized': '2024-03-29T04:18:57.627310+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-29T04:19:03.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-03-29T04:19:03.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.632889+0000', 'last_change': '2024-03-29T04:18:57.632889+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.632889+0000', 'last_undegraded': '2024-03-29T04:18:57.632889+0000', 'last_fullsized': '2024-03-29T04:18:57.632889+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T04:19:03.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2024-03-29T04:19:03.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.622136+0000', 'last_change': '2024-03-29T04:18:57.622136+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.622136+0000', 'last_undegraded': '2024-03-29T04:18:57.622136+0000', 'last_fullsized': '2024-03-29T04:18:57.622136+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-29T04:19:03.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-03-29T04:19:03.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.649106+0000', 'last_change': '2024-03-29T04:18:57.649106+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.649106+0000', 'last_undegraded': '2024-03-29T04:18:57.649106+0000', 'last_fullsized': '2024-03-29T04:18:57.649106+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:19:03.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2024-03-29T04:19:03.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.625166+0000', 'last_change': '2024-03-29T04:18:57.625166+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.625166+0000', 'last_undegraded': '2024-03-29T04:18:57.625166+0000', 'last_fullsized': '2024-03-29T04:18:57.625166+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-29T04:19:03.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-03-29T04:19:03.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.631700+0000', 'last_change': '2024-03-29T04:18:57.631700+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.631700+0000', 'last_undegraded': '2024-03-29T04:18:57.631700+0000', 'last_fullsized': '2024-03-29T04:18:57.631700+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-29T04:19:03.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-03-29T04:19:03.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.633061+0000', 'last_change': '2024-03-29T04:18:57.633061+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.633061+0000', 'last_undegraded': '2024-03-29T04:18:57.633061+0000', 'last_fullsized': '2024-03-29T04:18:57.633061+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T04:19:03.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-03-29T04:19:03.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.624796+0000', 'last_change': '2024-03-29T04:18:57.624796+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.624796+0000', 'last_undegraded': '2024-03-29T04:18:57.624796+0000', 'last_fullsized': '2024-03-29T04:18:57.624796+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-29T04:19:03.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-03-29T04:19:03.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.621515+0000', 'last_change': '2024-03-29T04:18:57.621515+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.621515+0000', 'last_undegraded': '2024-03-29T04:18:57.621515+0000', 'last_fullsized': '2024-03-29T04:18:57.621515+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-29T04:19:03.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-03-29T04:19:03.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.628402+0000', 'last_change': '2024-03-29T04:18:57.628402+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.628402+0000', 'last_undegraded': '2024-03-29T04:18:57.628402+0000', 'last_fullsized': '2024-03-29T04:18:57.628402+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-29T04:19:03.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-03-29T04:19:03.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.644151+0000', 'last_change': '2024-03-29T04:18:57.644151+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.644151+0000', 'last_undegraded': '2024-03-29T04:18:57.644151+0000', 'last_fullsized': '2024-03-29T04:18:57.644151+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T04:19:03.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2024-03-29T04:19:03.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.630826+0000', 'last_change': '2024-03-29T04:18:57.630826+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.630826+0000', 'last_undegraded': '2024-03-29T04:18:57.630826+0000', 'last_fullsized': '2024-03-29T04:18:57.630826+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T04:19:03.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-03-29T04:19:03.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:18:57.647972+0000', 'last_change': '2024-03-29T04:18:57.647972+0000', 'last_active': '2024-03-29T04:18:57.609853+0000', 'last_peered': '2024-03-29T04:18:57.609853+0000', 'last_clean': '2024-03-29T04:18:57.609853+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:18:57.647972+0000', 'last_undegraded': '2024-03-29T04:18:57.647972+0000', 'last_fullsized': '2024-03-29T04:18:57.647972+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'last_clean_scrub_stamp': '2024-03-29T04:18:57.609853+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:19:03.510 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-29T04:19:03.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.679+0000 7f3b79307640 1 Processor -- start 2024-03-29T04:19:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.679+0000 7f3b79307640 1 -- start start 2024-03-29T04:19:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.679+0000 7f3b79307640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3b740cd140 0x7f3b740cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.679+0000 7f3b79307640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b740cfb60 con 0x7f3b74142440 2024-03-29T04:19:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.679+0000 7f3b79307640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b740cfe40 con 0x7f3b7406bc20 2024-03-29T04:19:03.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.679+0000 7f3b79307640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f3b740d0120 con 0x7f3b740cd140 2024-03-29T04:19:03.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.679+0000 7f3b6bfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3b740cd140 0x7f3b740cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:03.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.679+0000 7f3b72ffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f3b74142440 0x7f3b740cf0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:52240/0 (socket says 172.21.15.22:52240) 2024-03-29T04:19:03.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.679+0000 7f3b72ffd640 1 -- 172.21.15.22:0/3822465790 learned_addr learned my addr 172.21.15.22:0/3822465790 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:19:03.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.683+0000 7f3b6bfff640 1 -- 172.21.15.22:0/3822465790 >> v1:172.21.15.151:6789/0 conn(0x7f3b7406bc20 legacy=0x7f3b7406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:03.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.683+0000 7f3b6bfff640 1 -- 172.21.15.22:0/3822465790 >> v1:172.21.15.22:6789/0 conn(0x7f3b74142440 legacy=0x7f3b740cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:03.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.683+0000 7f3b727fc640 1 -- 172.21.15.22:0/3822465790 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 838241185 0 0) 0x7f3b740cfb60 con 0x7f3b74142440 2024-03-29T04:19:03.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.683+0000 7f3b6bfff640 1 -- 172.21.15.22:0/3822465790 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b740d0400 con 0x7f3b740cd140 2024-03-29T04:19:03.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.683+0000 7f3b6bfff640 1 --2- 172.21.15.22:0/3822465790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b740cd140 0x7f3b740cd5d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c0097c0 tx=0x7f3b5c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32dc724f826e8cd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:03.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.683+0000 7f3b727fc640 1 -- 172.21.15.22:0/3822465790 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c0b0070 con 0x7f3b740cd140 2024-03-29T04:19:03.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.683+0000 7f3b727fc640 1 -- 172.21.15.22:0/3822465790 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b5c0ab600 con 0x7f3b740cd140 2024-03-29T04:19:03.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.683+0000 7f3b727fc640 1 -- 172.21.15.22:0/3822465790 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c0ab9d0 con 0x7f3b740cd140 2024-03-29T04:19:03.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b79307640 1 -- 172.21.15.22:0/3822465790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b740cd140 msgr2=0x7f3b740cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:03.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b79307640 1 --2- 172.21.15.22:0/3822465790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b740cd140 0x7f3b740cd5d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c0097c0 tx=0x7f3b5c01c4a0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b79307640 1 -- 172.21.15.22:0/3822465790 shutdown_connections 2024-03-29T04:19:03.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b79307640 1 --2- 172.21.15.22:0/3822465790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b740cd140 0x7f3b740cd5d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b79307640 1 -- 172.21.15.22:0/3822465790 >> 172.21.15.22:0/3822465790 conn(0x7f3b740b2b30 msgr2=0x7f3b740d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:03.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b79307640 1 -- 172.21.15.22:0/3822465790 shutdown_connections 2024-03-29T04:19:03.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b79307640 1 -- 172.21.15.22:0/3822465790 wait complete. 2024-03-29T04:19:03.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b79307640 1 Processor -- start 2024-03-29T04:19:03.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b79307640 1 -- start start 2024-03-29T04:19:03.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b79307640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b7406bc20 0x7f3b740a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:03.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b79307640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b740cd140 0x7f3b7413ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:03.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b79307640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b74142440 0x7f3b740b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:03.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b79307640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3b740ccbe0 con 0x7f3b7406bc20 2024-03-29T04:19:03.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b79307640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3b740cca60 con 0x7f3b740cd140 2024-03-29T04:19:03.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b79307640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f3b7416a150 con 0x7f3b74142440 2024-03-29T04:19:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b72ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b7406bc20 0x7f3b740a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b72ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b7406bc20 0x7f3b740a94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44414/0 (socket says 172.21.15.22:44414) 2024-03-29T04:19:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b72ffd640 1 -- 172.21.15.22:0/1214673133 learned_addr learned my addr 172.21.15.22:0/1214673133 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b6bfff640 1 --2- 172.21.15.22:0/1214673133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b740cd140 0x7f3b7413ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b737fe640 1 --2- 172.21.15.22:0/1214673133 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b74142440 0x7f3b740b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b72ffd640 1 -- 172.21.15.22:0/1214673133 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b74142440 msgr2=0x7f3b740b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b72ffd640 1 --2- 172.21.15.22:0/1214673133 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b74142440 0x7f3b740b50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b72ffd640 1 -- 172.21.15.22:0/1214673133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b740cd140 msgr2=0x7f3b7413ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b72ffd640 1 --2- 172.21.15.22:0/1214673133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b740cd140 0x7f3b7413ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b72ffd640 1 -- 172.21.15.22:0/1214673133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b74159770 con 0x7f3b7406bc20 2024-03-29T04:19:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b72ffd640 1 --2- 172.21.15.22:0/1214673133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b7406bc20 0x7f3b740a94d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400c410 tx=0x7f3b6400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b6b7fe640 1 -- 172.21.15.22:0/1214673133 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b640074e0 con 0x7f3b7406bc20 2024-03-29T04:19:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b6b7fe640 1 -- 172.21.15.22:0/1214673133 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b640047f0 con 0x7f3b7406bc20 2024-03-29T04:19:03.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b79307640 1 -- 172.21.15.22:0/1214673133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b7413f260 con 0x7f3b7406bc20 2024-03-29T04:19:03.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b6b7fe640 1 -- 172.21.15.22:0/1214673133 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64007680 con 0x7f3b7406bc20 2024-03-29T04:19:03.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.687+0000 7f3b79307640 1 -- 172.21.15.22:0/1214673133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b740a9e40 con 0x7f3b7406bc20 2024-03-29T04:19:03.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.691+0000 7f3b79307640 1 -- 172.21.15.22:0/1214673133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b740aa080 con 0x7f3b7406bc20 2024-03-29T04:19:03.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.691+0000 7f3b6b7fe640 1 -- 172.21.15.22:0/1214673133 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f3b64011070 con 0x7f3b7406bc20 2024-03-29T04:19:03.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.691+0000 7f3b6b7fe640 1 --2- 172.21.15.22:0/1214673133 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b54067fc0 0x7f3b5406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:03.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.691+0000 7f3b6b7fe640 1 -- 172.21.15.22:0/1214673133 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f3b640868f0 con 0x7f3b7406bc20 2024-03-29T04:19:03.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.691+0000 7f3b6bfff640 1 --2- 172.21.15.22:0/1214673133 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b54067fc0 0x7f3b5406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:03.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.691+0000 7f3b6bfff640 1 --2- 172.21.15.22:0/1214673133 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b54067fc0 0x7f3b5406a480 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c0ad000 tx=0x7f3b5c0a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:03.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.695+0000 7f3b6b7fe640 1 -- 172.21.15.22:0/1214673133 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f3b740aa080 con 0x7f3b7406bc20 2024-03-29T04:19:03.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.915+0000 7f3b79307640 1 -- 172.21.15.22:0/1214673133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f3b74068890 con 0x7f3b7406bc20 2024-03-29T04:19:03.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.915+0000 7f3b6b7fe640 1 -- 172.21.15.22:0/1214673133 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1034 (secure 0 0 0) 0x7f3b64056f60 con 0x7f3b7406bc20 2024-03-29T04:19:03.896 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:19:03.896 INFO:teuthology.orchestra.run.smithi022.stdout:{"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":113,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":40,"num_osds":8,"num_up_osds":8,"osd_up_since":1711685842,"num_in_osds":8,"osd_in_since":1711685833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":247283712,"bytes_avail":772846829568,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":18,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5529}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-03-29T04:18:39.715347+0000","services":{}},"progress_events":{}} 2024-03-29T04:19:03.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.915+0000 7f3b79307640 1 -- 172.21.15.22:0/1214673133 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b54067fc0 msgr2=0x7f3b5406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:03.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.915+0000 7f3b79307640 1 --2- 172.21.15.22:0/1214673133 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b54067fc0 0x7f3b5406a480 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c0ad000 tx=0x7f3b5c0a6000 comp rx=0 tx=0).stop 2024-03-29T04:19:03.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.915+0000 7f3b79307640 1 -- 172.21.15.22:0/1214673133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b7406bc20 msgr2=0x7f3b740a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:03.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.915+0000 7f3b79307640 1 --2- 172.21.15.22:0/1214673133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b7406bc20 0x7f3b740a94d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400c410 tx=0x7f3b6400c8d0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.915+0000 7f3b79307640 1 -- 172.21.15.22:0/1214673133 shutdown_connections 2024-03-29T04:19:03.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.915+0000 7f3b79307640 1 --2- 172.21.15.22:0/1214673133 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b74142440 0x7f3b740b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.915+0000 7f3b79307640 1 --2- 172.21.15.22:0/1214673133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b740cd140 0x7f3b7413ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.915+0000 7f3b79307640 1 --2- 172.21.15.22:0/1214673133 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b54067fc0 0x7f3b5406a480 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.915+0000 7f3b79307640 1 --2- 172.21.15.22:0/1214673133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b7406bc20 0x7f3b740a94d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:03.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.915+0000 7f3b79307640 1 -- 172.21.15.22:0/1214673133 >> 172.21.15.22:0/1214673133 conn(0x7f3b740b2b30 msgr2=0x7f3b7415ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:03.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.915+0000 7f3b79307640 1 -- 172.21.15.22:0/1214673133 shutdown_connections 2024-03-29T04:19:03.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:03.915+0000 7f3b79307640 1 -- 172.21.15.22:0/1214673133 wait complete. 2024-03-29T04:19:03.915 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-29T04:19:03.915 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:19:04.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.047+0000 7fefaf4eb640 1 Processor -- start 2024-03-29T04:19:04.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.047+0000 7fefaf4eb640 1 -- start start 2024-03-29T04:19:04.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.047+0000 7fefaf4eb640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fefa815bf50 0x7fefa815e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:04.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.047+0000 7fefaf4eb640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fefa80b42d0 0x7fefa80b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:04.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.047+0000 7fefaf4eb640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fefa8161f70 0x7fefa80b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:04.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.047+0000 7fefaf4eb640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fefa806b720 con 0x7fefa8161f70 2024-03-29T04:19:04.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.047+0000 7fefaf4eb640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fefa815c330 con 0x7fefa815bf50 2024-03-29T04:19:04.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.047+0000 7fefaf4eb640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fefa81624e0 con 0x7fefa80b42d0 2024-03-29T04:19:04.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.047+0000 7fefada88640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fefa8161f70 0x7fefa80b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:04.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.047+0000 7fefada88640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fefa8161f70 0x7fefa80b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44422/0 (socket says 172.21.15.22:44422) 2024-03-29T04:19:04.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.047+0000 7fefada88640 1 -- 172.21.15.22:0/1520476451 learned_addr learned my addr 172.21.15.22:0/1520476451 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:04.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.047+0000 7fefad287640 1 --2- 172.21.15.22:0/1520476451 >> v2:172.21.15.151:3300/0 conn(0x7fefa815bf50 0x7fefa815e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:04.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.047+0000 7fefaca86640 1 --2- 172.21.15.22:0/1520476451 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fefa80b42d0 0x7fefa80b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:04.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.047+0000 7fefada88640 1 -- 172.21.15.22:0/1520476451 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fefa80b42d0 msgr2=0x7fefa80b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:04.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.047+0000 7fefada88640 1 --2- 172.21.15.22:0/1520476451 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fefa80b42d0 0x7fefa80b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:04.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.047+0000 7fefada88640 1 -- 172.21.15.22:0/1520476451 >> v2:172.21.15.151:3300/0 conn(0x7fefa815bf50 msgr2=0x7fefa815e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:04.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.047+0000 7fefada88640 1 --2- 172.21.15.22:0/1520476451 >> v2:172.21.15.151:3300/0 conn(0x7fefa815bf50 0x7fefa815e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:04.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.047+0000 7fefada88640 1 -- 172.21.15.22:0/1520476451 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa81627c0 con 0x7fefa8161f70 2024-03-29T04:19:04.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefada88640 1 --2- 172.21.15.22:0/1520476451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefa8161f70 0x7fefa80b3bc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fefa80b3310 tx=0x7fefa001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb4822a5ecd26c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:04.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fef9bfff640 1 -- 172.21.15.22:0/1520476451 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa00aa430 con 0x7fefa8161f70 2024-03-29T04:19:04.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fef9bfff640 1 -- 172.21.15.22:0/1520476451 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefa00af070 con 0x7fefa8161f70 2024-03-29T04:19:04.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fef9bfff640 1 -- 172.21.15.22:0/1520476451 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa00aa7f0 con 0x7fefa8161f70 2024-03-29T04:19:04.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaf4eb640 1 -- 172.21.15.22:0/1520476451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefa8161f70 msgr2=0x7fefa80b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:04.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaf4eb640 1 --2- 172.21.15.22:0/1520476451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefa8161f70 0x7fefa80b3bc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fefa80b3310 tx=0x7fefa001c4f0 comp rx=0 tx=0).stop 2024-03-29T04:19:04.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaf4eb640 1 -- 172.21.15.22:0/1520476451 shutdown_connections 2024-03-29T04:19:04.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaf4eb640 1 --2- 172.21.15.22:0/1520476451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefa8161f70 0x7fefa80b3bc0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:04.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaf4eb640 1 --2- 172.21.15.22:0/1520476451 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fefa80b42d0 0x7fefa80b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:04.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaf4eb640 1 --2- 172.21.15.22:0/1520476451 >> v2:172.21.15.151:3300/0 conn(0x7fefa815bf50 0x7fefa815e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:04.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaf4eb640 1 -- 172.21.15.22:0/1520476451 >> 172.21.15.22:0/1520476451 conn(0x7fefa80b2980 msgr2=0x7fefa80d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:04.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaf4eb640 1 -- 172.21.15.22:0/1520476451 shutdown_connections 2024-03-29T04:19:04.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaf4eb640 1 -- 172.21.15.22:0/1520476451 wait complete. 2024-03-29T04:19:04.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaf4eb640 1 Processor -- start 2024-03-29T04:19:04.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaf4eb640 1 -- start start 2024-03-29T04:19:04.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaf4eb640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fefa80b42d0 0x7fefa81457b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:04.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaf4eb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefa815bf50 0x7fefa8142100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:04.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaf4eb640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefa8161f70 0x7fefa80adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:04.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaf4eb640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fefa806ed00 con 0x7fefa815bf50 2024-03-29T04:19:04.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaf4eb640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fefa806eb80 con 0x7fefa8161f70 2024-03-29T04:19:04.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaf4eb640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fefa806ee80 con 0x7fefa80b42d0 2024-03-29T04:19:04.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaca86640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefa815bf50 0x7fefa8142100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:04.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaca86640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefa815bf50 0x7fefa8142100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44438/0 (socket says 172.21.15.22:44438) 2024-03-29T04:19:04.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaca86640 1 -- 172.21.15.22:0/721891903 learned_addr learned my addr 172.21.15.22:0/721891903 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:04.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefada88640 1 --2- 172.21.15.22:0/721891903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefa8161f70 0x7fefa80adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:04.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefad287640 1 --2- 172.21.15.22:0/721891903 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fefa80b42d0 0x7fefa81457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:04.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaca86640 1 -- 172.21.15.22:0/721891903 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fefa80b42d0 msgr2=0x7fefa81457b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:04.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaca86640 1 --2- 172.21.15.22:0/721891903 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fefa80b42d0 0x7fefa81457b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:04.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaca86640 1 -- 172.21.15.22:0/721891903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefa8161f70 msgr2=0x7fefa80adbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:04.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaca86640 1 --2- 172.21.15.22:0/721891903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefa8161f70 0x7fefa80adbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:04.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaca86640 1 -- 172.21.15.22:0/721891903 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa8145cf0 con 0x7fefa815bf50 2024-03-29T04:19:04.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefada88640 1 --2- 172.21.15.22:0/721891903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefa8161f70 0x7fefa80adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:19:04.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefad287640 1 --2- 172.21.15.22:0/721891903 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fefa80b42d0 0x7fefa81457b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:19:04.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaca86640 1 --2- 172.21.15.22:0/721891903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefa815bf50 0x7fefa8142100 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fefa400d800 tx=0x7fefa400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:04.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fef99ffb640 1 -- 172.21.15.22:0/721891903 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4018020 con 0x7fefa815bf50 2024-03-29T04:19:04.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fef99ffb640 1 -- 172.21.15.22:0/721891903 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefa40096a0 con 0x7fefa815bf50 2024-03-29T04:19:04.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaf4eb640 1 -- 172.21.15.22:0/721891903 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa8143020 con 0x7fefa815bf50 2024-03-29T04:19:04.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fef99ffb640 1 -- 172.21.15.22:0/721891903 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4009a90 con 0x7fefa815bf50 2024-03-29T04:19:04.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.051+0000 7fefaf4eb640 1 -- 172.21.15.22:0/721891903 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa81435e0 con 0x7fefa815bf50 2024-03-29T04:19:04.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.055+0000 7fefaf4eb640 1 -- 172.21.15.22:0/721891903 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa8068890 con 0x7fefa815bf50 2024-03-29T04:19:04.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.055+0000 7fef99ffb640 1 -- 172.21.15.22:0/721891903 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fefa401b020 con 0x7fefa815bf50 2024-03-29T04:19:04.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.055+0000 7fef99ffb640 1 --2- 172.21.15.22:0/721891903 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fef8c067fc0 0x7fef8c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:04.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.055+0000 7fef99ffb640 1 -- 172.21.15.22:0/721891903 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fefa4086bd0 con 0x7fefa815bf50 2024-03-29T04:19:04.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.055+0000 7fefad287640 1 --2- 172.21.15.22:0/721891903 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fef8c067fc0 0x7fef8c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:04.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.055+0000 7fefad287640 1 --2- 172.21.15.22:0/721891903 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fef8c067fc0 0x7fef8c06a480 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009f50 tx=0x7fef9c00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:04.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.059+0000 7fef99ffb640 1 -- 172.21.15.22:0/721891903 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fefa4022020 con 0x7fefa815bf50 2024-03-29T04:19:04.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.211+0000 7fefaf4eb640 1 -- 172.21.15.22:0/721891903 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fefa80d1340 con 0x7fef8c067fc0 2024-03-29T04:19:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.227+0000 7fef99ffb640 1 -- 172.21.15.22:0/721891903 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251967 (secure 0 0 0) 0x7fef9c058850 con 0x7fef8c067fc0 2024-03-29T04:19:04.212 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:19:04.213 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-29T04:19:04.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.231+0000 7fefaf4eb640 1 -- 172.21.15.22:0/721891903 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fef8c067fc0 msgr2=0x7fef8c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:04.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.231+0000 7fefaf4eb640 1 --2- 172.21.15.22:0/721891903 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fef8c067fc0 0x7fef8c06a480 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009f50 tx=0x7fef9c00b040 comp rx=0 tx=0).stop 2024-03-29T04:19:04.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.231+0000 7fefaf4eb640 1 -- 172.21.15.22:0/721891903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefa815bf50 msgr2=0x7fefa8142100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:04.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.231+0000 7fefaf4eb640 1 --2- 172.21.15.22:0/721891903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefa815bf50 0x7fefa8142100 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fefa400d800 tx=0x7fefa400dcc0 comp rx=0 tx=0).stop 2024-03-29T04:19:04.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.231+0000 7fefaf4eb640 1 -- 172.21.15.22:0/721891903 shutdown_connections 2024-03-29T04:19:04.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.231+0000 7fefaf4eb640 1 --2- 172.21.15.22:0/721891903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefa8161f70 0x7fefa80adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:04.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.231+0000 7fefaf4eb640 1 --2- 172.21.15.22:0/721891903 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fef8c067fc0 0x7fef8c06a480 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:04.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.231+0000 7fefaf4eb640 1 --2- 172.21.15.22:0/721891903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefa815bf50 0x7fefa8142100 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:04.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.231+0000 7fefaf4eb640 1 --2- 172.21.15.22:0/721891903 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fefa80b42d0 0x7fefa81457b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:04.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.231+0000 7fefaf4eb640 1 -- 172.21.15.22:0/721891903 >> 172.21.15.22:0/721891903 conn(0x7fefa80b2980 msgr2=0x7fefa80b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:04.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.231+0000 7fefaf4eb640 1 -- 172.21.15.22:0/721891903 shutdown_connections 2024-03-29T04:19:04.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:04.231+0000 7fefaf4eb640 1 -- 172.21.15.22:0/721891903 wait complete. 2024-03-29T04:19:04.228 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-03-29T04:19:03.722296+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241832,"kb_used_data":4840,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732888,"statfs":{"total":773094113280,"available":772846477312,"internally_reserved":0,"allocated":4956160,"data_stored":2800244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":50,"apply_latency_ms":50,"commit_latency_ns":50000000,"apply_latency_ns":50000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.104306"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679228+0000","last_change":"2024-03-29T04:18:58.667680+0000","last_active":"2024-03-29T04:18:59.679228+0000","last_peered":"2024-03-29T04:18:59.679228+0000","last_clean":"2024-03-29T04:18:59.679228+0000","last_became_active":"2024-03-29T04:18:58.666927+0000","last_became_peered":"2024-03-29T04:18:58.666927+0000","last_unstale":"2024-03-29T04:18:59.679228+0000","last_undegraded":"2024-03-29T04:18:59.679228+0000","last_fullsized":"2024-03-29T04:18:59.679228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:43:42.447228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679463+0000","last_change":"2024-03-29T04:18:58.674238+0000","last_active":"2024-03-29T04:18:59.679463+0000","last_peered":"2024-03-29T04:18:59.679463+0000","last_clean":"2024-03-29T04:18:59.679463+0000","last_became_active":"2024-03-29T04:18:58.673550+0000","last_became_peered":"2024-03-29T04:18:58.673550+0000","last_unstale":"2024-03-29T04:18:59.679463+0000","last_undegraded":"2024-03-29T04:18:59.679463+0000","last_fullsized":"2024-03-29T04:18:59.679463+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:02:00.617566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680064+0000","last_change":"2024-03-29T04:18:58.685574+0000","last_active":"2024-03-29T04:18:59.680064+0000","last_peered":"2024-03-29T04:18:59.680064+0000","last_clean":"2024-03-29T04:18:59.680064+0000","last_became_active":"2024-03-29T04:18:58.685311+0000","last_became_peered":"2024-03-29T04:18:58.685311+0000","last_unstale":"2024-03-29T04:18:59.680064+0000","last_undegraded":"2024-03-29T04:18:59.680064+0000","last_fullsized":"2024-03-29T04:18:59.680064+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:22.612171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685955+0000","last_change":"2024-03-29T04:18:58.679166+0000","last_active":"2024-03-29T04:18:59.685955+0000","last_peered":"2024-03-29T04:18:59.685955+0000","last_clean":"2024-03-29T04:18:59.685955+0000","last_became_active":"2024-03-29T04:18:58.678739+0000","last_became_peered":"2024-03-29T04:18:58.678739+0000","last_unstale":"2024-03-29T04:18:59.685955+0000","last_undegraded":"2024-03-29T04:18:59.685955+0000","last_fullsized":"2024-03-29T04:18:59.685955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:56:07.063576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685428+0000","last_change":"2024-03-29T04:18:58.667545+0000","last_active":"2024-03-29T04:18:59.685428+0000","last_peered":"2024-03-29T04:18:59.685428+0000","last_clean":"2024-03-29T04:18:59.685428+0000","last_became_active":"2024-03-29T04:18:58.666778+0000","last_became_peered":"2024-03-29T04:18:58.666778+0000","last_unstale":"2024-03-29T04:18:59.685428+0000","last_undegraded":"2024-03-29T04:18:59.685428+0000","last_fullsized":"2024-03-29T04:18:59.685428+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:38.275333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:00.276713+0000","last_change":"2024-03-29T04:18:58.675636+0000","last_active":"2024-03-29T04:19:00.276713+0000","last_peered":"2024-03-29T04:19:00.276713+0000","last_clean":"2024-03-29T04:19:00.276713+0000","last_became_active":"2024-03-29T04:18:58.675323+0000","last_became_peered":"2024-03-29T04:18:58.675323+0000","last_unstale":"2024-03-29T04:19:00.276713+0000","last_undegraded":"2024-03-29T04:19:00.276713+0000","last_fullsized":"2024-03-29T04:19:00.276713+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:15:56.537650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.689657+0000","last_change":"2024-03-29T04:18:58.695819+0000","last_active":"2024-03-29T04:18:59.689657+0000","last_peered":"2024-03-29T04:18:59.689657+0000","last_clean":"2024-03-29T04:18:59.689657+0000","last_became_active":"2024-03-29T04:18:58.695069+0000","last_became_peered":"2024-03-29T04:18:58.695069+0000","last_unstale":"2024-03-29T04:18:59.689657+0000","last_undegraded":"2024-03-29T04:18:59.689657+0000","last_fullsized":"2024-03-29T04:18:59.689657+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:08:09.926082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.678898+0000","last_change":"2024-03-29T04:18:58.684531+0000","last_active":"2024-03-29T04:18:59.678898+0000","last_peered":"2024-03-29T04:18:59.678898+0000","last_clean":"2024-03-29T04:18:59.678898+0000","last_became_active":"2024-03-29T04:18:58.684234+0000","last_became_peered":"2024-03-29T04:18:58.684234+0000","last_unstale":"2024-03-29T04:18:59.678898+0000","last_undegraded":"2024-03-29T04:18:59.678898+0000","last_fullsized":"2024-03-29T04:18:59.678898+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:23:01.781748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680131+0000","last_change":"2024-03-29T04:18:58.684993+0000","last_active":"2024-03-29T04:18:59.680131+0000","last_peered":"2024-03-29T04:18:59.680131+0000","last_clean":"2024-03-29T04:18:59.680131+0000","last_became_active":"2024-03-29T04:18:58.684718+0000","last_became_peered":"2024-03-29T04:18:58.684718+0000","last_unstale":"2024-03-29T04:18:59.680131+0000","last_undegraded":"2024-03-29T04:18:59.680131+0000","last_fullsized":"2024-03-29T04:18:59.680131+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:15:56.537650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690369+0000","last_change":"2024-03-29T04:18:58.695211+0000","last_active":"2024-03-29T04:18:59.690369+0000","last_peered":"2024-03-29T04:18:59.690369+0000","last_clean":"2024-03-29T04:18:59.690369+0000","last_became_active":"2024-03-29T04:18:58.694519+0000","last_became_peered":"2024-03-29T04:18:58.694519+0000","last_unstale":"2024-03-29T04:18:59.690369+0000","last_undegraded":"2024-03-29T04:18:59.690369+0000","last_fullsized":"2024-03-29T04:18:59.690369+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:24:51.163863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690459+0000","last_change":"2024-03-29T04:18:58.701560+0000","last_active":"2024-03-29T04:18:59.690459+0000","last_peered":"2024-03-29T04:18:59.690459+0000","last_clean":"2024-03-29T04:18:59.690459+0000","last_became_active":"2024-03-29T04:18:58.699252+0000","last_became_peered":"2024-03-29T04:18:58.699252+0000","last_unstale":"2024-03-29T04:18:59.690459+0000","last_undegraded":"2024-03-29T04:18:59.690459+0000","last_fullsized":"2024-03-29T04:18:59.690459+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:43:42.447228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680479+0000","last_change":"2024-03-29T04:18:58.679122+0000","last_active":"2024-03-29T04:18:59.680479+0000","last_peered":"2024-03-29T04:18:59.680479+0000","last_clean":"2024-03-29T04:18:59.680479+0000","last_became_active":"2024-03-29T04:18:58.678691+0000","last_became_peered":"2024-03-29T04:18:58.678691+0000","last_unstale":"2024-03-29T04:18:59.680479+0000","last_undegraded":"2024-03-29T04:18:59.680479+0000","last_fullsized":"2024-03-29T04:18:59.680479+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:57.410807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.689847+0000","last_change":"2024-03-29T04:18:58.701524+0000","last_active":"2024-03-29T04:18:59.689847+0000","last_peered":"2024-03-29T04:18:59.689847+0000","last_clean":"2024-03-29T04:18:59.689847+0000","last_became_active":"2024-03-29T04:18:58.700452+0000","last_became_peered":"2024-03-29T04:18:58.700452+0000","last_unstale":"2024-03-29T04:18:59.689847+0000","last_undegraded":"2024-03-29T04:18:59.689847+0000","last_fullsized":"2024-03-29T04:18:59.689847+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:27:32.000679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679936+0000","last_change":"2024-03-29T04:18:58.670545+0000","last_active":"2024-03-29T04:18:59.679936+0000","last_peered":"2024-03-29T04:18:59.679936+0000","last_clean":"2024-03-29T04:18:59.679936+0000","last_became_active":"2024-03-29T04:18:58.669825+0000","last_became_peered":"2024-03-29T04:18:58.669825+0000","last_unstale":"2024-03-29T04:18:59.679936+0000","last_undegraded":"2024-03-29T04:18:59.679936+0000","last_fullsized":"2024-03-29T04:18:59.679936+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:39:47.279731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.689686+0000","last_change":"2024-03-29T04:18:58.693326+0000","last_active":"2024-03-29T04:18:59.689686+0000","last_peered":"2024-03-29T04:18:59.689686+0000","last_clean":"2024-03-29T04:18:59.689686+0000","last_became_active":"2024-03-29T04:18:58.682482+0000","last_became_peered":"2024-03-29T04:18:58.682482+0000","last_unstale":"2024-03-29T04:18:59.689686+0000","last_undegraded":"2024-03-29T04:18:59.689686+0000","last_fullsized":"2024-03-29T04:18:59.689686+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:13:22.147476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.188883+0000","last_change":"2024-03-29T04:18:58.712646+0000","last_active":"2024-03-29T04:19:01.188883+0000","last_peered":"2024-03-29T04:19:01.188883+0000","last_clean":"2024-03-29T04:19:01.188883+0000","last_became_active":"2024-03-29T04:18:58.711875+0000","last_became_peered":"2024-03-29T04:18:58.711875+0000","last_unstale":"2024-03-29T04:19:01.188883+0000","last_undegraded":"2024-03-29T04:19:01.188883+0000","last_fullsized":"2024-03-29T04:19:01.188883+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:15:56.537650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.190703+0000","last_change":"2024-03-29T04:18:58.713921+0000","last_active":"2024-03-29T04:19:01.190703+0000","last_peered":"2024-03-29T04:19:01.190703+0000","last_clean":"2024-03-29T04:19:01.190703+0000","last_became_active":"2024-03-29T04:18:58.713600+0000","last_became_peered":"2024-03-29T04:18:58.713600+0000","last_unstale":"2024-03-29T04:19:01.190703+0000","last_undegraded":"2024-03-29T04:19:01.190703+0000","last_fullsized":"2024-03-29T04:19:01.190703+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:09:58.485244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.191428+0000","last_change":"2024-03-29T04:18:58.678265+0000","last_active":"2024-03-29T04:19:01.191428+0000","last_peered":"2024-03-29T04:19:01.191428+0000","last_clean":"2024-03-29T04:19:01.191428+0000","last_became_active":"2024-03-29T04:18:58.677498+0000","last_became_peered":"2024-03-29T04:18:58.677498+0000","last_unstale":"2024-03-29T04:19:01.191428+0000","last_undegraded":"2024-03-29T04:19:01.191428+0000","last_fullsized":"2024-03-29T04:19:01.191428+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:35.314491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685058+0000","last_change":"2024-03-29T04:18:58.677361+0000","last_active":"2024-03-29T04:18:59.685058+0000","last_peered":"2024-03-29T04:18:59.685058+0000","last_clean":"2024-03-29T04:18:59.685058+0000","last_became_active":"2024-03-29T04:18:58.677022+0000","last_became_peered":"2024-03-29T04:18:58.677022+0000","last_unstale":"2024-03-29T04:18:59.685058+0000","last_undegraded":"2024-03-29T04:18:59.685058+0000","last_fullsized":"2024-03-29T04:18:59.685058+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:14:27.390921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.191082+0000","last_change":"2024-03-29T04:18:58.667731+0000","last_active":"2024-03-29T04:19:01.191082+0000","last_peered":"2024-03-29T04:19:01.191082+0000","last_clean":"2024-03-29T04:19:01.191082+0000","last_became_active":"2024-03-29T04:18:58.666974+0000","last_became_peered":"2024-03-29T04:18:58.666974+0000","last_unstale":"2024-03-29T04:19:01.191082+0000","last_undegraded":"2024-03-29T04:19:01.191082+0000","last_fullsized":"2024-03-29T04:19:01.191082+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:07:10.650606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690288+0000","last_change":"2024-03-29T04:18:58.683859+0000","last_active":"2024-03-29T04:18:59.690288+0000","last_peered":"2024-03-29T04:18:59.690288+0000","last_clean":"2024-03-29T04:18:59.690288+0000","last_became_active":"2024-03-29T04:18:58.683080+0000","last_became_peered":"2024-03-29T04:18:58.683080+0000","last_unstale":"2024-03-29T04:18:59.690288+0000","last_undegraded":"2024-03-29T04:18:59.690288+0000","last_fullsized":"2024-03-29T04:18:59.690288+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:23:01.781748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.681281+0000","last_change":"2024-03-29T04:18:58.686729+0000","last_active":"2024-03-29T04:18:59.681281+0000","last_peered":"2024-03-29T04:18:59.681281+0000","last_clean":"2024-03-29T04:18:59.681281+0000","last_became_active":"2024-03-29T04:18:58.684822+0000","last_became_peered":"2024-03-29T04:18:58.684822+0000","last_unstale":"2024-03-29T04:18:59.681281+0000","last_undegraded":"2024-03-29T04:18:59.681281+0000","last_fullsized":"2024-03-29T04:18:59.681281+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:57.410807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.189240+0000","last_change":"2024-03-29T04:18:58.699382+0000","last_active":"2024-03-29T04:19:01.189240+0000","last_peered":"2024-03-29T04:19:01.189240+0000","last_clean":"2024-03-29T04:19:01.189240+0000","last_became_active":"2024-03-29T04:18:58.698814+0000","last_became_peered":"2024-03-29T04:18:58.698814+0000","last_unstale":"2024-03-29T04:19:01.189240+0000","last_undegraded":"2024-03-29T04:19:01.189240+0000","last_fullsized":"2024-03-29T04:19:01.189240+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:29.155520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685354+0000","last_change":"2024-03-29T04:18:58.696560+0000","last_active":"2024-03-29T04:18:59.685354+0000","last_peered":"2024-03-29T04:18:59.685354+0000","last_clean":"2024-03-29T04:18:59.685354+0000","last_became_active":"2024-03-29T04:18:58.696138+0000","last_became_peered":"2024-03-29T04:18:58.696138+0000","last_unstale":"2024-03-29T04:18:59.685354+0000","last_undegraded":"2024-03-29T04:18:59.685354+0000","last_fullsized":"2024-03-29T04:18:59.685354+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:32:55.508364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.191515+0000","last_change":"2024-03-29T04:18:58.662099+0000","last_active":"2024-03-29T04:19:01.191515+0000","last_peered":"2024-03-29T04:19:01.191515+0000","last_clean":"2024-03-29T04:19:01.191515+0000","last_became_active":"2024-03-29T04:18:58.661339+0000","last_became_peered":"2024-03-29T04:18:58.661339+0000","last_unstale":"2024-03-29T04:19:01.191515+0000","last_undegraded":"2024-03-29T04:19:01.191515+0000","last_fullsized":"2024-03-29T04:19:01.191515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:07:37.211595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:00.277442+0000","last_change":"2024-03-29T04:18:58.671142+0000","last_active":"2024-03-29T04:19:00.277442+0000","last_peered":"2024-03-29T04:19:00.277442+0000","last_clean":"2024-03-29T04:19:00.277442+0000","last_became_active":"2024-03-29T04:18:58.670258+0000","last_became_peered":"2024-03-29T04:18:58.670258+0000","last_unstale":"2024-03-29T04:19:00.277442+0000","last_undegraded":"2024-03-29T04:19:00.277442+0000","last_fullsized":"2024-03-29T04:19:00.277442+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:22:11.812410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.191104+0000","last_change":"2024-03-29T04:18:58.688585+0000","last_active":"2024-03-29T04:19:01.191104+0000","last_peered":"2024-03-29T04:19:01.191104+0000","last_clean":"2024-03-29T04:19:01.191104+0000","last_became_active":"2024-03-29T04:18:58.687856+0000","last_became_peered":"2024-03-29T04:18:58.687856+0000","last_unstale":"2024-03-29T04:19:01.191104+0000","last_undegraded":"2024-03-29T04:19:01.191104+0000","last_fullsized":"2024-03-29T04:19:01.191104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:56:46.808829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690669+0000","last_change":"2024-03-29T04:18:58.695512+0000","last_active":"2024-03-29T04:18:59.690669+0000","last_peered":"2024-03-29T04:18:59.690669+0000","last_clean":"2024-03-29T04:18:59.690669+0000","last_became_active":"2024-03-29T04:18:58.694782+0000","last_became_peered":"2024-03-29T04:18:58.694782+0000","last_unstale":"2024-03-29T04:18:59.690669+0000","last_undegraded":"2024-03-29T04:18:59.690669+0000","last_fullsized":"2024-03-29T04:18:59.690669+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:06:56.949031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.190830+0000","last_change":"2024-03-29T04:18:58.677751+0000","last_active":"2024-03-29T04:19:01.190830+0000","last_peered":"2024-03-29T04:19:01.190830+0000","last_clean":"2024-03-29T04:19:01.190830+0000","last_became_active":"2024-03-29T04:18:58.676895+0000","last_became_peered":"2024-03-29T04:18:58.676895+0000","last_unstale":"2024-03-29T04:19:01.190830+0000","last_undegraded":"2024-03-29T04:19:01.190830+0000","last_fullsized":"2024-03-29T04:19:01.190830+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:58:16.237991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.190189+0000","last_change":"2024-03-29T04:18:58.676978+0000","last_active":"2024-03-29T04:19:01.190189+0000","last_peered":"2024-03-29T04:19:01.190189+0000","last_clean":"2024-03-29T04:19:01.190189+0000","last_became_active":"2024-03-29T04:18:58.676274+0000","last_became_peered":"2024-03-29T04:18:58.676274+0000","last_unstale":"2024-03-29T04:19:01.190189+0000","last_undegraded":"2024-03-29T04:19:01.190189+0000","last_fullsized":"2024-03-29T04:19:01.190189+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:46:57.038475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.189670+0000","last_change":"2024-03-29T04:18:58.695203+0000","last_active":"2024-03-29T04:19:01.189670+0000","last_peered":"2024-03-29T04:19:01.189670+0000","last_clean":"2024-03-29T04:19:01.189670+0000","last_became_active":"2024-03-29T04:18:58.687916+0000","last_became_peered":"2024-03-29T04:18:58.687916+0000","last_unstale":"2024-03-29T04:19:01.189670+0000","last_undegraded":"2024-03-29T04:19:01.189670+0000","last_fullsized":"2024-03-29T04:19:01.189670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:22:11.812410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:00.276489+0000","last_change":"2024-03-29T04:18:58.666952+0000","last_active":"2024-03-29T04:19:00.276489+0000","last_peered":"2024-03-29T04:19:00.276489+0000","last_clean":"2024-03-29T04:19:00.276489+0000","last_became_active":"2024-03-29T04:18:58.666176+0000","last_became_peered":"2024-03-29T04:18:58.666176+0000","last_unstale":"2024-03-29T04:19:00.276489+0000","last_undegraded":"2024-03-29T04:19:00.276489+0000","last_fullsized":"2024-03-29T04:19:00.276489+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:39:47.279731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.189769+0000","last_change":"2024-03-29T04:18:58.706576+0000","last_active":"2024-03-29T04:19:01.189769+0000","last_peered":"2024-03-29T04:19:01.189769+0000","last_clean":"2024-03-29T04:19:01.189769+0000","last_became_active":"2024-03-29T04:18:58.689768+0000","last_became_peered":"2024-03-29T04:18:58.689768+0000","last_unstale":"2024-03-29T04:19:01.189769+0000","last_undegraded":"2024-03-29T04:19:01.189769+0000","last_fullsized":"2024-03-29T04:19:01.189769+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:57.410807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.191573+0000","last_change":"2024-03-29T04:18:57.652890+0000","last_active":"2024-03-29T04:19:01.191573+0000","last_peered":"2024-03-29T04:19:01.191573+0000","last_clean":"2024-03-29T04:19:01.191573+0000","last_became_active":"2024-03-29T04:18:57.652244+0000","last_became_peered":"2024-03-29T04:18:57.652244+0000","last_unstale":"2024-03-29T04:19:01.191573+0000","last_undegraded":"2024-03-29T04:19:01.191573+0000","last_fullsized":"2024-03-29T04:19:01.191573+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:37.267037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:00.277370+0000","last_change":"2024-03-29T04:18:58.708190+0000","last_active":"2024-03-29T04:19:00.277370+0000","last_peered":"2024-03-29T04:19:00.277370+0000","last_clean":"2024-03-29T04:19:00.277370+0000","last_became_active":"2024-03-29T04:18:58.707780+0000","last_became_peered":"2024-03-29T04:18:58.707780+0000","last_unstale":"2024-03-29T04:19:00.277370+0000","last_undegraded":"2024-03-29T04:19:00.277370+0000","last_fullsized":"2024-03-29T04:19:00.277370+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:22.612171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680302+0000","last_change":"2024-03-29T04:18:57.646783+0000","last_active":"2024-03-29T04:18:59.680302+0000","last_peered":"2024-03-29T04:18:59.680302+0000","last_clean":"2024-03-29T04:18:59.680302+0000","last_became_active":"2024-03-29T04:18:57.646495+0000","last_became_peered":"2024-03-29T04:18:57.646495+0000","last_unstale":"2024-03-29T04:18:59.680302+0000","last_undegraded":"2024-03-29T04:18:59.680302+0000","last_fullsized":"2024-03-29T04:18:59.680302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:32.935677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680690+0000","last_change":"2024-03-29T04:18:58.668119+0000","last_active":"2024-03-29T04:18:59.680690+0000","last_peered":"2024-03-29T04:18:59.680690+0000","last_clean":"2024-03-29T04:18:59.680690+0000","last_became_active":"2024-03-29T04:18:58.667409+0000","last_became_peered":"2024-03-29T04:18:58.667409+0000","last_unstale":"2024-03-29T04:18:59.680690+0000","last_undegraded":"2024-03-29T04:18:59.680690+0000","last_fullsized":"2024-03-29T04:18:59.680690+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:22.612171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:00.277524+0000","last_change":"2024-03-29T04:18:57.654191+0000","last_active":"2024-03-29T04:19:00.277524+0000","last_peered":"2024-03-29T04:19:00.277524+0000","last_clean":"2024-03-29T04:19:00.277524+0000","last_became_active":"2024-03-29T04:18:57.653911+0000","last_became_peered":"2024-03-29T04:18:57.653911+0000","last_unstale":"2024-03-29T04:19:00.277524+0000","last_undegraded":"2024-03-29T04:19:00.277524+0000","last_fullsized":"2024-03-29T04:19:00.277524+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:16:07.746204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679196+0000","last_change":"2024-03-29T04:18:58.671727+0000","last_active":"2024-03-29T04:18:59.679196+0000","last_peered":"2024-03-29T04:18:59.679196+0000","last_clean":"2024-03-29T04:18:59.679196+0000","last_became_active":"2024-03-29T04:18:58.671354+0000","last_became_peered":"2024-03-29T04:18:58.671354+0000","last_unstale":"2024-03-29T04:18:59.679196+0000","last_undegraded":"2024-03-29T04:18:59.679196+0000","last_fullsized":"2024-03-29T04:18:59.679196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:23:56.383735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680306+0000","last_change":"2024-03-29T04:18:57.651021+0000","last_active":"2024-03-29T04:18:59.680306+0000","last_peered":"2024-03-29T04:18:59.680306+0000","last_clean":"2024-03-29T04:18:59.680306+0000","last_became_active":"2024-03-29T04:18:57.644336+0000","last_became_peered":"2024-03-29T04:18:57.644336+0000","last_unstale":"2024-03-29T04:18:59.680306+0000","last_undegraded":"2024-03-29T04:18:59.680306+0000","last_fullsized":"2024-03-29T04:18:59.680306+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:40:57.459208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679535+0000","last_change":"2024-03-29T04:18:58.663084+0000","last_active":"2024-03-29T04:18:59.679535+0000","last_peered":"2024-03-29T04:18:59.679535+0000","last_clean":"2024-03-29T04:18:59.679535+0000","last_became_active":"2024-03-29T04:18:58.662284+0000","last_became_peered":"2024-03-29T04:18:58.662284+0000","last_unstale":"2024-03-29T04:18:59.679535+0000","last_undegraded":"2024-03-29T04:18:59.679535+0000","last_fullsized":"2024-03-29T04:18:59.679535+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:35:33.293712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690330+0000","last_change":"2024-03-29T04:18:57.650116+0000","last_active":"2024-03-29T04:18:59.690330+0000","last_peered":"2024-03-29T04:18:59.690330+0000","last_clean":"2024-03-29T04:18:59.690330+0000","last_became_active":"2024-03-29T04:18:57.638212+0000","last_became_peered":"2024-03-29T04:18:57.638212+0000","last_unstale":"2024-03-29T04:18:59.690330+0000","last_undegraded":"2024-03-29T04:18:59.690330+0000","last_fullsized":"2024-03-29T04:18:59.690330+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:07:36.203300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690104+0000","last_change":"2024-03-29T04:18:58.684505+0000","last_active":"2024-03-29T04:18:59.690104+0000","last_peered":"2024-03-29T04:18:59.690104+0000","last_clean":"2024-03-29T04:18:59.690104+0000","last_became_active":"2024-03-29T04:18:58.683416+0000","last_became_peered":"2024-03-29T04:18:58.683416+0000","last_unstale":"2024-03-29T04:18:59.690104+0000","last_undegraded":"2024-03-29T04:18:59.690104+0000","last_fullsized":"2024-03-29T04:18:59.690104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:25:17.586041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.189045+0000","last_change":"2024-03-29T04:18:57.665159+0000","last_active":"2024-03-29T04:19:01.189045+0000","last_peered":"2024-03-29T04:19:01.189045+0000","last_clean":"2024-03-29T04:19:01.189045+0000","last_became_active":"2024-03-29T04:18:57.664442+0000","last_became_peered":"2024-03-29T04:18:57.664442+0000","last_unstale":"2024-03-29T04:19:01.189045+0000","last_undegraded":"2024-03-29T04:19:01.189045+0000","last_fullsized":"2024-03-29T04:19:01.189045+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:08:44.644431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.681031+0000","last_change":"2024-03-29T04:18:58.673773+0000","last_active":"2024-03-29T04:18:59.681031+0000","last_peered":"2024-03-29T04:18:59.681031+0000","last_clean":"2024-03-29T04:18:59.681031+0000","last_became_active":"2024-03-29T04:18:58.672730+0000","last_became_peered":"2024-03-29T04:18:58.672730+0000","last_unstale":"2024-03-29T04:18:59.681031+0000","last_undegraded":"2024-03-29T04:18:59.681031+0000","last_fullsized":"2024-03-29T04:18:59.681031+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:22:11.812410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:00.276741+0000","last_change":"2024-03-29T04:18:57.662680+0000","last_active":"2024-03-29T04:19:00.276741+0000","last_peered":"2024-03-29T04:19:00.276741+0000","last_clean":"2024-03-29T04:19:00.276741+0000","last_became_active":"2024-03-29T04:18:57.662148+0000","last_became_peered":"2024-03-29T04:18:57.662148+0000","last_unstale":"2024-03-29T04:19:00.276741+0000","last_undegraded":"2024-03-29T04:19:00.276741+0000","last_fullsized":"2024-03-29T04:19:00.276741+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:40:57.459208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679116+0000","last_change":"2024-03-29T04:18:58.669641+0000","last_active":"2024-03-29T04:18:59.679116+0000","last_peered":"2024-03-29T04:18:59.679116+0000","last_clean":"2024-03-29T04:18:59.679116+0000","last_became_active":"2024-03-29T04:18:58.668314+0000","last_became_peered":"2024-03-29T04:18:58.668314+0000","last_unstale":"2024-03-29T04:18:59.679116+0000","last_undegraded":"2024-03-29T04:18:59.679116+0000","last_fullsized":"2024-03-29T04:18:59.679116+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:24:51.163863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679676+0000","last_change":"2024-03-29T04:18:57.652332+0000","last_active":"2024-03-29T04:18:59.679676+0000","last_peered":"2024-03-29T04:18:59.679676+0000","last_clean":"2024-03-29T04:18:59.679676+0000","last_became_active":"2024-03-29T04:18:57.651656+0000","last_became_peered":"2024-03-29T04:18:57.651656+0000","last_unstale":"2024-03-29T04:18:59.679676+0000","last_undegraded":"2024-03-29T04:18:59.679676+0000","last_fullsized":"2024-03-29T04:18:59.679676+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:51.553019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685695+0000","last_change":"2024-03-29T04:18:58.675709+0000","last_active":"2024-03-29T04:18:59.685695+0000","last_peered":"2024-03-29T04:18:59.685695+0000","last_clean":"2024-03-29T04:18:59.685695+0000","last_became_active":"2024-03-29T04:18:58.675390+0000","last_became_peered":"2024-03-29T04:18:58.675390+0000","last_unstale":"2024-03-29T04:18:59.685695+0000","last_undegraded":"2024-03-29T04:18:59.685695+0000","last_fullsized":"2024-03-29T04:18:59.685695+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:58.175168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679666+0000","last_change":"2024-03-29T04:18:57.633865+0000","last_active":"2024-03-29T04:18:59.679666+0000","last_peered":"2024-03-29T04:18:59.679666+0000","last_clean":"2024-03-29T04:18:59.679666+0000","last_became_active":"2024-03-29T04:18:57.633449+0000","last_became_peered":"2024-03-29T04:18:57.633449+0000","last_unstale":"2024-03-29T04:18:59.679666+0000","last_undegraded":"2024-03-29T04:18:59.679666+0000","last_fullsized":"2024-03-29T04:18:59.679666+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:39:46.271436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.190158+0000","last_change":"2024-03-29T04:18:58.679651+0000","last_active":"2024-03-29T04:19:01.190158+0000","last_peered":"2024-03-29T04:19:01.190158+0000","last_clean":"2024-03-29T04:19:01.190158+0000","last_became_active":"2024-03-29T04:18:58.678853+0000","last_became_peered":"2024-03-29T04:18:58.678853+0000","last_unstale":"2024-03-29T04:19:01.190158+0000","last_undegraded":"2024-03-29T04:19:01.190158+0000","last_fullsized":"2024-03-29T04:19:01.190158+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:13:22.147476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690793+0000","last_change":"2024-03-29T04:18:57.665734+0000","last_active":"2024-03-29T04:18:59.690793+0000","last_peered":"2024-03-29T04:18:59.690793+0000","last_clean":"2024-03-29T04:18:59.690793+0000","last_became_active":"2024-03-29T04:18:57.662309+0000","last_became_peered":"2024-03-29T04:18:57.662309+0000","last_unstale":"2024-03-29T04:18:59.690793+0000","last_undegraded":"2024-03-29T04:18:59.690793+0000","last_fullsized":"2024-03-29T04:18:59.690793+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:35:32.285417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680175+0000","last_change":"2024-03-29T04:18:58.680125+0000","last_active":"2024-03-29T04:18:59.680175+0000","last_peered":"2024-03-29T04:18:59.680175+0000","last_clean":"2024-03-29T04:18:59.680175+0000","last_became_active":"2024-03-29T04:18:58.679866+0000","last_became_peered":"2024-03-29T04:18:58.679866+0000","last_unstale":"2024-03-29T04:18:59.680175+0000","last_undegraded":"2024-03-29T04:18:59.680175+0000","last_fullsized":"2024-03-29T04:18:59.680175+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:29.155520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690962+0000","last_change":"2024-03-29T04:18:57.660878+0000","last_active":"2024-03-29T04:18:59.690962+0000","last_peered":"2024-03-29T04:18:59.690962+0000","last_clean":"2024-03-29T04:18:59.690962+0000","last_became_active":"2024-03-29T04:18:57.659621+0000","last_became_peered":"2024-03-29T04:18:57.659621+0000","last_unstale":"2024-03-29T04:18:59.690962+0000","last_undegraded":"2024-03-29T04:18:59.690962+0000","last_fullsized":"2024-03-29T04:18:59.690962+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:21.603876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680600+0000","last_change":"2024-03-29T04:18:58.680251+0000","last_active":"2024-03-29T04:18:59.680600+0000","last_peered":"2024-03-29T04:18:59.680600+0000","last_clean":"2024-03-29T04:18:59.680600+0000","last_became_active":"2024-03-29T04:18:58.678048+0000","last_became_peered":"2024-03-29T04:18:58.678048+0000","last_unstale":"2024-03-29T04:18:59.680600+0000","last_undegraded":"2024-03-29T04:18:59.680600+0000","last_fullsized":"2024-03-29T04:18:59.680600+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:29.155520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679320+0000","last_change":"2024-03-29T04:18:57.650361+0000","last_active":"2024-03-29T04:18:59.679320+0000","last_peered":"2024-03-29T04:18:59.679320+0000","last_clean":"2024-03-29T04:18:59.679320+0000","last_became_active":"2024-03-29T04:18:57.650006+0000","last_became_peered":"2024-03-29T04:18:57.650006+0000","last_unstale":"2024-03-29T04:18:59.679320+0000","last_undegraded":"2024-03-29T04:18:59.679320+0000","last_fullsized":"2024-03-29T04:18:59.679320+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:35:32.285417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679695+0000","last_change":"2024-03-29T04:18:58.685094+0000","last_active":"2024-03-29T04:18:59.679695+0000","last_peered":"2024-03-29T04:18:59.679695+0000","last_clean":"2024-03-29T04:18:59.679695+0000","last_became_active":"2024-03-29T04:18:58.684840+0000","last_became_peered":"2024-03-29T04:18:58.684840+0000","last_unstale":"2024-03-29T04:18:59.679695+0000","last_undegraded":"2024-03-29T04:18:59.679695+0000","last_fullsized":"2024-03-29T04:18:59.679695+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:04.997398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679915+0000","last_change":"2024-03-29T04:18:57.633188+0000","last_active":"2024-03-29T04:18:59.679915+0000","last_peered":"2024-03-29T04:18:59.679915+0000","last_clean":"2024-03-29T04:18:59.679915+0000","last_became_active":"2024-03-29T04:18:57.632810+0000","last_became_peered":"2024-03-29T04:18:57.632810+0000","last_unstale":"2024-03-29T04:18:59.679915+0000","last_undegraded":"2024-03-29T04:18:59.679915+0000","last_fullsized":"2024-03-29T04:18:59.679915+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:03.989103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679531+0000","last_change":"2024-03-29T04:18:58.658312+0000","last_active":"2024-03-29T04:18:59.679531+0000","last_peered":"2024-03-29T04:18:59.679531+0000","last_clean":"2024-03-29T04:18:59.679531+0000","last_became_active":"2024-03-29T04:18:58.657363+0000","last_became_peered":"2024-03-29T04:18:58.657363+0000","last_unstale":"2024-03-29T04:18:59.679531+0000","last_undegraded":"2024-03-29T04:18:59.679531+0000","last_fullsized":"2024-03-29T04:18:59.679531+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:33:30.490101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:00.276582+0000","last_change":"2024-03-29T04:18:57.666809+0000","last_active":"2024-03-29T04:19:00.276582+0000","last_peered":"2024-03-29T04:19:00.276582+0000","last_clean":"2024-03-29T04:19:00.276582+0000","last_became_active":"2024-03-29T04:18:57.666382+0000","last_became_peered":"2024-03-29T04:18:57.666382+0000","last_unstale":"2024-03-29T04:19:00.276582+0000","last_undegraded":"2024-03-29T04:19:00.276582+0000","last_fullsized":"2024-03-29T04:19:00.276582+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:51.553019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.189095+0000","last_change":"2024-03-29T04:18:58.698349+0000","last_active":"2024-03-29T04:19:01.189095+0000","last_peered":"2024-03-29T04:19:01.189095+0000","last_clean":"2024-03-29T04:19:01.189095+0000","last_became_active":"2024-03-29T04:18:58.688432+0000","last_became_peered":"2024-03-29T04:18:58.688432+0000","last_unstale":"2024-03-29T04:19:01.189095+0000","last_undegraded":"2024-03-29T04:19:01.189095+0000","last_fullsized":"2024-03-29T04:19:01.189095+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:39:47.279731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.189137+0000","last_change":"2024-03-29T04:18:57.653609+0000","last_active":"2024-03-29T04:19:01.189137+0000","last_peered":"2024-03-29T04:19:01.189137+0000","last_clean":"2024-03-29T04:19:01.189137+0000","last_became_active":"2024-03-29T04:18:57.653049+0000","last_became_peered":"2024-03-29T04:18:57.653049+0000","last_unstale":"2024-03-29T04:19:01.189137+0000","last_undegraded":"2024-03-29T04:19:01.189137+0000","last_fullsized":"2024-03-29T04:19:01.189137+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:57.166873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679680+0000","last_change":"2024-03-29T04:18:58.666495+0000","last_active":"2024-03-29T04:18:59.679680+0000","last_peered":"2024-03-29T04:18:59.679680+0000","last_clean":"2024-03-29T04:18:59.679680+0000","last_became_active":"2024-03-29T04:18:58.665812+0000","last_became_peered":"2024-03-29T04:18:58.665812+0000","last_unstale":"2024-03-29T04:18:59.679680+0000","last_undegraded":"2024-03-29T04:18:59.679680+0000","last_fullsized":"2024-03-29T04:18:59.679680+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:03:55.569008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.188840+0000","last_change":"2024-03-29T04:18:57.664980+0000","last_active":"2024-03-29T04:19:01.188840+0000","last_peered":"2024-03-29T04:19:01.188840+0000","last_clean":"2024-03-29T04:19:01.188840+0000","last_became_active":"2024-03-29T04:18:57.664377+0000","last_became_peered":"2024-03-29T04:18:57.664377+0000","last_unstale":"2024-03-29T04:19:01.188840+0000","last_undegraded":"2024-03-29T04:19:01.188840+0000","last_fullsized":"2024-03-29T04:19:01.188840+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:56:06.055281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679320+0000","last_change":"2024-03-29T04:18:58.663161+0000","last_active":"2024-03-29T04:18:59.679320+0000","last_peered":"2024-03-29T04:18:59.679320+0000","last_clean":"2024-03-29T04:18:59.679320+0000","last_became_active":"2024-03-29T04:18:58.662297+0000","last_became_peered":"2024-03-29T04:18:58.662297+0000","last_unstale":"2024-03-29T04:18:59.679320+0000","last_undegraded":"2024-03-29T04:18:59.679320+0000","last_fullsized":"2024-03-29T04:18:59.679320+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:06:56.949031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690646+0000","last_change":"2024-03-29T04:18:57.666734+0000","last_active":"2024-03-29T04:18:59.690646+0000","last_peered":"2024-03-29T04:18:59.690646+0000","last_clean":"2024-03-29T04:18:59.690646+0000","last_became_active":"2024-03-29T04:18:57.666091+0000","last_became_peered":"2024-03-29T04:18:57.666091+0000","last_unstale":"2024-03-29T04:18:59.690646+0000","last_undegraded":"2024-03-29T04:18:59.690646+0000","last_fullsized":"2024-03-29T04:18:59.690646+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:53:46.438562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690718+0000","last_change":"2024-03-29T04:18:58.702282+0000","last_active":"2024-03-29T04:18:59.690718+0000","last_peered":"2024-03-29T04:18:59.690718+0000","last_clean":"2024-03-29T04:18:59.690718+0000","last_became_active":"2024-03-29T04:18:58.695850+0000","last_became_peered":"2024-03-29T04:18:58.695850+0000","last_unstale":"2024-03-29T04:18:59.690718+0000","last_undegraded":"2024-03-29T04:18:59.690718+0000","last_fullsized":"2024-03-29T04:18:59.690718+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:02:00.617566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680948+0000","last_change":"2024-03-29T04:18:57.666956+0000","last_active":"2024-03-29T04:18:59.680948+0000","last_peered":"2024-03-29T04:18:59.680948+0000","last_clean":"2024-03-29T04:18:59.680948+0000","last_became_active":"2024-03-29T04:18:57.666383+0000","last_became_peered":"2024-03-29T04:18:57.666383+0000","last_unstale":"2024-03-29T04:18:59.680948+0000","last_undegraded":"2024-03-29T04:18:59.680948+0000","last_fullsized":"2024-03-29T04:18:59.680948+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:46:56.030179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.681319+0000","last_change":"2024-03-29T04:18:58.665254+0000","last_active":"2024-03-29T04:18:59.681319+0000","last_peered":"2024-03-29T04:18:59.681319+0000","last_clean":"2024-03-29T04:18:59.681319+0000","last_became_active":"2024-03-29T04:18:58.664330+0000","last_became_peered":"2024-03-29T04:18:58.664330+0000","last_unstale":"2024-03-29T04:18:59.681319+0000","last_undegraded":"2024-03-29T04:18:59.681319+0000","last_fullsized":"2024-03-29T04:18:59.681319+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:08:09.926082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685328+0000","last_change":"2024-03-29T04:18:57.649911+0000","last_active":"2024-03-29T04:18:59.685328+0000","last_peered":"2024-03-29T04:18:59.685328+0000","last_clean":"2024-03-29T04:18:59.685328+0000","last_became_active":"2024-03-29T04:18:57.649663+0000","last_became_peered":"2024-03-29T04:18:57.649663+0000","last_unstale":"2024-03-29T04:18:59.685328+0000","last_undegraded":"2024-03-29T04:18:59.685328+0000","last_fullsized":"2024-03-29T04:18:59.685328+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:43.915489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685432+0000","last_change":"2024-03-29T04:18:58.679364+0000","last_active":"2024-03-29T04:18:59.685432+0000","last_peered":"2024-03-29T04:18:59.685432+0000","last_clean":"2024-03-29T04:18:59.685432+0000","last_became_active":"2024-03-29T04:18:58.679008+0000","last_became_peered":"2024-03-29T04:18:58.679008+0000","last_unstale":"2024-03-29T04:18:59.685432+0000","last_undegraded":"2024-03-29T04:18:59.685432+0000","last_fullsized":"2024-03-29T04:18:59.685432+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:18:31.187655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:00.276983+0000","last_change":"2024-03-29T04:18:57.654452+0000","last_active":"2024-03-29T04:19:00.276983+0000","last_peered":"2024-03-29T04:19:00.276983+0000","last_clean":"2024-03-29T04:19:00.276983+0000","last_became_active":"2024-03-29T04:18:57.654084+0000","last_became_peered":"2024-03-29T04:18:57.654084+0000","last_unstale":"2024-03-29T04:19:00.276983+0000","last_undegraded":"2024-03-29T04:19:00.276983+0000","last_fullsized":"2024-03-29T04:19:00.276983+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:46:56.030179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.189288+0000","last_change":"2024-03-29T04:18:58.679822+0000","last_active":"2024-03-29T04:19:01.189288+0000","last_peered":"2024-03-29T04:19:01.189288+0000","last_clean":"2024-03-29T04:19:01.189288+0000","last_became_active":"2024-03-29T04:18:58.679080+0000","last_became_peered":"2024-03-29T04:18:58.679080+0000","last_unstale":"2024-03-29T04:19:01.189288+0000","last_undegraded":"2024-03-29T04:19:01.189288+0000","last_fullsized":"2024-03-29T04:19:01.189288+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:22.612171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685131+0000","last_change":"2024-03-29T04:18:57.633652+0000","last_active":"2024-03-29T04:18:59.685131+0000","last_peered":"2024-03-29T04:18:59.685131+0000","last_clean":"2024-03-29T04:18:59.685131+0000","last_became_active":"2024-03-29T04:18:57.633156+0000","last_became_peered":"2024-03-29T04:18:57.633156+0000","last_unstale":"2024-03-29T04:18:59.685131+0000","last_undegraded":"2024-03-29T04:18:59.685131+0000","last_fullsized":"2024-03-29T04:18:59.685131+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:59:31.934017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680416+0000","last_change":"2024-03-29T04:18:58.699660+0000","last_active":"2024-03-29T04:18:59.680416+0000","last_peered":"2024-03-29T04:18:59.680416+0000","last_clean":"2024-03-29T04:18:59.680416+0000","last_became_active":"2024-03-29T04:18:58.699390+0000","last_became_peered":"2024-03-29T04:18:58.699390+0000","last_unstale":"2024-03-29T04:18:59.680416+0000","last_undegraded":"2024-03-29T04:18:59.680416+0000","last_fullsized":"2024-03-29T04:18:59.680416+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:22:11.812410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685123+0000","last_change":"2024-03-29T04:18:57.663600+0000","last_active":"2024-03-29T04:18:59.685123+0000","last_peered":"2024-03-29T04:18:59.685123+0000","last_clean":"2024-03-29T04:18:59.685123+0000","last_became_active":"2024-03-29T04:18:57.663253+0000","last_became_peered":"2024-03-29T04:18:57.663253+0000","last_unstale":"2024-03-29T04:18:59.685123+0000","last_undegraded":"2024-03-29T04:18:59.685123+0000","last_fullsized":"2024-03-29T04:18:59.685123+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:25:11.876318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.689912+0000","last_change":"2024-03-29T04:18:58.670617+0000","last_active":"2024-03-29T04:18:59.689912+0000","last_peered":"2024-03-29T04:18:59.689912+0000","last_clean":"2024-03-29T04:18:59.689912+0000","last_became_active":"2024-03-29T04:18:58.669817+0000","last_became_peered":"2024-03-29T04:18:58.669817+0000","last_unstale":"2024-03-29T04:18:59.689912+0000","last_undegraded":"2024-03-29T04:18:59.689912+0000","last_fullsized":"2024-03-29T04:18:59.689912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:09:58.485244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.189518+0000","last_change":"2024-03-29T04:18:57.647273+0000","last_active":"2024-03-29T04:19:01.189518+0000","last_peered":"2024-03-29T04:19:01.189518+0000","last_clean":"2024-03-29T04:19:01.189518+0000","last_became_active":"2024-03-29T04:18:57.646106+0000","last_became_peered":"2024-03-29T04:18:57.646106+0000","last_unstale":"2024-03-29T04:19:01.189518+0000","last_undegraded":"2024-03-29T04:19:01.189518+0000","last_fullsized":"2024-03-29T04:19:01.189518+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:32.935677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679590+0000","last_change":"2024-03-29T04:18:58.668993+0000","last_active":"2024-03-29T04:18:59.679590+0000","last_peered":"2024-03-29T04:18:59.679590+0000","last_clean":"2024-03-29T04:18:59.679590+0000","last_became_active":"2024-03-29T04:18:58.667339+0000","last_became_peered":"2024-03-29T04:18:58.667339+0000","last_unstale":"2024-03-29T04:18:59.679590+0000","last_undegraded":"2024-03-29T04:18:59.679590+0000","last_fullsized":"2024-03-29T04:18:59.679590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:36:33.283150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685589+0000","last_change":"2024-03-29T04:18:57.633301+0000","last_active":"2024-03-29T04:18:59.685589+0000","last_peered":"2024-03-29T04:18:59.685589+0000","last_clean":"2024-03-29T04:18:59.685589+0000","last_became_active":"2024-03-29T04:18:57.633009+0000","last_became_peered":"2024-03-29T04:18:57.633009+0000","last_unstale":"2024-03-29T04:18:59.685589+0000","last_undegraded":"2024-03-29T04:18:59.685589+0000","last_fullsized":"2024-03-29T04:18:59.685589+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:26:20.759660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:00.277026+0000","last_change":"2024-03-29T04:18:58.708059+0000","last_active":"2024-03-29T04:19:00.277026+0000","last_peered":"2024-03-29T04:19:00.277026+0000","last_clean":"2024-03-29T04:19:00.277026+0000","last_became_active":"2024-03-29T04:18:58.707750+0000","last_became_peered":"2024-03-29T04:18:58.707750+0000","last_unstale":"2024-03-29T04:19:00.277026+0000","last_undegraded":"2024-03-29T04:19:00.277026+0000","last_fullsized":"2024-03-29T04:19:00.277026+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:29.155520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690919+0000","last_change":"2024-03-29T04:18:57.660454+0000","last_active":"2024-03-29T04:18:59.690919+0000","last_peered":"2024-03-29T04:18:59.690919+0000","last_clean":"2024-03-29T04:18:59.690919+0000","last_became_active":"2024-03-29T04:18:57.650654+0000","last_became_peered":"2024-03-29T04:18:57.650654+0000","last_unstale":"2024-03-29T04:18:59.690919+0000","last_undegraded":"2024-03-29T04:18:59.690919+0000","last_fullsized":"2024-03-29T04:18:59.690919+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:03.989103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.190373+0000","last_change":"2024-03-29T04:18:58.693884+0000","last_active":"2024-03-29T04:19:01.190373+0000","last_peered":"2024-03-29T04:19:01.190373+0000","last_clean":"2024-03-29T04:19:01.190373+0000","last_became_active":"2024-03-29T04:18:58.693143+0000","last_became_peered":"2024-03-29T04:18:58.693143+0000","last_unstale":"2024-03-29T04:19:01.190373+0000","last_undegraded":"2024-03-29T04:19:01.190373+0000","last_fullsized":"2024-03-29T04:19:01.190373+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:27:32.000679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685656+0000","last_change":"2024-03-29T04:18:57.662501+0000","last_active":"2024-03-29T04:18:59.685656+0000","last_peered":"2024-03-29T04:18:59.685656+0000","last_clean":"2024-03-29T04:18:59.685656+0000","last_became_active":"2024-03-29T04:18:57.662145+0000","last_became_peered":"2024-03-29T04:18:57.662145+0000","last_unstale":"2024-03-29T04:18:59.685656+0000","last_undegraded":"2024-03-29T04:18:59.685656+0000","last_fullsized":"2024-03-29T04:18:59.685656+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:10:06.936644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679865+0000","last_change":"2024-03-29T04:18:58.673901+0000","last_active":"2024-03-29T04:18:59.679865+0000","last_peered":"2024-03-29T04:18:59.679865+0000","last_clean":"2024-03-29T04:18:59.679865+0000","last_became_active":"2024-03-29T04:18:58.673075+0000","last_became_peered":"2024-03-29T04:18:58.673075+0000","last_unstale":"2024-03-29T04:18:59.679865+0000","last_undegraded":"2024-03-29T04:18:59.679865+0000","last_fullsized":"2024-03-29T04:18:59.679865+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:30:41.634747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680393+0000","last_change":"2024-03-29T04:18:57.635473+0000","last_active":"2024-03-29T04:18:59.680393+0000","last_peered":"2024-03-29T04:18:59.680393+0000","last_clean":"2024-03-29T04:18:59.680393+0000","last_became_active":"2024-03-29T04:18:57.635129+0000","last_became_peered":"2024-03-29T04:18:57.635129+0000","last_unstale":"2024-03-29T04:18:59.680393+0000","last_undegraded":"2024-03-29T04:18:59.680393+0000","last_fullsized":"2024-03-29T04:18:59.680393+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:08:44.644431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679832+0000","last_change":"2024-03-29T04:18:58.663702+0000","last_active":"2024-03-29T04:18:59.679832+0000","last_peered":"2024-03-29T04:18:59.679832+0000","last_clean":"2024-03-29T04:18:59.679832+0000","last_became_active":"2024-03-29T04:18:58.662253+0000","last_became_peered":"2024-03-29T04:18:58.662253+0000","last_unstale":"2024-03-29T04:18:59.679832+0000","last_undegraded":"2024-03-29T04:18:59.679832+0000","last_fullsized":"2024-03-29T04:18:59.679832+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:39:47.279731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680436+0000","last_change":"2024-03-29T04:18:57.641648+0000","last_active":"2024-03-29T04:18:59.680436+0000","last_peered":"2024-03-29T04:18:59.680436+0000","last_clean":"2024-03-29T04:18:59.680436+0000","last_became_active":"2024-03-29T04:18:57.641274+0000","last_became_peered":"2024-03-29T04:18:57.641274+0000","last_unstale":"2024-03-29T04:18:59.680436+0000","last_undegraded":"2024-03-29T04:18:59.680436+0000","last_fullsized":"2024-03-29T04:18:59.680436+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:51.553019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:00.276577+0000","last_change":"2024-03-29T04:18:58.672593+0000","last_active":"2024-03-29T04:19:00.276577+0000","last_peered":"2024-03-29T04:19:00.276577+0000","last_clean":"2024-03-29T04:19:00.276577+0000","last_became_active":"2024-03-29T04:18:58.672222+0000","last_became_peered":"2024-03-29T04:18:58.672222+0000","last_unstale":"2024-03-29T04:19:00.276577+0000","last_undegraded":"2024-03-29T04:19:00.276577+0000","last_fullsized":"2024-03-29T04:19:00.276577+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:04.997398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.190132+0000","last_change":"2024-03-29T04:18:57.636067+0000","last_active":"2024-03-29T04:19:01.190132+0000","last_peered":"2024-03-29T04:19:01.190132+0000","last_clean":"2024-03-29T04:19:01.190132+0000","last_became_active":"2024-03-29T04:18:57.635270+0000","last_became_peered":"2024-03-29T04:18:57.635270+0000","last_unstale":"2024-03-29T04:19:01.190132+0000","last_undegraded":"2024-03-29T04:19:01.190132+0000","last_fullsized":"2024-03-29T04:19:01.190132+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:18:30.179360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680272+0000","last_change":"2024-03-29T04:18:58.658440+0000","last_active":"2024-03-29T04:18:59.680272+0000","last_peered":"2024-03-29T04:18:59.680272+0000","last_clean":"2024-03-29T04:18:59.680272+0000","last_became_active":"2024-03-29T04:18:58.657732+0000","last_became_peered":"2024-03-29T04:18:58.657732+0000","last_unstale":"2024-03-29T04:18:59.680272+0000","last_undegraded":"2024-03-29T04:18:59.680272+0000","last_fullsized":"2024-03-29T04:18:59.680272+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:15:56.537650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679883+0000","last_change":"2024-03-29T04:18:57.643649+0000","last_active":"2024-03-29T04:18:59.679883+0000","last_peered":"2024-03-29T04:18:59.679883+0000","last_clean":"2024-03-29T04:18:59.679883+0000","last_became_active":"2024-03-29T04:18:57.642751+0000","last_became_peered":"2024-03-29T04:18:57.642751+0000","last_unstale":"2024-03-29T04:18:59.679883+0000","last_undegraded":"2024-03-29T04:18:59.679883+0000","last_fullsized":"2024-03-29T04:18:59.679883+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:16:07.746204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.189874+0000","last_change":"2024-03-29T04:18:58.699368+0000","last_active":"2024-03-29T04:19:01.189874+0000","last_peered":"2024-03-29T04:19:01.189874+0000","last_clean":"2024-03-29T04:19:01.189874+0000","last_became_active":"2024-03-29T04:18:58.698691+0000","last_became_peered":"2024-03-29T04:18:58.698691+0000","last_unstale":"2024-03-29T04:19:01.189874+0000","last_undegraded":"2024-03-29T04:19:01.189874+0000","last_fullsized":"2024-03-29T04:19:01.189874+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:08:09.926082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.678997+0000","last_change":"2024-03-29T04:18:57.633269+0000","last_active":"2024-03-29T04:18:59.678997+0000","last_peered":"2024-03-29T04:18:59.678997+0000","last_clean":"2024-03-29T04:18:59.678997+0000","last_became_active":"2024-03-29T04:18:57.632982+0000","last_became_peered":"2024-03-29T04:18:57.632982+0000","last_unstale":"2024-03-29T04:18:59.678997+0000","last_undegraded":"2024-03-29T04:18:59.678997+0000","last_fullsized":"2024-03-29T04:18:59.678997+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:53:46.438562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690328+0000","last_change":"2024-03-29T04:18:58.706519+0000","last_active":"2024-03-29T04:18:59.690328+0000","last_peered":"2024-03-29T04:18:59.690328+0000","last_clean":"2024-03-29T04:18:59.690328+0000","last_became_active":"2024-03-29T04:18:58.702146+0000","last_became_peered":"2024-03-29T04:18:58.702146+0000","last_unstale":"2024-03-29T04:18:59.690328+0000","last_undegraded":"2024-03-29T04:18:59.690328+0000","last_fullsized":"2024-03-29T04:18:59.690328+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:23:56.383735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.190126+0000","last_change":"2024-03-29T04:18:57.655959+0000","last_active":"2024-03-29T04:19:01.190126+0000","last_peered":"2024-03-29T04:19:01.190126+0000","last_clean":"2024-03-29T04:19:01.190126+0000","last_became_active":"2024-03-29T04:18:57.651470+0000","last_became_peered":"2024-03-29T04:18:57.651470+0000","last_unstale":"2024-03-29T04:19:01.190126+0000","last_undegraded":"2024-03-29T04:19:01.190126+0000","last_fullsized":"2024-03-29T04:19:01.190126+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:51.553019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":206,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.189517+0000","last_change":"2024-03-29T04:17:25.361826+0000","last_active":"2024-03-29T04:19:01.189517+0000","last_peered":"2024-03-29T04:19:01.189517+0000","last_clean":"2024-03-29T04:19:01.189517+0000","last_became_active":"2024-03-29T04:17:25.361259+0000","last_became_peered":"2024-03-29T04:17:25.361259+0000","last_unstale":"2024-03-29T04:19:01.189517+0000","last_undegraded":"2024-03-29T04:19:01.189517+0000","last_fullsized":"2024-03-29T04:19:01.189517+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_clean_scrub_stamp":"2024-03-29T04:17:24.334639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:40.027873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1073152,"data_stored":793886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":202418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":201948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":202440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31648,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340192,"statfs":{"total":96636764160,"available":96604356608,"internally_reserved":0,"allocated":479232,"data_stored":203482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":466944,"data_stored":201906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31636,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":466944,"data_stored":201926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1060864,"data_stored":792238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:19:07.233 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:19:07.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.383+0000 7ff72b97f640 1 Processor -- start 2024-03-29T04:19:07.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff72b97f640 1 -- start start 2024-03-29T04:19:07.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff72b97f640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff724070490 0x7ff72415c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:07.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff72b97f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff72406de90 0x7ff72406e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:07.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff72b97f640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff72415c670 con 0x7ff7240b5b20 2024-03-29T04:19:07.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff72b97f640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7ff72415edd0 con 0x7ff724070490 2024-03-29T04:19:07.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff72b97f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff72415f0b0 con 0x7ff72406de90 2024-03-29T04:19:07.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff728f1a640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff7240b5b20 0x7ff72406d780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57028/0 (socket says 172.21.15.22:57028) 2024-03-29T04:19:07.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff728f1a640 1 -- 172.21.15.22:0/1963181355 learned_addr learned my addr 172.21.15.22:0/1963181355 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:19:07.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff72971b640 1 --2- 172.21.15.22:0/1963181355 >> v2:172.21.15.151:3300/0 conn(0x7ff724070490 0x7ff72415c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:07.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff71bfff640 1 -- 172.21.15.22:0/1963181355 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 693385058 0 0) 0x7ff714001120 con 0x7ff7240b5b20 2024-03-29T04:19:07.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff729f1c640 1 --2- 172.21.15.22:0/1963181355 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff72406de90 0x7ff72406e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:07.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff71bfff640 1 -- 172.21.15.22:0/1963181355 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff710003530 con 0x7ff7240b5b20 2024-03-29T04:19:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff71bfff640 1 -- 172.21.15.22:0/1963181355 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 887504505 0 0) 0x7ff710003530 con 0x7ff7240b5b20 2024-03-29T04:19:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff71bfff640 1 -- 172.21.15.22:0/1963181355 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff72406de90 msgr2=0x7ff72406e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff71bfff640 1 --2- 172.21.15.22:0/1963181355 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff72406de90 0x7ff72406e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff71bfff640 1 -- 172.21.15.22:0/1963181355 >> v2:172.21.15.151:3300/0 conn(0x7ff724070490 msgr2=0x7ff72415c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff71bfff640 1 --2- 172.21.15.22:0/1963181355 >> v2:172.21.15.151:3300/0 conn(0x7ff724070490 0x7ff72415c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff71bfff640 1 -- 172.21.15.22:0/1963181355 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72415f390 con 0x7ff7240b5b20 2024-03-29T04:19:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff71bfff640 1 -- 172.21.15.22:0/1963181355 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7ff714001850 con 0x7ff7240b5b20 2024-03-29T04:19:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff71bfff640 1 -- 172.21.15.22:0/1963181355 >> v1:172.21.15.22:6789/0 conn(0x7ff7240b5b20 legacy=0x7ff72406d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff71bfff640 1 --2- 172.21.15.22:0/1963181355 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff72406de90 0x7ff7100039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff71bfff640 1 --2- 172.21.15.22:0/1963181355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff710003f40 0x7ff710004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff71bfff640 1 --2- 172.21.15.22:0/1963181355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff710004d50 0x7ff7100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff71bfff640 1 -- 172.21.15.22:0/1963181355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff72415edd0 con 0x7ff710004d50 2024-03-29T04:19:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff71bfff640 1 -- 172.21.15.22:0/1963181355 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff72415f0b0 con 0x7ff710003f40 2024-03-29T04:19:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff71bfff640 1 -- 172.21.15.22:0/1963181355 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff71000bb60 con 0x7ff72406de90 2024-03-29T04:19:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff729f1c640 1 --2- 172.21.15.22:0/1963181355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff710004d50 0x7ff7100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:07.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff72971b640 1 --2- 172.21.15.22:0/1963181355 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff72406de90 0x7ff7100039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:07.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff728f1a640 1 --2- 172.21.15.22:0/1963181355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff710003f40 0x7ff710004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:07.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff729f1c640 1 -- 172.21.15.22:0/1963181355 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff72406de90 msgr2=0x7ff7100039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:07.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff729f1c640 1 --2- 172.21.15.22:0/1963181355 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff72406de90 0x7ff7100039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:07.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff729f1c640 1 -- 172.21.15.22:0/1963181355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff710003f40 msgr2=0x7ff710004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:07.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff729f1c640 1 --2- 172.21.15.22:0/1963181355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff710003f40 0x7ff710004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:07.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff729f1c640 1 -- 172.21.15.22:0/1963181355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff710003530 con 0x7ff710004d50 2024-03-29T04:19:07.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.387+0000 7ff728f1a640 1 --2- 172.21.15.22:0/1963181355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff710003f40 0x7ff710004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:19:07.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.391+0000 7ff729f1c640 1 --2- 172.21.15.22:0/1963181355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff710004d50 0x7ff7100051d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff7241592a0 tx=0x7ff71c01cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e9dd7615eec6e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:07.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.391+0000 7ff71bfff640 1 -- 172.21.15.22:0/1963181355 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c0b2070 con 0x7ff710004d50 2024-03-29T04:19:07.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.391+0000 7ff71bfff640 1 -- 172.21.15.22:0/1963181355 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff71c0a5bd0 con 0x7ff710004d50 2024-03-29T04:19:07.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.391+0000 7ff71bfff640 1 -- 172.21.15.22:0/1963181355 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c009be0 con 0x7ff710004d50 2024-03-29T04:19:07.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.391+0000 7ff72b97f640 1 -- 172.21.15.22:0/1963181355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff710004d50 msgr2=0x7ff7100051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:07.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.391+0000 7ff72b97f640 1 --2- 172.21.15.22:0/1963181355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff710004d50 0x7ff7100051d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff7241592a0 tx=0x7ff71c01cde0 comp rx=0 tx=0).stop 2024-03-29T04:19:07.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.391+0000 7ff729f1c640 1 -- 172.21.15.22:0/1963181355 reap_dead start 2024-03-29T04:19:07.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.391+0000 7ff72b97f640 1 -- 172.21.15.22:0/1963181355 shutdown_connections 2024-03-29T04:19:07.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.391+0000 7ff72b97f640 1 -- 172.21.15.22:0/1963181355 >> 172.21.15.22:0/1963181355 conn(0x7ff724072500 msgr2=0x7ff724072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:07.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.391+0000 7ff72b97f640 1 -- 172.21.15.22:0/1963181355 shutdown_connections 2024-03-29T04:19:07.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.391+0000 7ff72b97f640 1 -- 172.21.15.22:0/1963181355 wait complete. 2024-03-29T04:19:07.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.391+0000 7ff72b97f640 1 Processor -- start 2024-03-29T04:19:07.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.391+0000 7ff72b97f640 1 -- start start 2024-03-29T04:19:07.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.391+0000 7ff72b97f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff710004d50 0x7ff7240d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:07.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff72b97f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff72406d200 0x7ff72406b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:07.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff72b97f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff72413f9b0 0x7ff72415a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:07.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff72b97f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff72406f390 con 0x7ff72406d200 2024-03-29T04:19:07.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff72b97f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff72406f210 con 0x7ff710004d50 2024-03-29T04:19:07.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff72b97f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff72406f510 con 0x7ff72413f9b0 2024-03-29T04:19:07.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff728f1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff72406d200 0x7ff72406b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:07.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff728f1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff72406d200 0x7ff72406b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35858/0 (socket says 172.21.15.22:35858) 2024-03-29T04:19:07.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff728f1a640 1 -- 172.21.15.22:0/2715012027 learned_addr learned my addr 172.21.15.22:0/2715012027 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:07.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff72971b640 1 --2- 172.21.15.22:0/2715012027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff710004d50 0x7ff7240d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:07.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff729f1c640 1 --2- 172.21.15.22:0/2715012027 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff72413f9b0 0x7ff72415a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:07.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff728f1a640 1 -- 172.21.15.22:0/2715012027 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff72413f9b0 msgr2=0x7ff72415a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:07.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff728f1a640 1 --2- 172.21.15.22:0/2715012027 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff72413f9b0 0x7ff72415a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:07.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff728f1a640 1 -- 172.21.15.22:0/2715012027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff710004d50 msgr2=0x7ff7240d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:07.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff728f1a640 1 --2- 172.21.15.22:0/2715012027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff710004d50 0x7ff7240d09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:07.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff728f1a640 1 -- 172.21.15.22:0/2715012027 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72413fe40 con 0x7ff72406d200 2024-03-29T04:19:07.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff72971b640 1 --2- 172.21.15.22:0/2715012027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff710004d50 0x7ff7240d09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:19:07.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff728f1a640 1 --2- 172.21.15.22:0/2715012027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff72406d200 0x7ff72406b7c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff714007fd0 tx=0x7ff71400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:07.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff719ffb640 1 -- 172.21.15.22:0/2715012027 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714019070 con 0x7ff72406d200 2024-03-29T04:19:07.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff719ffb640 1 -- 172.21.15.22:0/2715012027 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7140074e0 con 0x7ff72406d200 2024-03-29T04:19:07.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff72b97f640 1 -- 172.21.15.22:0/2715012027 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff72406c160 con 0x7ff72406d200 2024-03-29T04:19:07.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff719ffb640 1 -- 172.21.15.22:0/2715012027 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7140049f0 con 0x7ff72406d200 2024-03-29T04:19:07.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff72b97f640 1 -- 172.21.15.22:0/2715012027 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff72406c720 con 0x7ff72406d200 2024-03-29T04:19:07.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff72b97f640 1 -- 172.21.15.22:0/2715012027 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff724068890 con 0x7ff72406d200 2024-03-29T04:19:07.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff719ffb640 1 -- 172.21.15.22:0/2715012027 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff71401c070 con 0x7ff72406d200 2024-03-29T04:19:07.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff719ffb640 1 --2- 172.21.15.22:0/2715012027 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff708067fc0 0x7ff70806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:07.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff719ffb640 1 -- 172.21.15.22:0/2715012027 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7ff714086130 con 0x7ff72406d200 2024-03-29T04:19:07.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.395+0000 7ff72971b640 1 --2- 172.21.15.22:0/2715012027 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff708067fc0 0x7ff70806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:07.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.399+0000 7ff72971b640 1 --2- 172.21.15.22:0/2715012027 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff708067fc0 0x7ff70806a480 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff720009b70 tx=0x7ff720008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:07.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.399+0000 7ff719ffb640 1 -- 172.21.15.22:0/2715012027 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff72415c670 con 0x7ff72406d200 2024-03-29T04:19:07.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.559+0000 7ff72b97f640 1 -- 172.21.15.22:0/2715012027 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff72415c9d0 con 0x7ff708067fc0 2024-03-29T04:19:07.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.575+0000 7ff719ffb640 1 -- 172.21.15.22:0/2715012027 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251967 (secure 0 0 0) 0x7ff72415c9d0 con 0x7ff708067fc0 2024-03-29T04:19:07.560 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:19:07.562 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-29T04:19:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.579+0000 7ff72b97f640 1 -- 172.21.15.22:0/2715012027 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff708067fc0 msgr2=0x7ff70806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.579+0000 7ff72b97f640 1 --2- 172.21.15.22:0/2715012027 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff708067fc0 0x7ff70806a480 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff720009b70 tx=0x7ff720008040 comp rx=0 tx=0).stop 2024-03-29T04:19:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.579+0000 7ff72b97f640 1 -- 172.21.15.22:0/2715012027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff72406d200 msgr2=0x7ff72406b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.579+0000 7ff72b97f640 1 --2- 172.21.15.22:0/2715012027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff72406d200 0x7ff72406b7c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff714007fd0 tx=0x7ff71400dd40 comp rx=0 tx=0).stop 2024-03-29T04:19:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.579+0000 7ff72b97f640 1 -- 172.21.15.22:0/2715012027 shutdown_connections 2024-03-29T04:19:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.579+0000 7ff72b97f640 1 --2- 172.21.15.22:0/2715012027 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff72413f9b0 0x7ff72415a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.579+0000 7ff72b97f640 1 --2- 172.21.15.22:0/2715012027 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff708067fc0 0x7ff70806a480 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.579+0000 7ff72b97f640 1 --2- 172.21.15.22:0/2715012027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff72406d200 0x7ff72406b7c0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.579+0000 7ff72b97f640 1 --2- 172.21.15.22:0/2715012027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff710004d50 0x7ff7240d09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.579+0000 7ff72b97f640 1 -- 172.21.15.22:0/2715012027 >> 172.21.15.22:0/2715012027 conn(0x7ff724072500 msgr2=0x7ff724159910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.579+0000 7ff72b97f640 1 -- 172.21.15.22:0/2715012027 shutdown_connections 2024-03-29T04:19:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.579+0000 7ff72b97f640 1 -- 172.21.15.22:0/2715012027 wait complete. 2024-03-29T04:19:07.585 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-03-29T04:19:05.723250+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241832,"kb_used_data":4840,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732888,"statfs":{"total":773094113280,"available":772846477312,"internally_reserved":0,"allocated":4956160,"data_stored":2800244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":50,"apply_latency_ms":50,"commit_latency_ns":50000000,"apply_latency_ns":50000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002620"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679228+0000","last_change":"2024-03-29T04:18:58.667680+0000","last_active":"2024-03-29T04:18:59.679228+0000","last_peered":"2024-03-29T04:18:59.679228+0000","last_clean":"2024-03-29T04:18:59.679228+0000","last_became_active":"2024-03-29T04:18:58.666927+0000","last_became_peered":"2024-03-29T04:18:58.666927+0000","last_unstale":"2024-03-29T04:18:59.679228+0000","last_undegraded":"2024-03-29T04:18:59.679228+0000","last_fullsized":"2024-03-29T04:18:59.679228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:43:42.447228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679463+0000","last_change":"2024-03-29T04:18:58.674238+0000","last_active":"2024-03-29T04:18:59.679463+0000","last_peered":"2024-03-29T04:18:59.679463+0000","last_clean":"2024-03-29T04:18:59.679463+0000","last_became_active":"2024-03-29T04:18:58.673550+0000","last_became_peered":"2024-03-29T04:18:58.673550+0000","last_unstale":"2024-03-29T04:18:59.679463+0000","last_undegraded":"2024-03-29T04:18:59.679463+0000","last_fullsized":"2024-03-29T04:18:59.679463+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:02:00.617566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680064+0000","last_change":"2024-03-29T04:18:58.685574+0000","last_active":"2024-03-29T04:18:59.680064+0000","last_peered":"2024-03-29T04:18:59.680064+0000","last_clean":"2024-03-29T04:18:59.680064+0000","last_became_active":"2024-03-29T04:18:58.685311+0000","last_became_peered":"2024-03-29T04:18:58.685311+0000","last_unstale":"2024-03-29T04:18:59.680064+0000","last_undegraded":"2024-03-29T04:18:59.680064+0000","last_fullsized":"2024-03-29T04:18:59.680064+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:22.612171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685955+0000","last_change":"2024-03-29T04:18:58.679166+0000","last_active":"2024-03-29T04:18:59.685955+0000","last_peered":"2024-03-29T04:18:59.685955+0000","last_clean":"2024-03-29T04:18:59.685955+0000","last_became_active":"2024-03-29T04:18:58.678739+0000","last_became_peered":"2024-03-29T04:18:58.678739+0000","last_unstale":"2024-03-29T04:18:59.685955+0000","last_undegraded":"2024-03-29T04:18:59.685955+0000","last_fullsized":"2024-03-29T04:18:59.685955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:56:07.063576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685428+0000","last_change":"2024-03-29T04:18:58.667545+0000","last_active":"2024-03-29T04:18:59.685428+0000","last_peered":"2024-03-29T04:18:59.685428+0000","last_clean":"2024-03-29T04:18:59.685428+0000","last_became_active":"2024-03-29T04:18:58.666778+0000","last_became_peered":"2024-03-29T04:18:58.666778+0000","last_unstale":"2024-03-29T04:18:59.685428+0000","last_undegraded":"2024-03-29T04:18:59.685428+0000","last_fullsized":"2024-03-29T04:18:59.685428+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:38.275333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:00.276713+0000","last_change":"2024-03-29T04:18:58.675636+0000","last_active":"2024-03-29T04:19:00.276713+0000","last_peered":"2024-03-29T04:19:00.276713+0000","last_clean":"2024-03-29T04:19:00.276713+0000","last_became_active":"2024-03-29T04:18:58.675323+0000","last_became_peered":"2024-03-29T04:18:58.675323+0000","last_unstale":"2024-03-29T04:19:00.276713+0000","last_undegraded":"2024-03-29T04:19:00.276713+0000","last_fullsized":"2024-03-29T04:19:00.276713+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:15:56.537650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.689657+0000","last_change":"2024-03-29T04:18:58.695819+0000","last_active":"2024-03-29T04:18:59.689657+0000","last_peered":"2024-03-29T04:18:59.689657+0000","last_clean":"2024-03-29T04:18:59.689657+0000","last_became_active":"2024-03-29T04:18:58.695069+0000","last_became_peered":"2024-03-29T04:18:58.695069+0000","last_unstale":"2024-03-29T04:18:59.689657+0000","last_undegraded":"2024-03-29T04:18:59.689657+0000","last_fullsized":"2024-03-29T04:18:59.689657+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:08:09.926082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.678898+0000","last_change":"2024-03-29T04:18:58.684531+0000","last_active":"2024-03-29T04:18:59.678898+0000","last_peered":"2024-03-29T04:18:59.678898+0000","last_clean":"2024-03-29T04:18:59.678898+0000","last_became_active":"2024-03-29T04:18:58.684234+0000","last_became_peered":"2024-03-29T04:18:58.684234+0000","last_unstale":"2024-03-29T04:18:59.678898+0000","last_undegraded":"2024-03-29T04:18:59.678898+0000","last_fullsized":"2024-03-29T04:18:59.678898+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:23:01.781748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680131+0000","last_change":"2024-03-29T04:18:58.684993+0000","last_active":"2024-03-29T04:18:59.680131+0000","last_peered":"2024-03-29T04:18:59.680131+0000","last_clean":"2024-03-29T04:18:59.680131+0000","last_became_active":"2024-03-29T04:18:58.684718+0000","last_became_peered":"2024-03-29T04:18:58.684718+0000","last_unstale":"2024-03-29T04:18:59.680131+0000","last_undegraded":"2024-03-29T04:18:59.680131+0000","last_fullsized":"2024-03-29T04:18:59.680131+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:15:56.537650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690369+0000","last_change":"2024-03-29T04:18:58.695211+0000","last_active":"2024-03-29T04:18:59.690369+0000","last_peered":"2024-03-29T04:18:59.690369+0000","last_clean":"2024-03-29T04:18:59.690369+0000","last_became_active":"2024-03-29T04:18:58.694519+0000","last_became_peered":"2024-03-29T04:18:58.694519+0000","last_unstale":"2024-03-29T04:18:59.690369+0000","last_undegraded":"2024-03-29T04:18:59.690369+0000","last_fullsized":"2024-03-29T04:18:59.690369+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:24:51.163863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690459+0000","last_change":"2024-03-29T04:18:58.701560+0000","last_active":"2024-03-29T04:18:59.690459+0000","last_peered":"2024-03-29T04:18:59.690459+0000","last_clean":"2024-03-29T04:18:59.690459+0000","last_became_active":"2024-03-29T04:18:58.699252+0000","last_became_peered":"2024-03-29T04:18:58.699252+0000","last_unstale":"2024-03-29T04:18:59.690459+0000","last_undegraded":"2024-03-29T04:18:59.690459+0000","last_fullsized":"2024-03-29T04:18:59.690459+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:43:42.447228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680479+0000","last_change":"2024-03-29T04:18:58.679122+0000","last_active":"2024-03-29T04:18:59.680479+0000","last_peered":"2024-03-29T04:18:59.680479+0000","last_clean":"2024-03-29T04:18:59.680479+0000","last_became_active":"2024-03-29T04:18:58.678691+0000","last_became_peered":"2024-03-29T04:18:58.678691+0000","last_unstale":"2024-03-29T04:18:59.680479+0000","last_undegraded":"2024-03-29T04:18:59.680479+0000","last_fullsized":"2024-03-29T04:18:59.680479+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:57.410807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.689847+0000","last_change":"2024-03-29T04:18:58.701524+0000","last_active":"2024-03-29T04:18:59.689847+0000","last_peered":"2024-03-29T04:18:59.689847+0000","last_clean":"2024-03-29T04:18:59.689847+0000","last_became_active":"2024-03-29T04:18:58.700452+0000","last_became_peered":"2024-03-29T04:18:58.700452+0000","last_unstale":"2024-03-29T04:18:59.689847+0000","last_undegraded":"2024-03-29T04:18:59.689847+0000","last_fullsized":"2024-03-29T04:18:59.689847+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:27:32.000679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679936+0000","last_change":"2024-03-29T04:18:58.670545+0000","last_active":"2024-03-29T04:18:59.679936+0000","last_peered":"2024-03-29T04:18:59.679936+0000","last_clean":"2024-03-29T04:18:59.679936+0000","last_became_active":"2024-03-29T04:18:58.669825+0000","last_became_peered":"2024-03-29T04:18:58.669825+0000","last_unstale":"2024-03-29T04:18:59.679936+0000","last_undegraded":"2024-03-29T04:18:59.679936+0000","last_fullsized":"2024-03-29T04:18:59.679936+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:39:47.279731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.689686+0000","last_change":"2024-03-29T04:18:58.693326+0000","last_active":"2024-03-29T04:18:59.689686+0000","last_peered":"2024-03-29T04:18:59.689686+0000","last_clean":"2024-03-29T04:18:59.689686+0000","last_became_active":"2024-03-29T04:18:58.682482+0000","last_became_peered":"2024-03-29T04:18:58.682482+0000","last_unstale":"2024-03-29T04:18:59.689686+0000","last_undegraded":"2024-03-29T04:18:59.689686+0000","last_fullsized":"2024-03-29T04:18:59.689686+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:13:22.147476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.188883+0000","last_change":"2024-03-29T04:18:58.712646+0000","last_active":"2024-03-29T04:19:01.188883+0000","last_peered":"2024-03-29T04:19:01.188883+0000","last_clean":"2024-03-29T04:19:01.188883+0000","last_became_active":"2024-03-29T04:18:58.711875+0000","last_became_peered":"2024-03-29T04:18:58.711875+0000","last_unstale":"2024-03-29T04:19:01.188883+0000","last_undegraded":"2024-03-29T04:19:01.188883+0000","last_fullsized":"2024-03-29T04:19:01.188883+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:15:56.537650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.190703+0000","last_change":"2024-03-29T04:18:58.713921+0000","last_active":"2024-03-29T04:19:01.190703+0000","last_peered":"2024-03-29T04:19:01.190703+0000","last_clean":"2024-03-29T04:19:01.190703+0000","last_became_active":"2024-03-29T04:18:58.713600+0000","last_became_peered":"2024-03-29T04:18:58.713600+0000","last_unstale":"2024-03-29T04:19:01.190703+0000","last_undegraded":"2024-03-29T04:19:01.190703+0000","last_fullsized":"2024-03-29T04:19:01.190703+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:09:58.485244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.191428+0000","last_change":"2024-03-29T04:18:58.678265+0000","last_active":"2024-03-29T04:19:01.191428+0000","last_peered":"2024-03-29T04:19:01.191428+0000","last_clean":"2024-03-29T04:19:01.191428+0000","last_became_active":"2024-03-29T04:18:58.677498+0000","last_became_peered":"2024-03-29T04:18:58.677498+0000","last_unstale":"2024-03-29T04:19:01.191428+0000","last_undegraded":"2024-03-29T04:19:01.191428+0000","last_fullsized":"2024-03-29T04:19:01.191428+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:35.314491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685058+0000","last_change":"2024-03-29T04:18:58.677361+0000","last_active":"2024-03-29T04:18:59.685058+0000","last_peered":"2024-03-29T04:18:59.685058+0000","last_clean":"2024-03-29T04:18:59.685058+0000","last_became_active":"2024-03-29T04:18:58.677022+0000","last_became_peered":"2024-03-29T04:18:58.677022+0000","last_unstale":"2024-03-29T04:18:59.685058+0000","last_undegraded":"2024-03-29T04:18:59.685058+0000","last_fullsized":"2024-03-29T04:18:59.685058+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:14:27.390921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.191082+0000","last_change":"2024-03-29T04:18:58.667731+0000","last_active":"2024-03-29T04:19:01.191082+0000","last_peered":"2024-03-29T04:19:01.191082+0000","last_clean":"2024-03-29T04:19:01.191082+0000","last_became_active":"2024-03-29T04:18:58.666974+0000","last_became_peered":"2024-03-29T04:18:58.666974+0000","last_unstale":"2024-03-29T04:19:01.191082+0000","last_undegraded":"2024-03-29T04:19:01.191082+0000","last_fullsized":"2024-03-29T04:19:01.191082+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:07:10.650606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690288+0000","last_change":"2024-03-29T04:18:58.683859+0000","last_active":"2024-03-29T04:18:59.690288+0000","last_peered":"2024-03-29T04:18:59.690288+0000","last_clean":"2024-03-29T04:18:59.690288+0000","last_became_active":"2024-03-29T04:18:58.683080+0000","last_became_peered":"2024-03-29T04:18:58.683080+0000","last_unstale":"2024-03-29T04:18:59.690288+0000","last_undegraded":"2024-03-29T04:18:59.690288+0000","last_fullsized":"2024-03-29T04:18:59.690288+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:23:01.781748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.681281+0000","last_change":"2024-03-29T04:18:58.686729+0000","last_active":"2024-03-29T04:18:59.681281+0000","last_peered":"2024-03-29T04:18:59.681281+0000","last_clean":"2024-03-29T04:18:59.681281+0000","last_became_active":"2024-03-29T04:18:58.684822+0000","last_became_peered":"2024-03-29T04:18:58.684822+0000","last_unstale":"2024-03-29T04:18:59.681281+0000","last_undegraded":"2024-03-29T04:18:59.681281+0000","last_fullsized":"2024-03-29T04:18:59.681281+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:57.410807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.189240+0000","last_change":"2024-03-29T04:18:58.699382+0000","last_active":"2024-03-29T04:19:01.189240+0000","last_peered":"2024-03-29T04:19:01.189240+0000","last_clean":"2024-03-29T04:19:01.189240+0000","last_became_active":"2024-03-29T04:18:58.698814+0000","last_became_peered":"2024-03-29T04:18:58.698814+0000","last_unstale":"2024-03-29T04:19:01.189240+0000","last_undegraded":"2024-03-29T04:19:01.189240+0000","last_fullsized":"2024-03-29T04:19:01.189240+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:29.155520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685354+0000","last_change":"2024-03-29T04:18:58.696560+0000","last_active":"2024-03-29T04:18:59.685354+0000","last_peered":"2024-03-29T04:18:59.685354+0000","last_clean":"2024-03-29T04:18:59.685354+0000","last_became_active":"2024-03-29T04:18:58.696138+0000","last_became_peered":"2024-03-29T04:18:58.696138+0000","last_unstale":"2024-03-29T04:18:59.685354+0000","last_undegraded":"2024-03-29T04:18:59.685354+0000","last_fullsized":"2024-03-29T04:18:59.685354+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:32:55.508364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.191515+0000","last_change":"2024-03-29T04:18:58.662099+0000","last_active":"2024-03-29T04:19:01.191515+0000","last_peered":"2024-03-29T04:19:01.191515+0000","last_clean":"2024-03-29T04:19:01.191515+0000","last_became_active":"2024-03-29T04:18:58.661339+0000","last_became_peered":"2024-03-29T04:18:58.661339+0000","last_unstale":"2024-03-29T04:19:01.191515+0000","last_undegraded":"2024-03-29T04:19:01.191515+0000","last_fullsized":"2024-03-29T04:19:01.191515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:07:37.211595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:00.277442+0000","last_change":"2024-03-29T04:18:58.671142+0000","last_active":"2024-03-29T04:19:00.277442+0000","last_peered":"2024-03-29T04:19:00.277442+0000","last_clean":"2024-03-29T04:19:00.277442+0000","last_became_active":"2024-03-29T04:18:58.670258+0000","last_became_peered":"2024-03-29T04:18:58.670258+0000","last_unstale":"2024-03-29T04:19:00.277442+0000","last_undegraded":"2024-03-29T04:19:00.277442+0000","last_fullsized":"2024-03-29T04:19:00.277442+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:22:11.812410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.191104+0000","last_change":"2024-03-29T04:18:58.688585+0000","last_active":"2024-03-29T04:19:01.191104+0000","last_peered":"2024-03-29T04:19:01.191104+0000","last_clean":"2024-03-29T04:19:01.191104+0000","last_became_active":"2024-03-29T04:18:58.687856+0000","last_became_peered":"2024-03-29T04:18:58.687856+0000","last_unstale":"2024-03-29T04:19:01.191104+0000","last_undegraded":"2024-03-29T04:19:01.191104+0000","last_fullsized":"2024-03-29T04:19:01.191104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:56:46.808829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690669+0000","last_change":"2024-03-29T04:18:58.695512+0000","last_active":"2024-03-29T04:18:59.690669+0000","last_peered":"2024-03-29T04:18:59.690669+0000","last_clean":"2024-03-29T04:18:59.690669+0000","last_became_active":"2024-03-29T04:18:58.694782+0000","last_became_peered":"2024-03-29T04:18:58.694782+0000","last_unstale":"2024-03-29T04:18:59.690669+0000","last_undegraded":"2024-03-29T04:18:59.690669+0000","last_fullsized":"2024-03-29T04:18:59.690669+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:06:56.949031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.190830+0000","last_change":"2024-03-29T04:18:58.677751+0000","last_active":"2024-03-29T04:19:01.190830+0000","last_peered":"2024-03-29T04:19:01.190830+0000","last_clean":"2024-03-29T04:19:01.190830+0000","last_became_active":"2024-03-29T04:18:58.676895+0000","last_became_peered":"2024-03-29T04:18:58.676895+0000","last_unstale":"2024-03-29T04:19:01.190830+0000","last_undegraded":"2024-03-29T04:19:01.190830+0000","last_fullsized":"2024-03-29T04:19:01.190830+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:58:16.237991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.190189+0000","last_change":"2024-03-29T04:18:58.676978+0000","last_active":"2024-03-29T04:19:01.190189+0000","last_peered":"2024-03-29T04:19:01.190189+0000","last_clean":"2024-03-29T04:19:01.190189+0000","last_became_active":"2024-03-29T04:18:58.676274+0000","last_became_peered":"2024-03-29T04:18:58.676274+0000","last_unstale":"2024-03-29T04:19:01.190189+0000","last_undegraded":"2024-03-29T04:19:01.190189+0000","last_fullsized":"2024-03-29T04:19:01.190189+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:46:57.038475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.189670+0000","last_change":"2024-03-29T04:18:58.695203+0000","last_active":"2024-03-29T04:19:01.189670+0000","last_peered":"2024-03-29T04:19:01.189670+0000","last_clean":"2024-03-29T04:19:01.189670+0000","last_became_active":"2024-03-29T04:18:58.687916+0000","last_became_peered":"2024-03-29T04:18:58.687916+0000","last_unstale":"2024-03-29T04:19:01.189670+0000","last_undegraded":"2024-03-29T04:19:01.189670+0000","last_fullsized":"2024-03-29T04:19:01.189670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:22:11.812410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:00.276489+0000","last_change":"2024-03-29T04:18:58.666952+0000","last_active":"2024-03-29T04:19:00.276489+0000","last_peered":"2024-03-29T04:19:00.276489+0000","last_clean":"2024-03-29T04:19:00.276489+0000","last_became_active":"2024-03-29T04:18:58.666176+0000","last_became_peered":"2024-03-29T04:18:58.666176+0000","last_unstale":"2024-03-29T04:19:00.276489+0000","last_undegraded":"2024-03-29T04:19:00.276489+0000","last_fullsized":"2024-03-29T04:19:00.276489+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:39:47.279731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.189769+0000","last_change":"2024-03-29T04:18:58.706576+0000","last_active":"2024-03-29T04:19:01.189769+0000","last_peered":"2024-03-29T04:19:01.189769+0000","last_clean":"2024-03-29T04:19:01.189769+0000","last_became_active":"2024-03-29T04:18:58.689768+0000","last_became_peered":"2024-03-29T04:18:58.689768+0000","last_unstale":"2024-03-29T04:19:01.189769+0000","last_undegraded":"2024-03-29T04:19:01.189769+0000","last_fullsized":"2024-03-29T04:19:01.189769+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:57.410807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.191573+0000","last_change":"2024-03-29T04:18:57.652890+0000","last_active":"2024-03-29T04:19:01.191573+0000","last_peered":"2024-03-29T04:19:01.191573+0000","last_clean":"2024-03-29T04:19:01.191573+0000","last_became_active":"2024-03-29T04:18:57.652244+0000","last_became_peered":"2024-03-29T04:18:57.652244+0000","last_unstale":"2024-03-29T04:19:01.191573+0000","last_undegraded":"2024-03-29T04:19:01.191573+0000","last_fullsized":"2024-03-29T04:19:01.191573+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:37.267037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:00.277370+0000","last_change":"2024-03-29T04:18:58.708190+0000","last_active":"2024-03-29T04:19:00.277370+0000","last_peered":"2024-03-29T04:19:00.277370+0000","last_clean":"2024-03-29T04:19:00.277370+0000","last_became_active":"2024-03-29T04:18:58.707780+0000","last_became_peered":"2024-03-29T04:18:58.707780+0000","last_unstale":"2024-03-29T04:19:00.277370+0000","last_undegraded":"2024-03-29T04:19:00.277370+0000","last_fullsized":"2024-03-29T04:19:00.277370+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:22.612171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680302+0000","last_change":"2024-03-29T04:18:57.646783+0000","last_active":"2024-03-29T04:18:59.680302+0000","last_peered":"2024-03-29T04:18:59.680302+0000","last_clean":"2024-03-29T04:18:59.680302+0000","last_became_active":"2024-03-29T04:18:57.646495+0000","last_became_peered":"2024-03-29T04:18:57.646495+0000","last_unstale":"2024-03-29T04:18:59.680302+0000","last_undegraded":"2024-03-29T04:18:59.680302+0000","last_fullsized":"2024-03-29T04:18:59.680302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:32.935677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680690+0000","last_change":"2024-03-29T04:18:58.668119+0000","last_active":"2024-03-29T04:18:59.680690+0000","last_peered":"2024-03-29T04:18:59.680690+0000","last_clean":"2024-03-29T04:18:59.680690+0000","last_became_active":"2024-03-29T04:18:58.667409+0000","last_became_peered":"2024-03-29T04:18:58.667409+0000","last_unstale":"2024-03-29T04:18:59.680690+0000","last_undegraded":"2024-03-29T04:18:59.680690+0000","last_fullsized":"2024-03-29T04:18:59.680690+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:22.612171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:00.277524+0000","last_change":"2024-03-29T04:18:57.654191+0000","last_active":"2024-03-29T04:19:00.277524+0000","last_peered":"2024-03-29T04:19:00.277524+0000","last_clean":"2024-03-29T04:19:00.277524+0000","last_became_active":"2024-03-29T04:18:57.653911+0000","last_became_peered":"2024-03-29T04:18:57.653911+0000","last_unstale":"2024-03-29T04:19:00.277524+0000","last_undegraded":"2024-03-29T04:19:00.277524+0000","last_fullsized":"2024-03-29T04:19:00.277524+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:16:07.746204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679196+0000","last_change":"2024-03-29T04:18:58.671727+0000","last_active":"2024-03-29T04:18:59.679196+0000","last_peered":"2024-03-29T04:18:59.679196+0000","last_clean":"2024-03-29T04:18:59.679196+0000","last_became_active":"2024-03-29T04:18:58.671354+0000","last_became_peered":"2024-03-29T04:18:58.671354+0000","last_unstale":"2024-03-29T04:18:59.679196+0000","last_undegraded":"2024-03-29T04:18:59.679196+0000","last_fullsized":"2024-03-29T04:18:59.679196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:23:56.383735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680306+0000","last_change":"2024-03-29T04:18:57.651021+0000","last_active":"2024-03-29T04:18:59.680306+0000","last_peered":"2024-03-29T04:18:59.680306+0000","last_clean":"2024-03-29T04:18:59.680306+0000","last_became_active":"2024-03-29T04:18:57.644336+0000","last_became_peered":"2024-03-29T04:18:57.644336+0000","last_unstale":"2024-03-29T04:18:59.680306+0000","last_undegraded":"2024-03-29T04:18:59.680306+0000","last_fullsized":"2024-03-29T04:18:59.680306+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:40:57.459208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679535+0000","last_change":"2024-03-29T04:18:58.663084+0000","last_active":"2024-03-29T04:18:59.679535+0000","last_peered":"2024-03-29T04:18:59.679535+0000","last_clean":"2024-03-29T04:18:59.679535+0000","last_became_active":"2024-03-29T04:18:58.662284+0000","last_became_peered":"2024-03-29T04:18:58.662284+0000","last_unstale":"2024-03-29T04:18:59.679535+0000","last_undegraded":"2024-03-29T04:18:59.679535+0000","last_fullsized":"2024-03-29T04:18:59.679535+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:35:33.293712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690330+0000","last_change":"2024-03-29T04:18:57.650116+0000","last_active":"2024-03-29T04:18:59.690330+0000","last_peered":"2024-03-29T04:18:59.690330+0000","last_clean":"2024-03-29T04:18:59.690330+0000","last_became_active":"2024-03-29T04:18:57.638212+0000","last_became_peered":"2024-03-29T04:18:57.638212+0000","last_unstale":"2024-03-29T04:18:59.690330+0000","last_undegraded":"2024-03-29T04:18:59.690330+0000","last_fullsized":"2024-03-29T04:18:59.690330+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:07:36.203300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690104+0000","last_change":"2024-03-29T04:18:58.684505+0000","last_active":"2024-03-29T04:18:59.690104+0000","last_peered":"2024-03-29T04:18:59.690104+0000","last_clean":"2024-03-29T04:18:59.690104+0000","last_became_active":"2024-03-29T04:18:58.683416+0000","last_became_peered":"2024-03-29T04:18:58.683416+0000","last_unstale":"2024-03-29T04:18:59.690104+0000","last_undegraded":"2024-03-29T04:18:59.690104+0000","last_fullsized":"2024-03-29T04:18:59.690104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:25:17.586041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.189045+0000","last_change":"2024-03-29T04:18:57.665159+0000","last_active":"2024-03-29T04:19:01.189045+0000","last_peered":"2024-03-29T04:19:01.189045+0000","last_clean":"2024-03-29T04:19:01.189045+0000","last_became_active":"2024-03-29T04:18:57.664442+0000","last_became_peered":"2024-03-29T04:18:57.664442+0000","last_unstale":"2024-03-29T04:19:01.189045+0000","last_undegraded":"2024-03-29T04:19:01.189045+0000","last_fullsized":"2024-03-29T04:19:01.189045+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:08:44.644431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.681031+0000","last_change":"2024-03-29T04:18:58.673773+0000","last_active":"2024-03-29T04:18:59.681031+0000","last_peered":"2024-03-29T04:18:59.681031+0000","last_clean":"2024-03-29T04:18:59.681031+0000","last_became_active":"2024-03-29T04:18:58.672730+0000","last_became_peered":"2024-03-29T04:18:58.672730+0000","last_unstale":"2024-03-29T04:18:59.681031+0000","last_undegraded":"2024-03-29T04:18:59.681031+0000","last_fullsized":"2024-03-29T04:18:59.681031+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:22:11.812410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:00.276741+0000","last_change":"2024-03-29T04:18:57.662680+0000","last_active":"2024-03-29T04:19:00.276741+0000","last_peered":"2024-03-29T04:19:00.276741+0000","last_clean":"2024-03-29T04:19:00.276741+0000","last_became_active":"2024-03-29T04:18:57.662148+0000","last_became_peered":"2024-03-29T04:18:57.662148+0000","last_unstale":"2024-03-29T04:19:00.276741+0000","last_undegraded":"2024-03-29T04:19:00.276741+0000","last_fullsized":"2024-03-29T04:19:00.276741+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:40:57.459208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679116+0000","last_change":"2024-03-29T04:18:58.669641+0000","last_active":"2024-03-29T04:18:59.679116+0000","last_peered":"2024-03-29T04:18:59.679116+0000","last_clean":"2024-03-29T04:18:59.679116+0000","last_became_active":"2024-03-29T04:18:58.668314+0000","last_became_peered":"2024-03-29T04:18:58.668314+0000","last_unstale":"2024-03-29T04:18:59.679116+0000","last_undegraded":"2024-03-29T04:18:59.679116+0000","last_fullsized":"2024-03-29T04:18:59.679116+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:24:51.163863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679676+0000","last_change":"2024-03-29T04:18:57.652332+0000","last_active":"2024-03-29T04:18:59.679676+0000","last_peered":"2024-03-29T04:18:59.679676+0000","last_clean":"2024-03-29T04:18:59.679676+0000","last_became_active":"2024-03-29T04:18:57.651656+0000","last_became_peered":"2024-03-29T04:18:57.651656+0000","last_unstale":"2024-03-29T04:18:59.679676+0000","last_undegraded":"2024-03-29T04:18:59.679676+0000","last_fullsized":"2024-03-29T04:18:59.679676+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:51.553019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685695+0000","last_change":"2024-03-29T04:18:58.675709+0000","last_active":"2024-03-29T04:18:59.685695+0000","last_peered":"2024-03-29T04:18:59.685695+0000","last_clean":"2024-03-29T04:18:59.685695+0000","last_became_active":"2024-03-29T04:18:58.675390+0000","last_became_peered":"2024-03-29T04:18:58.675390+0000","last_unstale":"2024-03-29T04:18:59.685695+0000","last_undegraded":"2024-03-29T04:18:59.685695+0000","last_fullsized":"2024-03-29T04:18:59.685695+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:58.175168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679666+0000","last_change":"2024-03-29T04:18:57.633865+0000","last_active":"2024-03-29T04:18:59.679666+0000","last_peered":"2024-03-29T04:18:59.679666+0000","last_clean":"2024-03-29T04:18:59.679666+0000","last_became_active":"2024-03-29T04:18:57.633449+0000","last_became_peered":"2024-03-29T04:18:57.633449+0000","last_unstale":"2024-03-29T04:18:59.679666+0000","last_undegraded":"2024-03-29T04:18:59.679666+0000","last_fullsized":"2024-03-29T04:18:59.679666+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:39:46.271436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.190158+0000","last_change":"2024-03-29T04:18:58.679651+0000","last_active":"2024-03-29T04:19:01.190158+0000","last_peered":"2024-03-29T04:19:01.190158+0000","last_clean":"2024-03-29T04:19:01.190158+0000","last_became_active":"2024-03-29T04:18:58.678853+0000","last_became_peered":"2024-03-29T04:18:58.678853+0000","last_unstale":"2024-03-29T04:19:01.190158+0000","last_undegraded":"2024-03-29T04:19:01.190158+0000","last_fullsized":"2024-03-29T04:19:01.190158+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:13:22.147476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690793+0000","last_change":"2024-03-29T04:18:57.665734+0000","last_active":"2024-03-29T04:18:59.690793+0000","last_peered":"2024-03-29T04:18:59.690793+0000","last_clean":"2024-03-29T04:18:59.690793+0000","last_became_active":"2024-03-29T04:18:57.662309+0000","last_became_peered":"2024-03-29T04:18:57.662309+0000","last_unstale":"2024-03-29T04:18:59.690793+0000","last_undegraded":"2024-03-29T04:18:59.690793+0000","last_fullsized":"2024-03-29T04:18:59.690793+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:35:32.285417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680175+0000","last_change":"2024-03-29T04:18:58.680125+0000","last_active":"2024-03-29T04:18:59.680175+0000","last_peered":"2024-03-29T04:18:59.680175+0000","last_clean":"2024-03-29T04:18:59.680175+0000","last_became_active":"2024-03-29T04:18:58.679866+0000","last_became_peered":"2024-03-29T04:18:58.679866+0000","last_unstale":"2024-03-29T04:18:59.680175+0000","last_undegraded":"2024-03-29T04:18:59.680175+0000","last_fullsized":"2024-03-29T04:18:59.680175+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:29.155520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690962+0000","last_change":"2024-03-29T04:18:57.660878+0000","last_active":"2024-03-29T04:18:59.690962+0000","last_peered":"2024-03-29T04:18:59.690962+0000","last_clean":"2024-03-29T04:18:59.690962+0000","last_became_active":"2024-03-29T04:18:57.659621+0000","last_became_peered":"2024-03-29T04:18:57.659621+0000","last_unstale":"2024-03-29T04:18:59.690962+0000","last_undegraded":"2024-03-29T04:18:59.690962+0000","last_fullsized":"2024-03-29T04:18:59.690962+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:21.603876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680600+0000","last_change":"2024-03-29T04:18:58.680251+0000","last_active":"2024-03-29T04:18:59.680600+0000","last_peered":"2024-03-29T04:18:59.680600+0000","last_clean":"2024-03-29T04:18:59.680600+0000","last_became_active":"2024-03-29T04:18:58.678048+0000","last_became_peered":"2024-03-29T04:18:58.678048+0000","last_unstale":"2024-03-29T04:18:59.680600+0000","last_undegraded":"2024-03-29T04:18:59.680600+0000","last_fullsized":"2024-03-29T04:18:59.680600+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:29.155520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679320+0000","last_change":"2024-03-29T04:18:57.650361+0000","last_active":"2024-03-29T04:18:59.679320+0000","last_peered":"2024-03-29T04:18:59.679320+0000","last_clean":"2024-03-29T04:18:59.679320+0000","last_became_active":"2024-03-29T04:18:57.650006+0000","last_became_peered":"2024-03-29T04:18:57.650006+0000","last_unstale":"2024-03-29T04:18:59.679320+0000","last_undegraded":"2024-03-29T04:18:59.679320+0000","last_fullsized":"2024-03-29T04:18:59.679320+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:35:32.285417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679695+0000","last_change":"2024-03-29T04:18:58.685094+0000","last_active":"2024-03-29T04:18:59.679695+0000","last_peered":"2024-03-29T04:18:59.679695+0000","last_clean":"2024-03-29T04:18:59.679695+0000","last_became_active":"2024-03-29T04:18:58.684840+0000","last_became_peered":"2024-03-29T04:18:58.684840+0000","last_unstale":"2024-03-29T04:18:59.679695+0000","last_undegraded":"2024-03-29T04:18:59.679695+0000","last_fullsized":"2024-03-29T04:18:59.679695+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:04.997398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679915+0000","last_change":"2024-03-29T04:18:57.633188+0000","last_active":"2024-03-29T04:18:59.679915+0000","last_peered":"2024-03-29T04:18:59.679915+0000","last_clean":"2024-03-29T04:18:59.679915+0000","last_became_active":"2024-03-29T04:18:57.632810+0000","last_became_peered":"2024-03-29T04:18:57.632810+0000","last_unstale":"2024-03-29T04:18:59.679915+0000","last_undegraded":"2024-03-29T04:18:59.679915+0000","last_fullsized":"2024-03-29T04:18:59.679915+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:03.989103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679531+0000","last_change":"2024-03-29T04:18:58.658312+0000","last_active":"2024-03-29T04:18:59.679531+0000","last_peered":"2024-03-29T04:18:59.679531+0000","last_clean":"2024-03-29T04:18:59.679531+0000","last_became_active":"2024-03-29T04:18:58.657363+0000","last_became_peered":"2024-03-29T04:18:58.657363+0000","last_unstale":"2024-03-29T04:18:59.679531+0000","last_undegraded":"2024-03-29T04:18:59.679531+0000","last_fullsized":"2024-03-29T04:18:59.679531+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:33:30.490101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:00.276582+0000","last_change":"2024-03-29T04:18:57.666809+0000","last_active":"2024-03-29T04:19:00.276582+0000","last_peered":"2024-03-29T04:19:00.276582+0000","last_clean":"2024-03-29T04:19:00.276582+0000","last_became_active":"2024-03-29T04:18:57.666382+0000","last_became_peered":"2024-03-29T04:18:57.666382+0000","last_unstale":"2024-03-29T04:19:00.276582+0000","last_undegraded":"2024-03-29T04:19:00.276582+0000","last_fullsized":"2024-03-29T04:19:00.276582+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:51.553019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.189095+0000","last_change":"2024-03-29T04:18:58.698349+0000","last_active":"2024-03-29T04:19:01.189095+0000","last_peered":"2024-03-29T04:19:01.189095+0000","last_clean":"2024-03-29T04:19:01.189095+0000","last_became_active":"2024-03-29T04:18:58.688432+0000","last_became_peered":"2024-03-29T04:18:58.688432+0000","last_unstale":"2024-03-29T04:19:01.189095+0000","last_undegraded":"2024-03-29T04:19:01.189095+0000","last_fullsized":"2024-03-29T04:19:01.189095+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:39:47.279731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.189137+0000","last_change":"2024-03-29T04:18:57.653609+0000","last_active":"2024-03-29T04:19:01.189137+0000","last_peered":"2024-03-29T04:19:01.189137+0000","last_clean":"2024-03-29T04:19:01.189137+0000","last_became_active":"2024-03-29T04:18:57.653049+0000","last_became_peered":"2024-03-29T04:18:57.653049+0000","last_unstale":"2024-03-29T04:19:01.189137+0000","last_undegraded":"2024-03-29T04:19:01.189137+0000","last_fullsized":"2024-03-29T04:19:01.189137+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:57.166873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679680+0000","last_change":"2024-03-29T04:18:58.666495+0000","last_active":"2024-03-29T04:18:59.679680+0000","last_peered":"2024-03-29T04:18:59.679680+0000","last_clean":"2024-03-29T04:18:59.679680+0000","last_became_active":"2024-03-29T04:18:58.665812+0000","last_became_peered":"2024-03-29T04:18:58.665812+0000","last_unstale":"2024-03-29T04:18:59.679680+0000","last_undegraded":"2024-03-29T04:18:59.679680+0000","last_fullsized":"2024-03-29T04:18:59.679680+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:03:55.569008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.188840+0000","last_change":"2024-03-29T04:18:57.664980+0000","last_active":"2024-03-29T04:19:01.188840+0000","last_peered":"2024-03-29T04:19:01.188840+0000","last_clean":"2024-03-29T04:19:01.188840+0000","last_became_active":"2024-03-29T04:18:57.664377+0000","last_became_peered":"2024-03-29T04:18:57.664377+0000","last_unstale":"2024-03-29T04:19:01.188840+0000","last_undegraded":"2024-03-29T04:19:01.188840+0000","last_fullsized":"2024-03-29T04:19:01.188840+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:56:06.055281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679320+0000","last_change":"2024-03-29T04:18:58.663161+0000","last_active":"2024-03-29T04:18:59.679320+0000","last_peered":"2024-03-29T04:18:59.679320+0000","last_clean":"2024-03-29T04:18:59.679320+0000","last_became_active":"2024-03-29T04:18:58.662297+0000","last_became_peered":"2024-03-29T04:18:58.662297+0000","last_unstale":"2024-03-29T04:18:59.679320+0000","last_undegraded":"2024-03-29T04:18:59.679320+0000","last_fullsized":"2024-03-29T04:18:59.679320+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:06:56.949031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690646+0000","last_change":"2024-03-29T04:18:57.666734+0000","last_active":"2024-03-29T04:18:59.690646+0000","last_peered":"2024-03-29T04:18:59.690646+0000","last_clean":"2024-03-29T04:18:59.690646+0000","last_became_active":"2024-03-29T04:18:57.666091+0000","last_became_peered":"2024-03-29T04:18:57.666091+0000","last_unstale":"2024-03-29T04:18:59.690646+0000","last_undegraded":"2024-03-29T04:18:59.690646+0000","last_fullsized":"2024-03-29T04:18:59.690646+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:53:46.438562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690718+0000","last_change":"2024-03-29T04:18:58.702282+0000","last_active":"2024-03-29T04:18:59.690718+0000","last_peered":"2024-03-29T04:18:59.690718+0000","last_clean":"2024-03-29T04:18:59.690718+0000","last_became_active":"2024-03-29T04:18:58.695850+0000","last_became_peered":"2024-03-29T04:18:58.695850+0000","last_unstale":"2024-03-29T04:18:59.690718+0000","last_undegraded":"2024-03-29T04:18:59.690718+0000","last_fullsized":"2024-03-29T04:18:59.690718+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:02:00.617566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680948+0000","last_change":"2024-03-29T04:18:57.666956+0000","last_active":"2024-03-29T04:18:59.680948+0000","last_peered":"2024-03-29T04:18:59.680948+0000","last_clean":"2024-03-29T04:18:59.680948+0000","last_became_active":"2024-03-29T04:18:57.666383+0000","last_became_peered":"2024-03-29T04:18:57.666383+0000","last_unstale":"2024-03-29T04:18:59.680948+0000","last_undegraded":"2024-03-29T04:18:59.680948+0000","last_fullsized":"2024-03-29T04:18:59.680948+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:46:56.030179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.681319+0000","last_change":"2024-03-29T04:18:58.665254+0000","last_active":"2024-03-29T04:18:59.681319+0000","last_peered":"2024-03-29T04:18:59.681319+0000","last_clean":"2024-03-29T04:18:59.681319+0000","last_became_active":"2024-03-29T04:18:58.664330+0000","last_became_peered":"2024-03-29T04:18:58.664330+0000","last_unstale":"2024-03-29T04:18:59.681319+0000","last_undegraded":"2024-03-29T04:18:59.681319+0000","last_fullsized":"2024-03-29T04:18:59.681319+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:08:09.926082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685328+0000","last_change":"2024-03-29T04:18:57.649911+0000","last_active":"2024-03-29T04:18:59.685328+0000","last_peered":"2024-03-29T04:18:59.685328+0000","last_clean":"2024-03-29T04:18:59.685328+0000","last_became_active":"2024-03-29T04:18:57.649663+0000","last_became_peered":"2024-03-29T04:18:57.649663+0000","last_unstale":"2024-03-29T04:18:59.685328+0000","last_undegraded":"2024-03-29T04:18:59.685328+0000","last_fullsized":"2024-03-29T04:18:59.685328+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:43.915489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685432+0000","last_change":"2024-03-29T04:18:58.679364+0000","last_active":"2024-03-29T04:18:59.685432+0000","last_peered":"2024-03-29T04:18:59.685432+0000","last_clean":"2024-03-29T04:18:59.685432+0000","last_became_active":"2024-03-29T04:18:58.679008+0000","last_became_peered":"2024-03-29T04:18:58.679008+0000","last_unstale":"2024-03-29T04:18:59.685432+0000","last_undegraded":"2024-03-29T04:18:59.685432+0000","last_fullsized":"2024-03-29T04:18:59.685432+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:18:31.187655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:00.276983+0000","last_change":"2024-03-29T04:18:57.654452+0000","last_active":"2024-03-29T04:19:00.276983+0000","last_peered":"2024-03-29T04:19:00.276983+0000","last_clean":"2024-03-29T04:19:00.276983+0000","last_became_active":"2024-03-29T04:18:57.654084+0000","last_became_peered":"2024-03-29T04:18:57.654084+0000","last_unstale":"2024-03-29T04:19:00.276983+0000","last_undegraded":"2024-03-29T04:19:00.276983+0000","last_fullsized":"2024-03-29T04:19:00.276983+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:46:56.030179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.189288+0000","last_change":"2024-03-29T04:18:58.679822+0000","last_active":"2024-03-29T04:19:01.189288+0000","last_peered":"2024-03-29T04:19:01.189288+0000","last_clean":"2024-03-29T04:19:01.189288+0000","last_became_active":"2024-03-29T04:18:58.679080+0000","last_became_peered":"2024-03-29T04:18:58.679080+0000","last_unstale":"2024-03-29T04:19:01.189288+0000","last_undegraded":"2024-03-29T04:19:01.189288+0000","last_fullsized":"2024-03-29T04:19:01.189288+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:22.612171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685131+0000","last_change":"2024-03-29T04:18:57.633652+0000","last_active":"2024-03-29T04:18:59.685131+0000","last_peered":"2024-03-29T04:18:59.685131+0000","last_clean":"2024-03-29T04:18:59.685131+0000","last_became_active":"2024-03-29T04:18:57.633156+0000","last_became_peered":"2024-03-29T04:18:57.633156+0000","last_unstale":"2024-03-29T04:18:59.685131+0000","last_undegraded":"2024-03-29T04:18:59.685131+0000","last_fullsized":"2024-03-29T04:18:59.685131+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:59:31.934017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680416+0000","last_change":"2024-03-29T04:18:58.699660+0000","last_active":"2024-03-29T04:18:59.680416+0000","last_peered":"2024-03-29T04:18:59.680416+0000","last_clean":"2024-03-29T04:18:59.680416+0000","last_became_active":"2024-03-29T04:18:58.699390+0000","last_became_peered":"2024-03-29T04:18:58.699390+0000","last_unstale":"2024-03-29T04:18:59.680416+0000","last_undegraded":"2024-03-29T04:18:59.680416+0000","last_fullsized":"2024-03-29T04:18:59.680416+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:22:11.812410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685123+0000","last_change":"2024-03-29T04:18:57.663600+0000","last_active":"2024-03-29T04:18:59.685123+0000","last_peered":"2024-03-29T04:18:59.685123+0000","last_clean":"2024-03-29T04:18:59.685123+0000","last_became_active":"2024-03-29T04:18:57.663253+0000","last_became_peered":"2024-03-29T04:18:57.663253+0000","last_unstale":"2024-03-29T04:18:59.685123+0000","last_undegraded":"2024-03-29T04:18:59.685123+0000","last_fullsized":"2024-03-29T04:18:59.685123+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:25:11.876318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.689912+0000","last_change":"2024-03-29T04:18:58.670617+0000","last_active":"2024-03-29T04:18:59.689912+0000","last_peered":"2024-03-29T04:18:59.689912+0000","last_clean":"2024-03-29T04:18:59.689912+0000","last_became_active":"2024-03-29T04:18:58.669817+0000","last_became_peered":"2024-03-29T04:18:58.669817+0000","last_unstale":"2024-03-29T04:18:59.689912+0000","last_undegraded":"2024-03-29T04:18:59.689912+0000","last_fullsized":"2024-03-29T04:18:59.689912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:09:58.485244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.189518+0000","last_change":"2024-03-29T04:18:57.647273+0000","last_active":"2024-03-29T04:19:01.189518+0000","last_peered":"2024-03-29T04:19:01.189518+0000","last_clean":"2024-03-29T04:19:01.189518+0000","last_became_active":"2024-03-29T04:18:57.646106+0000","last_became_peered":"2024-03-29T04:18:57.646106+0000","last_unstale":"2024-03-29T04:19:01.189518+0000","last_undegraded":"2024-03-29T04:19:01.189518+0000","last_fullsized":"2024-03-29T04:19:01.189518+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:32.935677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679590+0000","last_change":"2024-03-29T04:18:58.668993+0000","last_active":"2024-03-29T04:18:59.679590+0000","last_peered":"2024-03-29T04:18:59.679590+0000","last_clean":"2024-03-29T04:18:59.679590+0000","last_became_active":"2024-03-29T04:18:58.667339+0000","last_became_peered":"2024-03-29T04:18:58.667339+0000","last_unstale":"2024-03-29T04:18:59.679590+0000","last_undegraded":"2024-03-29T04:18:59.679590+0000","last_fullsized":"2024-03-29T04:18:59.679590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:36:33.283150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685589+0000","last_change":"2024-03-29T04:18:57.633301+0000","last_active":"2024-03-29T04:18:59.685589+0000","last_peered":"2024-03-29T04:18:59.685589+0000","last_clean":"2024-03-29T04:18:59.685589+0000","last_became_active":"2024-03-29T04:18:57.633009+0000","last_became_peered":"2024-03-29T04:18:57.633009+0000","last_unstale":"2024-03-29T04:18:59.685589+0000","last_undegraded":"2024-03-29T04:18:59.685589+0000","last_fullsized":"2024-03-29T04:18:59.685589+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:26:20.759660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:00.277026+0000","last_change":"2024-03-29T04:18:58.708059+0000","last_active":"2024-03-29T04:19:00.277026+0000","last_peered":"2024-03-29T04:19:00.277026+0000","last_clean":"2024-03-29T04:19:00.277026+0000","last_became_active":"2024-03-29T04:18:58.707750+0000","last_became_peered":"2024-03-29T04:18:58.707750+0000","last_unstale":"2024-03-29T04:19:00.277026+0000","last_undegraded":"2024-03-29T04:19:00.277026+0000","last_fullsized":"2024-03-29T04:19:00.277026+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:29.155520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690919+0000","last_change":"2024-03-29T04:18:57.660454+0000","last_active":"2024-03-29T04:18:59.690919+0000","last_peered":"2024-03-29T04:18:59.690919+0000","last_clean":"2024-03-29T04:18:59.690919+0000","last_became_active":"2024-03-29T04:18:57.650654+0000","last_became_peered":"2024-03-29T04:18:57.650654+0000","last_unstale":"2024-03-29T04:18:59.690919+0000","last_undegraded":"2024-03-29T04:18:59.690919+0000","last_fullsized":"2024-03-29T04:18:59.690919+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:03.989103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.190373+0000","last_change":"2024-03-29T04:18:58.693884+0000","last_active":"2024-03-29T04:19:01.190373+0000","last_peered":"2024-03-29T04:19:01.190373+0000","last_clean":"2024-03-29T04:19:01.190373+0000","last_became_active":"2024-03-29T04:18:58.693143+0000","last_became_peered":"2024-03-29T04:18:58.693143+0000","last_unstale":"2024-03-29T04:19:01.190373+0000","last_undegraded":"2024-03-29T04:19:01.190373+0000","last_fullsized":"2024-03-29T04:19:01.190373+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:27:32.000679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.685656+0000","last_change":"2024-03-29T04:18:57.662501+0000","last_active":"2024-03-29T04:18:59.685656+0000","last_peered":"2024-03-29T04:18:59.685656+0000","last_clean":"2024-03-29T04:18:59.685656+0000","last_became_active":"2024-03-29T04:18:57.662145+0000","last_became_peered":"2024-03-29T04:18:57.662145+0000","last_unstale":"2024-03-29T04:18:59.685656+0000","last_undegraded":"2024-03-29T04:18:59.685656+0000","last_fullsized":"2024-03-29T04:18:59.685656+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:10:06.936644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679865+0000","last_change":"2024-03-29T04:18:58.673901+0000","last_active":"2024-03-29T04:18:59.679865+0000","last_peered":"2024-03-29T04:18:59.679865+0000","last_clean":"2024-03-29T04:18:59.679865+0000","last_became_active":"2024-03-29T04:18:58.673075+0000","last_became_peered":"2024-03-29T04:18:58.673075+0000","last_unstale":"2024-03-29T04:18:59.679865+0000","last_undegraded":"2024-03-29T04:18:59.679865+0000","last_fullsized":"2024-03-29T04:18:59.679865+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:30:41.634747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680393+0000","last_change":"2024-03-29T04:18:57.635473+0000","last_active":"2024-03-29T04:18:59.680393+0000","last_peered":"2024-03-29T04:18:59.680393+0000","last_clean":"2024-03-29T04:18:59.680393+0000","last_became_active":"2024-03-29T04:18:57.635129+0000","last_became_peered":"2024-03-29T04:18:57.635129+0000","last_unstale":"2024-03-29T04:18:59.680393+0000","last_undegraded":"2024-03-29T04:18:59.680393+0000","last_fullsized":"2024-03-29T04:18:59.680393+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:08:44.644431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679832+0000","last_change":"2024-03-29T04:18:58.663702+0000","last_active":"2024-03-29T04:18:59.679832+0000","last_peered":"2024-03-29T04:18:59.679832+0000","last_clean":"2024-03-29T04:18:59.679832+0000","last_became_active":"2024-03-29T04:18:58.662253+0000","last_became_peered":"2024-03-29T04:18:58.662253+0000","last_unstale":"2024-03-29T04:18:59.679832+0000","last_undegraded":"2024-03-29T04:18:59.679832+0000","last_fullsized":"2024-03-29T04:18:59.679832+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:39:47.279731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680436+0000","last_change":"2024-03-29T04:18:57.641648+0000","last_active":"2024-03-29T04:18:59.680436+0000","last_peered":"2024-03-29T04:18:59.680436+0000","last_clean":"2024-03-29T04:18:59.680436+0000","last_became_active":"2024-03-29T04:18:57.641274+0000","last_became_peered":"2024-03-29T04:18:57.641274+0000","last_unstale":"2024-03-29T04:18:59.680436+0000","last_undegraded":"2024-03-29T04:18:59.680436+0000","last_fullsized":"2024-03-29T04:18:59.680436+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:51.553019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:00.276577+0000","last_change":"2024-03-29T04:18:58.672593+0000","last_active":"2024-03-29T04:19:00.276577+0000","last_peered":"2024-03-29T04:19:00.276577+0000","last_clean":"2024-03-29T04:19:00.276577+0000","last_became_active":"2024-03-29T04:18:58.672222+0000","last_became_peered":"2024-03-29T04:18:58.672222+0000","last_unstale":"2024-03-29T04:19:00.276577+0000","last_undegraded":"2024-03-29T04:19:00.276577+0000","last_fullsized":"2024-03-29T04:19:00.276577+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:04.997398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.190132+0000","last_change":"2024-03-29T04:18:57.636067+0000","last_active":"2024-03-29T04:19:01.190132+0000","last_peered":"2024-03-29T04:19:01.190132+0000","last_clean":"2024-03-29T04:19:01.190132+0000","last_became_active":"2024-03-29T04:18:57.635270+0000","last_became_peered":"2024-03-29T04:18:57.635270+0000","last_unstale":"2024-03-29T04:19:01.190132+0000","last_undegraded":"2024-03-29T04:19:01.190132+0000","last_fullsized":"2024-03-29T04:19:01.190132+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:18:30.179360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.680272+0000","last_change":"2024-03-29T04:18:58.658440+0000","last_active":"2024-03-29T04:18:59.680272+0000","last_peered":"2024-03-29T04:18:59.680272+0000","last_clean":"2024-03-29T04:18:59.680272+0000","last_became_active":"2024-03-29T04:18:58.657732+0000","last_became_peered":"2024-03-29T04:18:58.657732+0000","last_unstale":"2024-03-29T04:18:59.680272+0000","last_undegraded":"2024-03-29T04:18:59.680272+0000","last_fullsized":"2024-03-29T04:18:59.680272+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:15:56.537650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.679883+0000","last_change":"2024-03-29T04:18:57.643649+0000","last_active":"2024-03-29T04:18:59.679883+0000","last_peered":"2024-03-29T04:18:59.679883+0000","last_clean":"2024-03-29T04:18:59.679883+0000","last_became_active":"2024-03-29T04:18:57.642751+0000","last_became_peered":"2024-03-29T04:18:57.642751+0000","last_unstale":"2024-03-29T04:18:59.679883+0000","last_undegraded":"2024-03-29T04:18:59.679883+0000","last_fullsized":"2024-03-29T04:18:59.679883+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:16:07.746204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.189874+0000","last_change":"2024-03-29T04:18:58.699368+0000","last_active":"2024-03-29T04:19:01.189874+0000","last_peered":"2024-03-29T04:19:01.189874+0000","last_clean":"2024-03-29T04:19:01.189874+0000","last_became_active":"2024-03-29T04:18:58.698691+0000","last_became_peered":"2024-03-29T04:18:58.698691+0000","last_unstale":"2024-03-29T04:19:01.189874+0000","last_undegraded":"2024-03-29T04:19:01.189874+0000","last_fullsized":"2024-03-29T04:19:01.189874+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:08:09.926082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.678997+0000","last_change":"2024-03-29T04:18:57.633269+0000","last_active":"2024-03-29T04:18:59.678997+0000","last_peered":"2024-03-29T04:18:59.678997+0000","last_clean":"2024-03-29T04:18:59.678997+0000","last_became_active":"2024-03-29T04:18:57.632982+0000","last_became_peered":"2024-03-29T04:18:57.632982+0000","last_unstale":"2024-03-29T04:18:59.678997+0000","last_undegraded":"2024-03-29T04:18:59.678997+0000","last_fullsized":"2024-03-29T04:18:59.678997+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:53:46.438562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:18:59.690328+0000","last_change":"2024-03-29T04:18:58.706519+0000","last_active":"2024-03-29T04:18:59.690328+0000","last_peered":"2024-03-29T04:18:59.690328+0000","last_clean":"2024-03-29T04:18:59.690328+0000","last_became_active":"2024-03-29T04:18:58.702146+0000","last_became_peered":"2024-03-29T04:18:58.702146+0000","last_unstale":"2024-03-29T04:18:59.690328+0000","last_undegraded":"2024-03-29T04:18:59.690328+0000","last_fullsized":"2024-03-29T04:18:59.690328+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:57.609853+0000","last_clean_scrub_stamp":"2024-03-29T04:18:57.609853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:23:56.383735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.190126+0000","last_change":"2024-03-29T04:18:57.655959+0000","last_active":"2024-03-29T04:19:01.190126+0000","last_peered":"2024-03-29T04:19:01.190126+0000","last_clean":"2024-03-29T04:19:01.190126+0000","last_became_active":"2024-03-29T04:18:57.651470+0000","last_became_peered":"2024-03-29T04:18:57.651470+0000","last_unstale":"2024-03-29T04:19:01.190126+0000","last_undegraded":"2024-03-29T04:19:01.190126+0000","last_fullsized":"2024-03-29T04:19:01.190126+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:18:56.601558+0000","last_clean_scrub_stamp":"2024-03-29T04:18:56.601558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:20:51.553019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":206,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:19:01.189517+0000","last_change":"2024-03-29T04:17:25.361826+0000","last_active":"2024-03-29T04:19:01.189517+0000","last_peered":"2024-03-29T04:19:01.189517+0000","last_clean":"2024-03-29T04:19:01.189517+0000","last_became_active":"2024-03-29T04:17:25.361259+0000","last_became_peered":"2024-03-29T04:17:25.361259+0000","last_unstale":"2024-03-29T04:19:01.189517+0000","last_undegraded":"2024-03-29T04:19:01.189517+0000","last_fullsized":"2024-03-29T04:19:01.189517+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_clean_scrub_stamp":"2024-03-29T04:17:24.334639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:40.027873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1073152,"data_stored":793886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":202418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":201948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":202440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31648,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340192,"statfs":{"total":96636764160,"available":96604356608,"internally_reserved":0,"allocated":479232,"data_stored":203482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":466944,"data_stored":201906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31636,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":466944,"data_stored":201926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1060864,"data_stored":792238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:19:07.588 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-29T04:19:07.589 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-29T04:19:07.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.779+0000 7fe242c83640 1 Processor -- start 2024-03-29T04:19:07.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.779+0000 7fe242c83640 1 -- start start 2024-03-29T04:19:07.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.779+0000 7fe242c83640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe23c0b3790 0x7fe23c0b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:07.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.779+0000 7fe242c83640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe23c06db20 0x7fe23c06df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:07.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.779+0000 7fe242c83640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe23c06bd70 con 0x7fe23c0b4480 2024-03-29T04:19:07.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.779+0000 7fe242c83640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fe23c0b40b0 con 0x7fe23c0b3790 2024-03-29T04:19:07.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.779+0000 7fe242c83640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe23c06c000 con 0x7fe23c06db20 2024-03-29T04:19:07.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.783+0000 7fe240a1f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe23c0b3790 0x7fe23c0b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:07.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.783+0000 7fe23bfff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe23c0b4480 0x7fe23c06b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57036/0 (socket says 172.21.15.22:57036) 2024-03-29T04:19:07.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.783+0000 7fe23bfff640 1 -- 172.21.15.22:0/3059962791 learned_addr learned my addr 172.21.15.22:0/3059962791 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:19:07.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.783+0000 7fe241220640 1 --2- 172.21.15.22:0/3059962791 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe23c06db20 0x7fe23c06df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:07.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.783+0000 7fe240a1f640 1 -- 172.21.15.22:0/3059962791 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe23c06db20 msgr2=0x7fe23c06df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:07.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.783+0000 7fe240a1f640 1 --2- 172.21.15.22:0/3059962791 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe23c06db20 0x7fe23c06df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:07.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.783+0000 7fe240a1f640 1 -- 172.21.15.22:0/3059962791 >> v1:172.21.15.22:6789/0 conn(0x7fe23c0b4480 legacy=0x7fe23c06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:07.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.783+0000 7fe240a1f640 1 -- 172.21.15.22:0/3059962791 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23c06c2e0 con 0x7fe23c0b3790 2024-03-29T04:19:07.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.783+0000 7fe240a1f640 1 --2- 172.21.15.22:0/3059962791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe23c0b3790 0x7fe23c0b3b70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe2300097c0 tx=0x7fe23001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81003c726939c49 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:07.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.783+0000 7fe23b7fe640 1 -- 172.21.15.22:0/3059962791 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2300a7070 con 0x7fe23c0b3790 2024-03-29T04:19:07.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.783+0000 7fe23b7fe640 1 -- 172.21.15.22:0/3059962791 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2300b0070 con 0x7fe23c0b3790 2024-03-29T04:19:07.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.783+0000 7fe23b7fe640 1 -- 172.21.15.22:0/3059962791 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2300ab890 con 0x7fe23c0b3790 2024-03-29T04:19:07.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.783+0000 7fe242c83640 1 -- 172.21.15.22:0/3059962791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe23c0b3790 msgr2=0x7fe23c0b3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:07.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.783+0000 7fe242c83640 1 --2- 172.21.15.22:0/3059962791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe23c0b3790 0x7fe23c0b3b70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe2300097c0 tx=0x7fe23001c470 comp rx=0 tx=0).stop 2024-03-29T04:19:07.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.783+0000 7fe242c83640 1 -- 172.21.15.22:0/3059962791 shutdown_connections 2024-03-29T04:19:07.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.783+0000 7fe242c83640 1 --2- 172.21.15.22:0/3059962791 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe23c06db20 0x7fe23c06df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:07.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.783+0000 7fe242c83640 1 --2- 172.21.15.22:0/3059962791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe23c0b3790 0x7fe23c0b3b70 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:07.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.783+0000 7fe242c83640 1 -- 172.21.15.22:0/3059962791 >> 172.21.15.22:0/3059962791 conn(0x7fe23c0d0760 msgr2=0x7fe23c0d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:07.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.783+0000 7fe242c83640 1 -- 172.21.15.22:0/3059962791 shutdown_connections 2024-03-29T04:19:07.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.783+0000 7fe242c83640 1 -- 172.21.15.22:0/3059962791 wait complete. 2024-03-29T04:19:07.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe242c83640 1 Processor -- start 2024-03-29T04:19:07.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe242c83640 1 -- start start 2024-03-29T04:19:07.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe242c83640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe23c06db20 0x7fe23c13e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:07.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe242c83640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe23c0b3790 0x7fe23c157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:07.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe242c83640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe23c0b4480 0x7fe23c0b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:07.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe242c83640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c06fbb0 con 0x7fe23c0b4480 2024-03-29T04:19:07.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe242c83640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c06fa30 con 0x7fe23c06db20 2024-03-29T04:19:07.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe242c83640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe23c16b300 con 0x7fe23c0b3790 2024-03-29T04:19:07.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe241220640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe23c0b4480 0x7fe23c0b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:07.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe241220640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe23c0b4480 0x7fe23c0b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35870/0 (socket says 172.21.15.22:35870) 2024-03-29T04:19:07.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe241220640 1 -- 172.21.15.22:0/139019913 learned_addr learned my addr 172.21.15.22:0/139019913 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:07.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe240a1f640 1 --2- 172.21.15.22:0/139019913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe23c06db20 0x7fe23c13e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:07.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe23bfff640 1 --2- 172.21.15.22:0/139019913 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe23c0b3790 0x7fe23c157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:07.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe241220640 1 -- 172.21.15.22:0/139019913 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe23c0b3790 msgr2=0x7fe23c157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:07.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe241220640 1 --2- 172.21.15.22:0/139019913 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe23c0b3790 0x7fe23c157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:07.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe241220640 1 -- 172.21.15.22:0/139019913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe23c06db20 msgr2=0x7fe23c13e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:07.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe241220640 1 --2- 172.21.15.22:0/139019913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe23c06db20 0x7fe23c13e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:07.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe241220640 1 -- 172.21.15.22:0/139019913 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23c157a80 con 0x7fe23c0b4480 2024-03-29T04:19:07.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe23bfff640 1 --2- 172.21.15.22:0/139019913 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe23c0b3790 0x7fe23c157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:19:07.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe241220640 1 --2- 172.21.15.22:0/139019913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe23c0b4480 0x7fe23c0b14c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe234007e40 tx=0x7fe23400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:07.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe2397fa640 1 -- 172.21.15.22:0/139019913 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe234014620 con 0x7fe23c0b4480 2024-03-29T04:19:07.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe2397fa640 1 -- 172.21.15.22:0/139019913 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe234019070 con 0x7fe23c0b4480 2024-03-29T04:19:07.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe242c83640 1 -- 172.21.15.22:0/139019913 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe23c0b1a00 con 0x7fe23c0b4480 2024-03-29T04:19:07.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe2397fa640 1 -- 172.21.15.22:0/139019913 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe234014a20 con 0x7fe23c0b4480 2024-03-29T04:19:07.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe242c83640 1 -- 172.21.15.22:0/139019913 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe23c0b1f80 con 0x7fe23c0b4480 2024-03-29T04:19:07.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.787+0000 7fe242c83640 1 -- 172.21.15.22:0/139019913 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe23c068900 con 0x7fe23c0b4480 2024-03-29T04:19:07.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.791+0000 7fe2397fa640 1 -- 172.21.15.22:0/139019913 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe234014bc0 con 0x7fe23c0b4480 2024-03-29T04:19:07.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.791+0000 7fe2397fa640 1 --2- 172.21.15.22:0/139019913 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe220067ff0 0x7fe22006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:07.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.791+0000 7fe2397fa640 1 -- 172.21.15.22:0/139019913 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fe234086e10 con 0x7fe23c0b4480 2024-03-29T04:19:07.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.791+0000 7fe240a1f640 1 --2- 172.21.15.22:0/139019913 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe220067ff0 0x7fe22006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:07.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.791+0000 7fe240a1f640 1 --2- 172.21.15.22:0/139019913 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe220067ff0 0x7fe22006a4b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe2300af000 tx=0x7fe2300a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:07.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:07.795+0000 7fe2397fa640 1 -- 172.21.15.22:0/139019913 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fe2340574c0 con 0x7fe23c0b4480 2024-03-29T04:19:07.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.015+0000 7fe242c83640 1 -- 172.21.15.22:0/139019913 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe23c068230 con 0x7fe23c0b4480 2024-03-29T04:19:08.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.019+0000 7fe2397fa640 1 -- 172.21.15.22:0/139019913 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7fe234056dc0 con 0x7fe23c0b4480 2024-03-29T04:19:08.001 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2024-03-29T04:19:08.001 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQChQAZmViasLhAA6Vlzyow8ABjacgHj4nOI/A== 2024-03-29T04:19:08.001 INFO:teuthology.orchestra.run.smithi022.stdout: caps mds = "allow" 2024-03-29T04:19:08.001 INFO:teuthology.orchestra.run.smithi022.stdout: caps mon = "allow r" 2024-03-29T04:19:08.001 INFO:teuthology.orchestra.run.smithi022.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-29T04:19:08.001 INFO:teuthology.orchestra.run.smithi022.stderr:updated caps for client.0 2024-03-29T04:19:08.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.023+0000 7fe242c83640 1 -- 172.21.15.22:0/139019913 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe220067ff0 msgr2=0x7fe22006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:08.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.023+0000 7fe242c83640 1 --2- 172.21.15.22:0/139019913 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe220067ff0 0x7fe22006a4b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe2300af000 tx=0x7fe2300a7000 comp rx=0 tx=0).stop 2024-03-29T04:19:08.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.023+0000 7fe242c83640 1 -- 172.21.15.22:0/139019913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe23c0b4480 msgr2=0x7fe23c0b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:08.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.023+0000 7fe242c83640 1 --2- 172.21.15.22:0/139019913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe23c0b4480 0x7fe23c0b14c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe234007e40 tx=0x7fe23400e4b0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.023+0000 7fe242c83640 1 -- 172.21.15.22:0/139019913 shutdown_connections 2024-03-29T04:19:08.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.023+0000 7fe242c83640 1 --2- 172.21.15.22:0/139019913 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe220067ff0 0x7fe22006a4b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.023+0000 7fe242c83640 1 --2- 172.21.15.22:0/139019913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe23c0b4480 0x7fe23c0b14c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.023+0000 7fe242c83640 1 --2- 172.21.15.22:0/139019913 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe23c0b3790 0x7fe23c157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.023+0000 7fe242c83640 1 --2- 172.21.15.22:0/139019913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe23c06db20 0x7fe23c13e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.023+0000 7fe242c83640 1 -- 172.21.15.22:0/139019913 >> 172.21.15.22:0/139019913 conn(0x7fe23c0d0760 msgr2=0x7fe23c1612a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:08.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.023+0000 7fe242c83640 1 -- 172.21.15.22:0/139019913 shutdown_connections 2024-03-29T04:19:08.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.023+0000 7fe242c83640 1 -- 172.21.15.22:0/139019913 wait complete. 2024-03-29T04:19:08.020 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:19:08.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.159+0000 7f89aead6640 1 Processor -- start 2024-03-29T04:19:08.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89aead6640 1 -- start start 2024-03-29T04:19:08.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89aead6640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89a806e4f0 0x7f89a8162030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89aead6640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f89a815f710 0x7f89a8162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89aead6640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89a8162cf0 con 0x7f89a81553b0 2024-03-29T04:19:08.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89aead6640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f89a8162ab0 con 0x7f89a815f710 2024-03-29T04:19:08.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89aead6640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f89a815ec30 con 0x7f89a806e4f0 2024-03-29T04:19:08.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89ac872640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f89a81553b0 0x7f89a806dde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57050/0 (socket says 172.21.15.22:57050) 2024-03-29T04:19:08.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89ac872640 1 -- 172.21.15.22:0/2457418905 learned_addr learned my addr 172.21.15.22:0/2457418905 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:19:08.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89ad073640 1 --2- 172.21.15.22:0/2457418905 >> v2:172.21.15.151:3300/0 conn(0x7f89a815f710 0x7f89a8162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89a7fff640 1 --2- 172.21.15.22:0/2457418905 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89a806e4f0 0x7f89a8162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89a77fe640 1 -- 172.21.15.22:0/2457418905 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3252153829 0 0) 0x7f89a8162cf0 con 0x7f89a81553b0 2024-03-29T04:19:08.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89a77fe640 1 -- 172.21.15.22:0/2457418905 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8998003530 con 0x7f89a81553b0 2024-03-29T04:19:08.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89a77fe640 1 -- 172.21.15.22:0/2457418905 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1628468979 0 0) 0x7f8998003530 con 0x7f89a81553b0 2024-03-29T04:19:08.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89a77fe640 1 -- 172.21.15.22:0/2457418905 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89a806e4f0 msgr2=0x7f89a8162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:08.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89a77fe640 1 --2- 172.21.15.22:0/2457418905 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89a806e4f0 0x7f89a8162030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89a77fe640 1 -- 172.21.15.22:0/2457418905 >> v2:172.21.15.151:3300/0 conn(0x7f89a815f710 msgr2=0x7f89a8162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:08.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89a77fe640 1 --2- 172.21.15.22:0/2457418905 >> v2:172.21.15.151:3300/0 conn(0x7f89a815f710 0x7f89a8162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89a77fe640 1 -- 172.21.15.22:0/2457418905 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a815ef10 con 0x7f89a81553b0 2024-03-29T04:19:08.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89a77fe640 1 -- 172.21.15.22:0/2457418905 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f899c001680 con 0x7f89a81553b0 2024-03-29T04:19:08.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89a77fe640 1 -- 172.21.15.22:0/2457418905 >> v1:172.21.15.22:6789/0 conn(0x7f89a81553b0 legacy=0x7f89a806dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:08.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89a77fe640 1 --2- 172.21.15.22:0/2457418905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89980039c0 0x7f8998003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89a77fe640 1 --2- 172.21.15.22:0/2457418905 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89a806e4f0 0x7f8998004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89a77fe640 1 --2- 172.21.15.22:0/2457418905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8998004d50 0x7f89980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89a77fe640 1 -- 172.21.15.22:0/2457418905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f89a8162ab0 con 0x7f8998004d50 2024-03-29T04:19:08.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89a77fe640 1 -- 172.21.15.22:0/2457418905 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f89a815ec30 con 0x7f89980039c0 2024-03-29T04:19:08.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89a77fe640 1 -- 172.21.15.22:0/2457418905 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f899800b9d0 con 0x7f89a806e4f0 2024-03-29T04:19:08.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89ad073640 1 --2- 172.21.15.22:0/2457418905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8998004d50 0x7f89980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89ac872640 1 --2- 172.21.15.22:0/2457418905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89980039c0 0x7f8998003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89a7fff640 1 --2- 172.21.15.22:0/2457418905 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89a806e4f0 0x7f8998004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89ad073640 1 -- 172.21.15.22:0/2457418905 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89a806e4f0 msgr2=0x7f8998004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:08.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89ad073640 1 --2- 172.21.15.22:0/2457418905 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89a806e4f0 0x7f8998004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89ad073640 1 -- 172.21.15.22:0/2457418905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89980039c0 msgr2=0x7f8998003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:08.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89ad073640 1 --2- 172.21.15.22:0/2457418905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89980039c0 0x7f8998003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.163+0000 7f89ad073640 1 -- 172.21.15.22:0/2457418905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8998003530 con 0x7f8998004d50 2024-03-29T04:19:08.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89ad073640 1 --2- 172.21.15.22:0/2457418905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8998004d50 0x7f89980051d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009fc0 tx=0x7f89a001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13ee369aa90201e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:08.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89a77fe640 1 -- 172.21.15.22:0/2457418905 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a00b2070 con 0x7f8998004d50 2024-03-29T04:19:08.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89a77fe640 1 -- 172.21.15.22:0/2457418905 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89a00a5c90 con 0x7f8998004d50 2024-03-29T04:19:08.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89a77fe640 1 -- 172.21.15.22:0/2457418905 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0009c10 con 0x7f8998004d50 2024-03-29T04:19:08.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89aead6640 1 -- 172.21.15.22:0/2457418905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8998004d50 msgr2=0x7f89980051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:08.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89aead6640 1 --2- 172.21.15.22:0/2457418905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8998004d50 0x7f89980051d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009fc0 tx=0x7f89a001cc80 comp rx=0 tx=0).stop 2024-03-29T04:19:08.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89ad073640 1 -- 172.21.15.22:0/2457418905 reap_dead start 2024-03-29T04:19:08.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89aead6640 1 -- 172.21.15.22:0/2457418905 shutdown_connections 2024-03-29T04:19:08.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89aead6640 1 -- 172.21.15.22:0/2457418905 >> 172.21.15.22:0/2457418905 conn(0x7f89a80b3d60 msgr2=0x7f89a80b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:08.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89aead6640 1 -- 172.21.15.22:0/2457418905 shutdown_connections 2024-03-29T04:19:08.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89aead6640 1 -- 172.21.15.22:0/2457418905 wait complete. 2024-03-29T04:19:08.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89aead6640 1 Processor -- start 2024-03-29T04:19:08.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89aead6640 1 -- start start 2024-03-29T04:19:08.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89aead6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8998004d50 0x7f89a80c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89aead6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89a8141360 0x7f89a806d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89aead6640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89a806d850 0x7f89a8143620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89aead6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f89a8155910 con 0x7f8998004d50 2024-03-29T04:19:08.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89aead6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f89a8155790 con 0x7f89a8141360 2024-03-29T04:19:08.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89aead6640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f89a8162030 con 0x7f89a806d850 2024-03-29T04:19:08.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89ac872640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8998004d50 0x7f89a80c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89ac872640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8998004d50 0x7f89a80c48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35880/0 (socket says 172.21.15.22:35880) 2024-03-29T04:19:08.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89ac872640 1 -- 172.21.15.22:0/2033793327 learned_addr learned my addr 172.21.15.22:0/2033793327 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:08.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89ad073640 1 --2- 172.21.15.22:0/2033793327 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89a806d850 0x7f89a8143620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.167+0000 7f89a7fff640 1 --2- 172.21.15.22:0/2033793327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89a8141360 0x7f89a806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.171+0000 7f89ac872640 1 -- 172.21.15.22:0/2033793327 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89a806d850 msgr2=0x7f89a8143620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:08.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.171+0000 7f89ac872640 1 --2- 172.21.15.22:0/2033793327 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89a806d850 0x7f89a8143620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.171+0000 7f89ac872640 1 -- 172.21.15.22:0/2033793327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89a8141360 msgr2=0x7f89a806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:08.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.171+0000 7f89ac872640 1 --2- 172.21.15.22:0/2033793327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89a8141360 0x7f89a806d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.171+0000 7f89ac872640 1 -- 172.21.15.22:0/2033793327 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a8144350 con 0x7f8998004d50 2024-03-29T04:19:08.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.171+0000 7f89ac872640 1 --2- 172.21.15.22:0/2033793327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8998004d50 0x7f89a80c48d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f899c00e820 tx=0x7f899c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:08.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.171+0000 7f89a57fa640 1 -- 172.21.15.22:0/2033793327 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c00c8d0 con 0x7f8998004d50 2024-03-29T04:19:08.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.171+0000 7f89a57fa640 1 -- 172.21.15.22:0/2033793327 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f899c018070 con 0x7f8998004d50 2024-03-29T04:19:08.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.171+0000 7f89aead6640 1 -- 172.21.15.22:0/2033793327 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a80afe40 con 0x7f8998004d50 2024-03-29T04:19:08.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.171+0000 7f89a57fa640 1 -- 172.21.15.22:0/2033793327 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c00cc70 con 0x7f8998004d50 2024-03-29T04:19:08.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.171+0000 7f89aead6640 1 -- 172.21.15.22:0/2033793327 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a80b0400 con 0x7f8998004d50 2024-03-29T04:19:08.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.171+0000 7f89aead6640 1 -- 172.21.15.22:0/2033793327 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a8068890 con 0x7f8998004d50 2024-03-29T04:19:08.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.171+0000 7f89a57fa640 1 -- 172.21.15.22:0/2033793327 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f899c01d460 con 0x7f8998004d50 2024-03-29T04:19:08.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.171+0000 7f89a57fa640 1 --2- 172.21.15.22:0/2033793327 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f898c067fc0 0x7f898c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.171+0000 7f89a57fa640 1 -- 172.21.15.22:0/2033793327 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f899c086580 con 0x7f8998004d50 2024-03-29T04:19:08.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.171+0000 7f89a7fff640 1 --2- 172.21.15.22:0/2033793327 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f898c067fc0 0x7f898c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.171+0000 7f89a7fff640 1 --2- 172.21.15.22:0/2033793327 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f898c067fc0 0x7f898c06a480 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f89a80d0540 tx=0x7f8994008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:08.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.175+0000 7f89a57fa640 1 -- 172.21.15.22:0/2033793327 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f899c056ba0 con 0x7f8998004d50 2024-03-29T04:19:08.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.371+0000 7f89aead6640 1 -- 172.21.15.22:0/2033793327 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89a806dc30 con 0x7f8998004d50 2024-03-29T04:19:08.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.371+0000 7f89a57fa640 1 -- 172.21.15.22:0/2033793327 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5181 (secure 0 0 0) 0x7f899c0564a0 con 0x7f8998004d50 2024-03-29T04:19:08.353 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:19:08.353 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/3112156479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":3112156479},{"type":"v1","addr":"172.21.15.151:6835","nonce":3112156479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5308,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/491273202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":491273202},{"type":"v1","addr":"172.21.15.151:6837","nonce":491273202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/434199693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":434199693},{"type":"v1","addr":"172.21.15.22:6833","nonce":434199693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:18:57.990202+0000","modified":"2024-03-29T04:19:01.052542+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6835/2966923505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2966923505},{"type":"v1","addr":"172.21.15.22:6835","nonce":2966923505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529]},"id":2}]} 2024-03-29T04:19:08.353 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 18 2024-03-29T04:19:08.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.375+0000 7f89aead6640 1 -- 172.21.15.22:0/2033793327 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f898c067fc0 msgr2=0x7f898c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:08.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.375+0000 7f89aead6640 1 --2- 172.21.15.22:0/2033793327 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f898c067fc0 0x7f898c06a480 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f89a80d0540 tx=0x7f8994008040 comp rx=0 tx=0).stop 2024-03-29T04:19:08.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.375+0000 7f89aead6640 1 -- 172.21.15.22:0/2033793327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8998004d50 msgr2=0x7f89a80c48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:08.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.375+0000 7f89aead6640 1 --2- 172.21.15.22:0/2033793327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8998004d50 0x7f89a80c48d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f899c00e820 tx=0x7f899c00ece0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.375+0000 7f89aead6640 1 -- 172.21.15.22:0/2033793327 shutdown_connections 2024-03-29T04:19:08.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.375+0000 7f89aead6640 1 --2- 172.21.15.22:0/2033793327 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f89a806d850 0x7f89a8143620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.375+0000 7f89aead6640 1 --2- 172.21.15.22:0/2033793327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89a8141360 0x7f89a806d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.375+0000 7f89aead6640 1 --2- 172.21.15.22:0/2033793327 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f898c067fc0 0x7f898c06a480 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.375+0000 7f89aead6640 1 --2- 172.21.15.22:0/2033793327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8998004d50 0x7f89a80c48d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.375+0000 7f89aead6640 1 -- 172.21.15.22:0/2033793327 >> 172.21.15.22:0/2033793327 conn(0x7f89a80b3d60 msgr2=0x7f89a80cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:08.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.375+0000 7f89aead6640 1 -- 172.21.15.22:0/2033793327 shutdown_connections 2024-03-29T04:19:08.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.375+0000 7f89aead6640 1 -- 172.21.15.22:0/2033793327 wait complete. 2024-03-29T04:19:08.773 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T04:18:57.990202+0000', 'modified': '2024-03-29T04:19:01.052542+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5529}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5529': {'gid': 5529, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.22:6835/2966923505', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6834', 'nonce': 2966923505}, {'type': 'v1', 'addr': '172.21.15.22:6835', 'nonce': 2966923505}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5529, 'qdb_cluster': [5529]} 2024-03-29T04:19:08.774 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T04:19:08.774 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-29T04:19:08.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.919+0000 7fc142a8b640 1 Processor -- start 2024-03-29T04:19:08.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.919+0000 7fc142a8b640 1 -- start start 2024-03-29T04:19:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.919+0000 7fc142a8b640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fc13c070490 0x7fc13c15c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.919+0000 7fc142a8b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc13c0b5b20 0x7fc13c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.919+0000 7fc142a8b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc13c15c670 con 0x7fc13c06dcc0 2024-03-29T04:19:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.919+0000 7fc142a8b640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fc13c15edd0 con 0x7fc13c0b5b20 2024-03-29T04:19:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.919+0000 7fc142a8b640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fc13c15f0b0 con 0x7fc13c070490 2024-03-29T04:19:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.923+0000 7fc141028640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fc13c06dcc0 0x7fc13c0b3e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57054/0 (socket says 172.21.15.22:57054) 2024-03-29T04:19:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.923+0000 7fc13bfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc13c0b5b20 0x7fc13c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.923+0000 7fc141028640 1 -- 172.21.15.22:0/67552421 learned_addr learned my addr 172.21.15.22:0/67552421 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:19:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.923+0000 7fc140827640 1 --2- 172.21.15.22:0/67552421 >> v2:172.21.15.151:3300/0 conn(0x7fc13c070490 0x7fc13c15c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.923+0000 7fc13bfff640 1 -- 172.21.15.22:0/67552421 >> v2:172.21.15.151:3300/0 conn(0x7fc13c070490 msgr2=0x7fc13c15c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:08.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.923+0000 7fc13bfff640 1 --2- 172.21.15.22:0/67552421 >> v2:172.21.15.151:3300/0 conn(0x7fc13c070490 0x7fc13c15c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.923+0000 7fc13bfff640 1 -- 172.21.15.22:0/67552421 >> v1:172.21.15.22:6789/0 conn(0x7fc13c06dcc0 legacy=0x7fc13c0b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:08.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.923+0000 7fc13bfff640 1 -- 172.21.15.22:0/67552421 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13c15f390 con 0x7fc13c0b5b20 2024-03-29T04:19:08.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.923+0000 7fc13b7fe640 1 -- 172.21.15.22:0/67552421 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2999251573 0 0) 0x7fc1340014a0 con 0x7fc13c06dcc0 2024-03-29T04:19:08.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc13bfff640 1 --2- 172.21.15.22:0/67552421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc13c0b5b20 0x7fc13c06d780 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0b5af0 tx=0x7fc12801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424a24c21a3f30ac server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:08.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc13b7fe640 1 -- 172.21.15.22:0/67552421 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1280b0070 con 0x7fc13c0b5b20 2024-03-29T04:19:08.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc13b7fe640 1 -- 172.21.15.22:0/67552421 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1280ab600 con 0x7fc13c0b5b20 2024-03-29T04:19:08.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc13b7fe640 1 -- 172.21.15.22:0/67552421 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1280ab9d0 con 0x7fc13c0b5b20 2024-03-29T04:19:08.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc142a8b640 1 -- 172.21.15.22:0/67552421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc13c0b5b20 msgr2=0x7fc13c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:08.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc142a8b640 1 --2- 172.21.15.22:0/67552421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc13c0b5b20 0x7fc13c06d780 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0b5af0 tx=0x7fc12801c360 comp rx=0 tx=0).stop 2024-03-29T04:19:08.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc142a8b640 1 -- 172.21.15.22:0/67552421 shutdown_connections 2024-03-29T04:19:08.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc142a8b640 1 --2- 172.21.15.22:0/67552421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc13c0b5b20 0x7fc13c06d780 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc142a8b640 1 --2- 172.21.15.22:0/67552421 >> v2:172.21.15.151:3300/0 conn(0x7fc13c070490 0x7fc13c15c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc142a8b640 1 -- 172.21.15.22:0/67552421 >> 172.21.15.22:0/67552421 conn(0x7fc13c072500 msgr2=0x7fc13c072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:08.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc142a8b640 1 -- 172.21.15.22:0/67552421 shutdown_connections 2024-03-29T04:19:08.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc142a8b640 1 -- 172.21.15.22:0/67552421 wait complete. 2024-03-29T04:19:08.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc142a8b640 1 Processor -- start 2024-03-29T04:19:08.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc142a8b640 1 -- start start 2024-03-29T04:19:08.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc142a8b640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc13c06dcc0 0x7fc13c157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc142a8b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc13c070490 0x7fc13c158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc142a8b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc13c0b5b20 0x7fc13c150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc142a8b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c1683d0 con 0x7fc13c0b5b20 2024-03-29T04:19:08.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc142a8b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c168250 con 0x7fc13c070490 2024-03-29T04:19:08.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc142a8b640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc13c168550 con 0x7fc13c06dcc0 2024-03-29T04:19:08.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc141028640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc13c0b5b20 0x7fc13c150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc141028640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc13c0b5b20 0x7fc13c150de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35892/0 (socket says 172.21.15.22:35892) 2024-03-29T04:19:08.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc141028640 1 -- 172.21.15.22:0/646666589 learned_addr learned my addr 172.21.15.22:0/646666589 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:08.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc13bfff640 1 --2- 172.21.15.22:0/646666589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc13c070490 0x7fc13c158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.927+0000 7fc140827640 1 --2- 172.21.15.22:0/646666589 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc13c06dcc0 0x7fc13c157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.931+0000 7fc141028640 1 -- 172.21.15.22:0/646666589 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc13c06dcc0 msgr2=0x7fc13c157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:08.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.931+0000 7fc141028640 1 --2- 172.21.15.22:0/646666589 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc13c06dcc0 0x7fc13c157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.931+0000 7fc141028640 1 -- 172.21.15.22:0/646666589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc13c070490 msgr2=0x7fc13c158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:08.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.931+0000 7fc141028640 1 --2- 172.21.15.22:0/646666589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc13c070490 0x7fc13c158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.931+0000 7fc141028640 1 -- 172.21.15.22:0/646666589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13c157880 con 0x7fc13c0b5b20 2024-03-29T04:19:08.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.931+0000 7fc13bfff640 1 --2- 172.21.15.22:0/646666589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc13c070490 0x7fc13c158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:19:08.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.931+0000 7fc141028640 1 --2- 172.21.15.22:0/646666589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc13c0b5b20 0x7fc13c150de0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc13400cb20 tx=0x7fc1340021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:08.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.931+0000 7fc1397fa640 1 -- 172.21.15.22:0/646666589 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13400ac00 con 0x7fc13c0b5b20 2024-03-29T04:19:08.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.931+0000 7fc1397fa640 1 -- 172.21.15.22:0/646666589 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc13400adc0 con 0x7fc13c0b5b20 2024-03-29T04:19:08.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.931+0000 7fc142a8b640 1 -- 172.21.15.22:0/646666589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc13c157b60 con 0x7fc13c0b5b20 2024-03-29T04:19:08.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.931+0000 7fc1397fa640 1 -- 172.21.15.22:0/646666589 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc134012640 con 0x7fc13c0b5b20 2024-03-29T04:19:08.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.931+0000 7fc142a8b640 1 -- 172.21.15.22:0/646666589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc13c158120 con 0x7fc13c0b5b20 2024-03-29T04:19:08.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.931+0000 7fc1397fa640 1 -- 172.21.15.22:0/646666589 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fc1340127e0 con 0x7fc13c0b5b20 2024-03-29T04:19:08.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.931+0000 7fc1397fa640 1 --2- 172.21.15.22:0/646666589 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc120067fc0 0x7fc12006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.931+0000 7fc145c0d000 1 Processor -- start 2024-03-29T04:19:08.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.931+0000 7fc1397fa640 1 -- 172.21.15.22:0/646666589 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fc134086ce0 con 0x7fc13c0b5b20 2024-03-29T04:19:08.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.931+0000 7fc145c0d000 1 -- start start 2024-03-29T04:19:08.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.931+0000 7fc140827640 1 --2- 172.21.15.22:0/646666589 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc120067fc0 0x7fc12006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x557b328ec4a0 0x557b328ec880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b328ecdc0 0x557b3289fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328a00b0 0x557b328a0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557b326f2950 con 0x557b328a00b0 2024-03-29T04:19:08.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557b3268f990 con 0x557b328ecdc0 2024-03-29T04:19:08.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x557b326ee900 con 0x557b328ec4a0 2024-03-29T04:19:08.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc141028640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328a00b0 0x557b328a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc141028640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328a00b0 0x557b328a0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35898/0 (socket says 172.21.15.22:35898) 2024-03-29T04:19:08.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc141028640 1 -- 172.21.15.22:0/1859441119 learned_addr learned my addr 172.21.15.22:0/1859441119 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:08.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc140827640 1 --2- 172.21.15.22:0/1859441119 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x557b328ec4a0 0x557b328ec880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc13bfff640 1 --2- 172.21.15.22:0/1859441119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b328ecdc0 0x557b3289fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc141028640 1 -- 172.21.15.22:0/1859441119 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x557b328ec4a0 msgr2=0x557b328ec880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:08.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc141028640 1 --2- 172.21.15.22:0/1859441119 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x557b328ec4a0 0x557b328ec880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc141028640 1 -- 172.21.15.22:0/1859441119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b328ecdc0 msgr2=0x557b3289fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:08.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc140827640 1 --2- 172.21.15.22:0/646666589 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc120067fc0 0x7fc12006a480 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc13000b930 tx=0x7fc130008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:08.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc141028640 1 --2- 172.21.15.22:0/1859441119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b328ecdc0 0x557b3289fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc141028640 1 -- 172.21.15.22:0/1859441119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b32737100 con 0x557b328a00b0 2024-03-29T04:19:08.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc141028640 1 --2- 172.21.15.22:0/1859441119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328a00b0 0x557b328a0490 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc1340568f0 tx=0x7fc1340572f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae48f431b60ff31 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:08.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc1267fc640 1 -- 172.21.15.22:0/1859441119 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13405f7d0 con 0x557b328a00b0 2024-03-29T04:19:08.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc1267fc640 1 -- 172.21.15.22:0/1859441119 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc134054ef0 con 0x557b328a00b0 2024-03-29T04:19:08.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc1267fc640 1 -- 172.21.15.22:0/1859441119 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13405f340 con 0x557b328a00b0 2024-03-29T04:19:08.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 -- 172.21.15.22:0/1859441119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328a00b0 msgr2=0x557b328a0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:08.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 --2- 172.21.15.22:0/1859441119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328a00b0 0x557b328a0490 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc1340568f0 tx=0x7fc1340572f0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 -- 172.21.15.22:0/1859441119 shutdown_connections 2024-03-29T04:19:08.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 --2- 172.21.15.22:0/1859441119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328a00b0 0x557b328a0490 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 --2- 172.21.15.22:0/1859441119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b328ecdc0 0x557b3289fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 --2- 172.21.15.22:0/1859441119 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x557b328ec4a0 0x557b328ec880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 -- 172.21.15.22:0/1859441119 >> 172.21.15.22:0/1859441119 conn(0x557b3278ab80 msgr2=0x557b328cf460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:08.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 -- 172.21.15.22:0/1859441119 shutdown_connections 2024-03-29T04:19:08.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 -- 172.21.15.22:0/1859441119 wait complete. 2024-03-29T04:19:08.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 -- start start 2024-03-29T04:19:08.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 Processor -- start 2024-03-29T04:19:08.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b328a00b0 0x557b328a7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328ec4a0 0x557b328d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x557b328ecdc0 0x557b328d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557b3268a7b0 con 0x557b328ec4a0 2024-03-29T04:19:08.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557b326b03e0 con 0x557b328a00b0 2024-03-29T04:19:08.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x557b326f6f80 con 0x557b328ecdc0 2024-03-29T04:19:08.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc145c0d000 10 client.0 _ll_register_callbacks cb 0x557b327530f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:19:08.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc13bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328ec4a0 0x557b328d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc13bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328ec4a0 0x557b328d0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35902/0 (socket says 172.21.15.22:35902) 2024-03-29T04:19:08.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc13bfff640 1 -- 172.21.15.22:0/1198021333 learned_addr learned my addr 172.21.15.22:0/1198021333 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:08.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc140827640 1 --2- 172.21.15.22:0/1198021333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b328a00b0 0x557b328a7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.935+0000 7fc141028640 1 --2- 172.21.15.22:0/1198021333 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x557b328ecdc0 0x557b328d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc13bfff640 1 -- 172.21.15.22:0/1198021333 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x557b328ecdc0 msgr2=0x557b328d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:08.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc13bfff640 1 --2- 172.21.15.22:0/1198021333 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x557b328ecdc0 0x557b328d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc13bfff640 1 -- 172.21.15.22:0/1198021333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b328a00b0 msgr2=0x557b328a7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:08.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc13bfff640 1 --2- 172.21.15.22:0/1198021333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b328a00b0 0x557b328a7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc13bfff640 1 -- 172.21.15.22:0/1198021333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b326bc8c0 con 0x557b328ec4a0 2024-03-29T04:19:08.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc13bfff640 1 --2- 172.21.15.22:0/1198021333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328ec4a0 0x557b328d0690 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc128019f90 tx=0x7fc12801aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892fccfa1db78687 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:08.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc11b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-29T04:19:08.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc11b7fe640 1 -- 172.21.15.22:0/1198021333 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1280b0040 con 0x557b328ec4a0 2024-03-29T04:19:08.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc145c0d000 1 -- 172.21.15.22:0/1198021333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc134054ef0 con 0x557b328ec4a0 2024-03-29T04:19:08.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc11b7fe640 1 -- 172.21.15.22:0/1198021333 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fc1280abcb0 con 0x557b328ec4a0 2024-03-29T04:19:08.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc11b7fe640 1 -- 172.21.15.22:0/1198021333 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fc1280c6050 con 0x557b328ec4a0 2024-03-29T04:19:08.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc145c0d000 10 client.5724 fetch_fsmap learned FSMap version 18 2024-03-29T04:19:08.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc145c0d000 1 -- 172.21.15.22:0/1198021333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x557b3267ae80 con 0x557b328ec4a0 2024-03-29T04:19:08.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc11b7fe640 1 -- 172.21.15.22:0/1198021333 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc134054ef0 con 0x557b328ec4a0 2024-03-29T04:19:08.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc11b7fe640 1 -- 172.21.15.22:0/1198021333 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fc1280c9020 con 0x557b328ec4a0 2024-03-29T04:19:08.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc145c0d000 10 client.5724 fetch_fsmap finished waiting for FSMap version 18 2024-03-29T04:19:08.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc145c0d000 10 client.5724 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-29T04:19:08.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc145c0d000 20 client.5724 populate_metadata read hostname 'smithi022' 2024-03-29T04:19:08.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc145c0d000 1 --2- 172.21.15.22:0/1198021333 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x557b328fc540 0x557b328fe930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc145c0d000 4 client.5724 mds_command: new command op to 5529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:19:08.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc145c0d000 1 -- 172.21.15.22:0/1198021333 --> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b3268dbc0 con 0x557b328fc540 2024-03-29T04:19:08.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc140827640 1 --2- 172.21.15.22:0/1198021333 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x557b328fc540 0x557b328fe930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc11b7fe640 10 client.5724 ms_handle_connect on v2:172.21.15.22:6834/2966923505 2024-03-29T04:19:08.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc140827640 1 --2- 172.21.15.22:0/1198021333 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x557b328fc540 0x557b328fe930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63d460048887bd72 server_cookie=56c3400f7e39d293 in_seq=0 out_seq=0 2024-03-29T04:19:08.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc11b7fe640 1 -- 172.21.15.22:0/1198021333 <== mds.0 v2:172.21.15.22:6834/2966923505 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x557b3268dbc0 con 0x557b328fc540 2024-03-29T04:19:08.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc11b7fe640 10 client.5724 handle_command_reply: tid=0 2024-03-29T04:19:08.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc145c0d000 1 client.5724 shutdown 2024-03-29T04:19:08.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc145c0d000 1 -- 172.21.15.22:0/1198021333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328ec4a0 msgr2=0x557b328d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:08.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc145c0d000 1 --2- 172.21.15.22:0/1198021333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328ec4a0 0x557b328d0690 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc128019f90 tx=0x7fc12801aba0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc145c0d000 1 -- 172.21.15.22:0/1198021333 shutdown_connections 2024-03-29T04:19:08.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc145c0d000 1 --2- 172.21.15.22:0/1198021333 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x557b328fc540 0x557b328fe930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc145c0d000 1 --2- 172.21.15.22:0/1198021333 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x557b328ecdc0 0x557b328d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc145c0d000 1 --2- 172.21.15.22:0/1198021333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328ec4a0 0x557b328d0690 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc145c0d000 1 --2- 172.21.15.22:0/1198021333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b328a00b0 0x557b328a7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc145c0d000 1 -- 172.21.15.22:0/1198021333 >> 172.21.15.22:0/1198021333 conn(0x557b3278ab80 msgr2=0x557b3289f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:08.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc11b7fe640 0 client.5724 ms_handle_reset on v2:172.21.15.22:6834/2966923505 2024-03-29T04:19:08.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc145c0d000 1 -- 172.21.15.22:0/1198021333 shutdown_connections 2024-03-29T04:19:08.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc145c0d000 1 -- 172.21.15.22:0/1198021333 wait complete. 2024-03-29T04:19:08.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.939+0000 7fc145c0d000 20 client.5724 trim_cache size 0 max 16384 2024-03-29T04:19:08.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc145c0d000 1 Processor -- start 2024-03-29T04:19:08.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc145c0d000 1 -- start start 2024-03-29T04:19:08.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc145c0d000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x557b328a00b0 0x557b328f9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc145c0d000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b328ec4a0 0x557b328f9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc145c0d000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328ecdc0 0x557b3289e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc145c0d000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c204500 con 0x557b328ecdc0 2024-03-29T04:19:08.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc145c0d000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c2034e0 con 0x557b328ec4a0 2024-03-29T04:19:08.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc145c0d000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fc13c0d0200 con 0x557b328a00b0 2024-03-29T04:19:08.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc141028640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328ecdc0 0x557b3289e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc141028640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328ecdc0 0x557b3289e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35904/0 (socket says 172.21.15.22:35904) 2024-03-29T04:19:08.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc141028640 1 -- 172.21.15.22:0/1684392246 learned_addr learned my addr 172.21.15.22:0/1684392246 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:08.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc13bfff640 1 --2- 172.21.15.22:0/1684392246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b328ec4a0 0x557b328f9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc140827640 1 --2- 172.21.15.22:0/1684392246 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x557b328a00b0 0x557b328f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc141028640 1 -- 172.21.15.22:0/1684392246 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x557b328a00b0 msgr2=0x557b328f9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:08.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc141028640 1 --2- 172.21.15.22:0/1684392246 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x557b328a00b0 0x557b328f9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc141028640 1 -- 172.21.15.22:0/1684392246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b328ec4a0 msgr2=0x557b328f9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:08.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc141028640 1 --2- 172.21.15.22:0/1684392246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b328ec4a0 0x557b328f9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc141028640 1 -- 172.21.15.22:0/1684392246 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b326ad490 con 0x557b328ecdc0 2024-03-29T04:19:08.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc141028640 1 --2- 172.21.15.22:0/1684392246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328ecdc0 0x557b3289e2d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc1340568f0 tx=0x7fc134086000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2486fc66a22e11e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:08.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc125ffb640 1 -- 172.21.15.22:0/1684392246 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc134054ef0 con 0x557b328ecdc0 2024-03-29T04:19:08.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc125ffb640 1 -- 172.21.15.22:0/1684392246 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc134053260 con 0x557b328ecdc0 2024-03-29T04:19:08.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.979+0000 7fc125ffb640 1 -- 172.21.15.22:0/1684392246 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13405f540 con 0x557b328ecdc0 2024-03-29T04:19:08.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc145c0d000 1 -- 172.21.15.22:0/1684392246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328ecdc0 msgr2=0x557b3289e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:08.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc145c0d000 1 --2- 172.21.15.22:0/1684392246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328ecdc0 0x557b3289e2d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc1340568f0 tx=0x7fc134086000 comp rx=0 tx=0).stop 2024-03-29T04:19:08.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc145c0d000 1 -- 172.21.15.22:0/1684392246 shutdown_connections 2024-03-29T04:19:08.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc145c0d000 1 --2- 172.21.15.22:0/1684392246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328ecdc0 0x557b3289e2d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc145c0d000 1 --2- 172.21.15.22:0/1684392246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b328ec4a0 0x557b328f9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc145c0d000 1 --2- 172.21.15.22:0/1684392246 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x557b328a00b0 0x557b328f9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc145c0d000 1 -- 172.21.15.22:0/1684392246 >> 172.21.15.22:0/1684392246 conn(0x557b3278ab80 msgr2=0x557b328a4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:08.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc145c0d000 1 -- 172.21.15.22:0/1684392246 shutdown_connections 2024-03-29T04:19:08.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc145c0d000 1 -- 172.21.15.22:0/1684392246 wait complete. 2024-03-29T04:19:08.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc145c0d000 1 -- start start 2024-03-29T04:19:08.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc145c0d000 1 Processor -- start 2024-03-29T04:19:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc145c0d000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x557b328a00b0 0x557b3295ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc145c0d000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328ec4a0 0x557b3295f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc145c0d000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b328ecdc0 0x557b32900050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc145c0d000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557b32698e80 con 0x557b328ec4a0 2024-03-29T04:19:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc145c0d000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c202ea0 con 0x557b328ecdc0 2024-03-29T04:19:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc145c0d000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x557b326ef8d0 con 0x557b328a00b0 2024-03-29T04:19:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc145c0d000 10 client.0 _ll_register_callbacks cb 0x7fc1280ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:19:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc13bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328ec4a0 0x557b3295f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc13bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328ec4a0 0x557b3295f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35912/0 (socket says 172.21.15.22:35912) 2024-03-29T04:19:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc13bfff640 1 -- 172.21.15.22:0/2069356177 learned_addr learned my addr 172.21.15.22:0/2069356177 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc141028640 1 --2- 172.21.15.22:0/2069356177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b328ecdc0 0x557b32900050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc140827640 1 --2- 172.21.15.22:0/2069356177 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x557b328a00b0 0x557b3295ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc13bfff640 1 -- 172.21.15.22:0/2069356177 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x557b328a00b0 msgr2=0x557b3295ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc13bfff640 1 --2- 172.21.15.22:0/2069356177 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x557b328a00b0 0x557b3295ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc13bfff640 1 -- 172.21.15.22:0/2069356177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b328ecdc0 msgr2=0x557b32900050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc13bfff640 1 --2- 172.21.15.22:0/2069356177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b328ecdc0 0x557b32900050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc13bfff640 1 -- 172.21.15.22:0/2069356177 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b326abe80 con 0x557b328ec4a0 2024-03-29T04:19:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc140827640 1 --2- 172.21.15.22:0/2069356177 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x557b328a00b0 0x557b3295ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:19:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc13bfff640 1 --2- 172.21.15.22:0/2069356177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328ec4a0 0x557b3295f2d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc12801c8d0 tx=0x7fc128009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e2d296d9574086 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc11b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-29T04:19:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc11b7fe640 1 -- 172.21.15.22:0/2069356177 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1280d0060 con 0x557b328ec4a0 2024-03-29T04:19:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc11b7fe640 1 -- 172.21.15.22:0/2069356177 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fc128004230 con 0x557b328ec4a0 2024-03-29T04:19:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc145c0d000 1 -- 172.21.15.22:0/2069356177 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc134053260 con 0x557b328ec4a0 2024-03-29T04:19:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc11b7fe640 1 -- 172.21.15.22:0/2069356177 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fc1280d6050 con 0x557b328ec4a0 2024-03-29T04:19:08.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc145c0d000 10 client.5730 fetch_fsmap learned FSMap version 18 2024-03-29T04:19:08.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.983+0000 7fc145c0d000 1 -- 172.21.15.22:0/2069356177 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x557b32699270 con 0x557b328ec4a0 2024-03-29T04:19:08.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc11b7fe640 1 -- 172.21.15.22:0/2069356177 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc134053260 con 0x557b328ec4a0 2024-03-29T04:19:08.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc11b7fe640 1 -- 172.21.15.22:0/2069356177 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fc1280d9020 con 0x557b328ec4a0 2024-03-29T04:19:08.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc145c0d000 10 client.5730 fetch_fsmap finished waiting for FSMap version 18 2024-03-29T04:19:08.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc145c0d000 10 client.5730 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-29T04:19:08.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc145c0d000 20 client.5730 populate_metadata read hostname 'smithi022' 2024-03-29T04:19:08.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc145c0d000 1 --2- 172.21.15.22:0/2069356177 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x557b32900590 0x557b32900970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:08.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc145c0d000 4 client.5730 mds_command: new command op to 5529 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T04:19:08.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc145c0d000 1 -- 172.21.15.22:0/2069356177 --> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] -- command(tid 0: {"prefix": "status"}) -- 0x557b326ffe90 con 0x557b32900590 2024-03-29T04:19:08.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc140827640 1 --2- 172.21.15.22:0/2069356177 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x557b32900590 0x557b32900970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:08.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc11b7fe640 10 client.5730 ms_handle_connect on v2:172.21.15.22:6834/2966923505 2024-03-29T04:19:08.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc140827640 1 --2- 172.21.15.22:0/2069356177 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x557b32900590 0x557b32900970 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ff9a25ffc427f63 server_cookie=989e812d39ee07bf in_seq=0 out_seq=0 2024-03-29T04:19:08.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc11b7fe640 1 -- 172.21.15.22:0/2069356177 <== mds.0 v2:172.21.15.22:6834/2966923505 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x557b326ffe90 con 0x557b32900590 2024-03-29T04:19:08.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc11b7fe640 10 client.5730 handle_command_reply: tid=0 2024-03-29T04:19:08.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc145c0d000 1 client.5730 shutdown 2024-03-29T04:19:08.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc145c0d000 1 -- 172.21.15.22:0/2069356177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328ec4a0 msgr2=0x557b3295f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:08.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc145c0d000 1 --2- 172.21.15.22:0/2069356177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328ec4a0 0x557b3295f2d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc12801c8d0 tx=0x7fc128009190 comp rx=0 tx=0).stop 2024-03-29T04:19:08.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc145c0d000 1 -- 172.21.15.22:0/2069356177 shutdown_connections 2024-03-29T04:19:08.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc145c0d000 1 --2- 172.21.15.22:0/2069356177 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x557b32900590 0x557b32900970 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc145c0d000 1 --2- 172.21.15.22:0/2069356177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b328ecdc0 0x557b32900050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc145c0d000 1 --2- 172.21.15.22:0/2069356177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b328ec4a0 0x557b3295f2d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc11b7fe640 0 client.5730 ms_handle_reset on v2:172.21.15.22:6834/2966923505 2024-03-29T04:19:08.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc145c0d000 1 --2- 172.21.15.22:0/2069356177 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x557b328a00b0 0x557b3295ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc145c0d000 1 -- 172.21.15.22:0/2069356177 >> 172.21.15.22:0/2069356177 conn(0x557b3278ab80 msgr2=0x557b328f9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:08.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc145c0d000 1 -- 172.21.15.22:0/2069356177 shutdown_connections 2024-03-29T04:19:08.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc145c0d000 1 -- 172.21.15.22:0/2069356177 wait complete. 2024-03-29T04:19:08.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc145c0d000 20 client.5730 trim_cache size 0 max 16384 2024-03-29T04:19:08.974 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-29T04:19:08.974 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "9f2f9779-fe00-4763-9aa1-fd3768e2f135", 2024-03-29T04:19:08.974 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-03-29T04:19:08.974 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 5529, 2024-03-29T04:19:08.974 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-03-29T04:19:08.974 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-03-29T04:19:08.974 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-03-29T04:19:08.974 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 10.31991545, 2024-03-29T04:19:08.974 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 18, 2024-03-29T04:19:08.975 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 40, 2024-03-29T04:19:08.975 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 40, 2024-03-29T04:19:08.975 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 17.375195021 2024-03-29T04:19:08.975 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-29T04:19:08.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc142a8b640 1 -- 172.21.15.22:0/646666589 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc120067fc0 msgr2=0x7fc12006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:08.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc142a8b640 1 --2- 172.21.15.22:0/646666589 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc120067fc0 0x7fc12006a480 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc13000b930 tx=0x7fc130008040 comp rx=0 tx=0).stop 2024-03-29T04:19:08.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc142a8b640 1 -- 172.21.15.22:0/646666589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc13c0b5b20 msgr2=0x7fc13c150de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:08.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc142a8b640 1 --2- 172.21.15.22:0/646666589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc13c0b5b20 0x7fc13c150de0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc13400cb20 tx=0x7fc1340021d0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc142a8b640 1 -- 172.21.15.22:0/646666589 shutdown_connections 2024-03-29T04:19:08.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc142a8b640 1 --2- 172.21.15.22:0/646666589 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fc120067fc0 0x7fc12006a480 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc142a8b640 1 --2- 172.21.15.22:0/646666589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc13c0b5b20 0x7fc13c150de0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc142a8b640 1 --2- 172.21.15.22:0/646666589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc13c070490 0x7fc13c158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc142a8b640 1 --2- 172.21.15.22:0/646666589 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fc13c06dcc0 0x7fc13c157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:08.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc142a8b640 1 -- 172.21.15.22:0/646666589 >> 172.21.15.22:0/646666589 conn(0x7fc13c072500 msgr2=0x7fc13c152300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:08.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc142a8b640 1 -- 172.21.15.22:0/646666589 shutdown_connections 2024-03-29T04:19:08.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:08.987+0000 7fc142a8b640 1 -- 172.21.15.22:0/646666589 wait complete. 2024-03-29T04:19:08.985 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:19:09.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58ef24b640 1 Processor -- start 2024-03-29T04:19:09.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58ef24b640 1 -- start start 2024-03-29T04:19:09.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58ef24b640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f58e806c850 0x7f58e815c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:09.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58ef24b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58e80afb90 con 0x7f58e80b49e0 2024-03-29T04:19:09.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58ef24b640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58e80afe70 con 0x7f58e80b4dc0 2024-03-29T04:19:09.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58ef24b640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f58e80b0150 con 0x7f58e806c850 2024-03-29T04:19:09.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58ecfe7640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f58e80b49e0 0x7f58e80b42a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57056/0 (socket says 172.21.15.22:57056) 2024-03-29T04:19:09.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58ecfe7640 1 -- 172.21.15.22:0/1512593673 learned_addr learned my addr 172.21.15.22:0/1512593673 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:19:09.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58e7fff640 1 --2- 172.21.15.22:0/1512593673 >> v2:172.21.15.151:3300/0 conn(0x7f58e806c850 0x7f58e815c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:09.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58e77fe640 1 -- 172.21.15.22:0/1512593673 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2471479632 0 0) 0x7f58e80afb90 con 0x7f58e80b49e0 2024-03-29T04:19:09.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58e77fe640 1 -- 172.21.15.22:0/1512593673 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58d8003530 con 0x7f58e80b49e0 2024-03-29T04:19:09.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58e77fe640 1 -- 172.21.15.22:0/1512593673 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2766585998 0 0) 0x7f58d8003530 con 0x7f58e80b49e0 2024-03-29T04:19:09.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58e77fe640 1 -- 172.21.15.22:0/1512593673 >> v2:172.21.15.151:3300/0 conn(0x7f58e806c850 msgr2=0x7f58e815c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:09.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58e77fe640 1 --2- 172.21.15.22:0/1512593673 >> v2:172.21.15.151:3300/0 conn(0x7f58e806c850 0x7f58e815c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:09.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58e77fe640 1 -- 172.21.15.22:0/1512593673 >> v1:172.21.15.151:6789/0 conn(0x7f58e80b4dc0 legacy=0x7f58e806dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:09.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58e77fe640 1 -- 172.21.15.22:0/1512593673 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e80b37c0 con 0x7f58e80b49e0 2024-03-29T04:19:09.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58e77fe640 1 -- 172.21.15.22:0/1512593673 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f58dc001680 con 0x7f58e80b49e0 2024-03-29T04:19:09.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58e77fe640 1 -- 172.21.15.22:0/1512593673 >> v1:172.21.15.22:6789/0 conn(0x7f58e80b49e0 legacy=0x7f58e80b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:09.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58e77fe640 1 --2- 172.21.15.22:0/1512593673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58d80039c0 0x7f58d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:09.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58e77fe640 1 --2- 172.21.15.22:0/1512593673 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58d8004810 0x7f58d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:09.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58e77fe640 1 --2- 172.21.15.22:0/1512593673 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58d80051d0 0x7f58d800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:09.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58e77fe640 1 -- 172.21.15.22:0/1512593673 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f58e80b0150 con 0x7f58d80039c0 2024-03-29T04:19:09.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58e77fe640 1 -- 172.21.15.22:0/1512593673 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f58d800c070 con 0x7f58d80051d0 2024-03-29T04:19:09.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58e77fe640 1 -- 172.21.15.22:0/1512593673 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f58d800c270 con 0x7f58d8004810 2024-03-29T04:19:09.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58ecfe7640 1 --2- 172.21.15.22:0/1512593673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58d80039c0 0x7f58d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:09.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58ed7e8640 1 --2- 172.21.15.22:0/1512593673 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58d80051d0 0x7f58d800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:09.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58e7fff640 1 --2- 172.21.15.22:0/1512593673 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58d8004810 0x7f58d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:09.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58ecfe7640 1 -- 172.21.15.22:0/1512593673 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58d8004810 msgr2=0x7f58d8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:09.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58ecfe7640 1 --2- 172.21.15.22:0/1512593673 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58d8004810 0x7f58d8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:09.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58ecfe7640 1 -- 172.21.15.22:0/1512593673 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58d80051d0 msgr2=0x7f58d800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:09.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58ecfe7640 1 --2- 172.21.15.22:0/1512593673 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58d80051d0 0x7f58d800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:09.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58ecfe7640 1 -- 172.21.15.22:0/1512593673 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d8003530 con 0x7f58d80039c0 2024-03-29T04:19:09.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58ed7e8640 1 -- 172.21.15.22:0/1512593673 reap_dead start 2024-03-29T04:19:09.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.119+0000 7f58e7fff640 1 --2- 172.21.15.22:0/1512593673 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58d8004810 0x7f58d8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:19:09.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.123+0000 7f58ecfe7640 1 --2- 172.21.15.22:0/1512593673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58d80039c0 0x7f58d8003de0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00c1c0 tx=0x7f58dc01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bdfeb34e4284d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:09.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.123+0000 7f58e77fe640 1 -- 172.21.15.22:0/1512593673 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc007d00 con 0x7f58d80039c0 2024-03-29T04:19:09.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.123+0000 7f58e77fe640 1 -- 172.21.15.22:0/1512593673 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58dc0a9020 con 0x7f58d80039c0 2024-03-29T04:19:09.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.123+0000 7f58e77fe640 1 -- 172.21.15.22:0/1512593673 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc005ae0 con 0x7f58d80039c0 2024-03-29T04:19:09.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.123+0000 7f58ef24b640 1 -- 172.21.15.22:0/1512593673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58d80039c0 msgr2=0x7f58d8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:09.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.123+0000 7f58ef24b640 1 --2- 172.21.15.22:0/1512593673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58d80039c0 0x7f58d8003de0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00c1c0 tx=0x7f58dc01cc60 comp rx=0 tx=0).stop 2024-03-29T04:19:09.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.123+0000 7f58ef24b640 1 -- 172.21.15.22:0/1512593673 shutdown_connections 2024-03-29T04:19:09.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.123+0000 7f58ef24b640 1 --2- 172.21.15.22:0/1512593673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58d80039c0 0x7f58d8003de0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:09.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.123+0000 7f58ef24b640 1 -- 172.21.15.22:0/1512593673 >> 172.21.15.22:0/1512593673 conn(0x7f58e80b2b30 msgr2=0x7f58e80d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:09.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.123+0000 7f58ef24b640 1 -- 172.21.15.22:0/1512593673 shutdown_connections 2024-03-29T04:19:09.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.123+0000 7f58ef24b640 1 -- 172.21.15.22:0/1512593673 wait complete. 2024-03-29T04:19:09.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58ef24b640 1 Processor -- start 2024-03-29T04:19:09.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58ef24b640 1 -- start start 2024-03-29T04:19:09.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58ef24b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58d80039c0 0x7f58e814e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:09.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58ef24b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58e815a090 0x7f58e815a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:09.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58ef24b640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58e815c810 0x7f58e815cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:09.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58ef24b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f58e80b46b0 con 0x7f58e815a090 2024-03-29T04:19:09.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58ef24b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f58e80b51d0 con 0x7f58d80039c0 2024-03-29T04:19:09.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58ef24b640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f58e8155190 con 0x7f58e815c810 2024-03-29T04:19:09.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58e7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58e815a090 0x7f58e815a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:09.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58e7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58e815a090 0x7f58e815a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35940/0 (socket says 172.21.15.22:35940) 2024-03-29T04:19:09.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58e7fff640 1 -- 172.21.15.22:0/795680847 learned_addr learned my addr 172.21.15.22:0/795680847 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:09.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58ecfe7640 1 --2- 172.21.15.22:0/795680847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58d80039c0 0x7f58e814e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:09.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58e7fff640 1 -- 172.21.15.22:0/795680847 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58e815c810 msgr2=0x7f58e815cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:09.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58ed7e8640 1 --2- 172.21.15.22:0/795680847 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58e815c810 0x7f58e815cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:09.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58e7fff640 1 --2- 172.21.15.22:0/795680847 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58e815c810 0x7f58e815cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:09.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58e7fff640 1 -- 172.21.15.22:0/795680847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58d80039c0 msgr2=0x7f58e814e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:09.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58e7fff640 1 --2- 172.21.15.22:0/795680847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58d80039c0 0x7f58e814e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:09.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58e7fff640 1 -- 172.21.15.22:0/795680847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e815dce0 con 0x7f58e815a090 2024-03-29T04:19:09.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58e7fff640 1 --2- 172.21.15.22:0/795680847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58e815a090 0x7f58e815a510 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f58d40107c0 tx=0x7f58d4010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:09.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58e57fa640 1 -- 172.21.15.22:0/795680847 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d400f040 con 0x7f58e815a090 2024-03-29T04:19:09.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58e57fa640 1 -- 172.21.15.22:0/795680847 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d400eb60 con 0x7f58e815a090 2024-03-29T04:19:09.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58ef24b640 1 -- 172.21.15.22:0/795680847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e815d1b0 con 0x7f58e815a090 2024-03-29T04:19:09.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58e57fa640 1 -- 172.21.15.22:0/795680847 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4010e70 con 0x7f58e815a090 2024-03-29T04:19:09.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58ef24b640 1 -- 172.21.15.22:0/795680847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e815d440 con 0x7f58e815a090 2024-03-29T04:19:09.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.127+0000 7f58ef24b640 1 -- 172.21.15.22:0/795680847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e8068890 con 0x7f58e815a090 2024-03-29T04:19:09.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.131+0000 7f58e57fa640 1 -- 172.21.15.22:0/795680847 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f58d401c400 con 0x7f58e815a090 2024-03-29T04:19:09.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.131+0000 7f58e57fa640 1 --2- 172.21.15.22:0/795680847 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f58cc067fc0 0x7f58cc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:09.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.131+0000 7f58e57fa640 1 -- 172.21.15.22:0/795680847 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f58d4086b80 con 0x7f58e815a090 2024-03-29T04:19:09.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.131+0000 7f58ecfe7640 1 --2- 172.21.15.22:0/795680847 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f58cc067fc0 0x7f58cc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:09.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.131+0000 7f58ecfe7640 1 --2- 172.21.15.22:0/795680847 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f58cc067fc0 0x7f58cc06a480 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0aa000 tx=0x7f58e80b0040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:09.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.135+0000 7f58e57fa640 1 -- 172.21.15.22:0/795680847 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f58d4018030 con 0x7f58e815a090 2024-03-29T04:19:09.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.343+0000 7f58ef24b640 1 -- 172.21.15.22:0/795680847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f58e81557f0 con 0x7f58e815a090 2024-03-29T04:19:09.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.347+0000 7f58e57fa640 1 -- 172.21.15.22:0/795680847 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f58d4057190 con 0x7f58e815a090 2024-03-29T04:19:09.327 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T04:19:09.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.347+0000 7f58ef24b640 1 -- 172.21.15.22:0/795680847 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f58cc067fc0 msgr2=0x7f58cc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:09.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.347+0000 7f58ef24b640 1 --2- 172.21.15.22:0/795680847 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f58cc067fc0 0x7f58cc06a480 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0aa000 tx=0x7f58e80b0040 comp rx=0 tx=0).stop 2024-03-29T04:19:09.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.347+0000 7f58ef24b640 1 -- 172.21.15.22:0/795680847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58e815a090 msgr2=0x7f58e815a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:09.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.347+0000 7f58ef24b640 1 --2- 172.21.15.22:0/795680847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58e815a090 0x7f58e815a510 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f58d40107c0 tx=0x7f58d4010c80 comp rx=0 tx=0).stop 2024-03-29T04:19:09.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.347+0000 7f58ef24b640 1 -- 172.21.15.22:0/795680847 shutdown_connections 2024-03-29T04:19:09.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.347+0000 7f58ef24b640 1 --2- 172.21.15.22:0/795680847 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f58e815c810 0x7f58e815cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:09.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.347+0000 7f58ef24b640 1 --2- 172.21.15.22:0/795680847 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f58cc067fc0 0x7f58cc06a480 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:09.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.347+0000 7f58ef24b640 1 --2- 172.21.15.22:0/795680847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58e815a090 0x7f58e815a510 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:09.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.347+0000 7f58ef24b640 1 --2- 172.21.15.22:0/795680847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58d80039c0 0x7f58e814e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:09.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.347+0000 7f58ef24b640 1 -- 172.21.15.22:0/795680847 >> 172.21.15.22:0/795680847 conn(0x7f58e80b2b30 msgr2=0x7f58e814eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:09.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.347+0000 7f58ef24b640 1 -- 172.21.15.22:0/795680847 shutdown_connections 2024-03-29T04:19:09.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09.347+0000 7f58ef24b640 1 -- 172.21.15.22:0/795680847 wait complete. 2024-03-29T04:19:09.344 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:19:09.344 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:19:09.344 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-03-29T04:19:09.344 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T04:19:09.345 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T04:19:09.345 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:19:09.345 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:19:09.345 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:19:09.345 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:19:09.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:09.345 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-03-29T04:19:09.350 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:19:09.351 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:19:09.351 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:19:09.351 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:19:09.351 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-03-29T04:19:09.351 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:19:09.351 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T04:19:09.351 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-03-29T04:19:09.351 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-03-29T04:19:09.351 INFO:teuthology.orchestra.run.smithi022.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:19:09.351 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:19:09.351 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-03-29T04:19:09.351 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-29T04:19:09.351 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42395sec preferred_lft 42395sec 2024-03-29T04:19:09.351 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-03-29T04:19:09.351 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:19:09.352 INFO:teuthology.orchestra.run.smithi022.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T04:19:09.352 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-03-29T04:19:09.352 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-03-29T04:19:09.352 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T04:19:09.352 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:19:09.352 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-03-29T04:19:09.352 INFO:teuthology.orchestra.run.smithi022.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T04:19:09.352 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 02:42:df:ac:77:fb brd ff:ff:ff:ff:ff:ff 2024-03-29T04:19:09.352 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T04:19:09.352 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:19:09.352 INFO:teuthology.orchestra.run.smithi022.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:19:09.352 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-29T04:19:09.352 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T04:19:09.352 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:19:09.353 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-29T04:19:09.353 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:19:09.353 INFO:teuthology.orchestra.run.smithi022.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:19:09.353 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:19:09.353 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-29T04:19:09.353 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:19:09.353 INFO:teuthology.orchestra.run.smithi022.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:19:09.353 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:19:09.353 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-29T04:19:09.353 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:19:09.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:09.354 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:19:09.404 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-03-29T04:19:09.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:09.406 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-03-29T04:19:09.452 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-03-29T04:19:09.452 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:19:09.452 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-29T04:19:09.452 INFO:teuthology.orchestra.run.smithi022.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T04:19:09.452 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-29T04:19:09.453 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:19:09.453 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:19:09.453 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-29T04:19:09.453 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T04:19:09.453 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T04:19:09.453 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T04:19:09.453 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-29T04:19:09.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:19:09.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:19:09.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:09.573 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-03-29T04:19:09.619 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T04:19:09.619 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T04:19:09.619 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:19:09.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:09.620 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-03-29T04:19:09.676 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T04:19:09.676 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:19:09.676 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T04:19:09.723 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:19:09.723 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:19:09.723 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T04:19:09.771 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-03-29T04:19:09.828 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:19:09.876 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-03-29T04:19:09.876 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-03-29T04:19:09.876 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-03-29T04:19:09.876 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-03-29T04:19:09.876 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-03-29T04:19:09.876 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-03-29T04:19:09.876 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-03-29T04:19:09.876 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-03-29T04:19:09.877 INFO:teuthology.orchestra.run.smithi022.stdout:/run/lock 2024-03-29T04:19:09.877 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-03-29T04:19:09.877 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-03-29T04:19:09.877 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-03-29T04:19:09.877 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:19:09.877 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-03-29T04:19:09.877 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-03-29T04:19:09.877 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-03-29T04:19:09.877 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-03-29T04:19:09.877 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/fs/nfsd 2024-03-29T04:19:09.877 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-03-29T04:19:09.877 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-03-29T04:19:09.877 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysusers.service 2024-03-29T04:19:09.877 INFO:teuthology.orchestra.run.smithi022.stdout:/run/rpc_pipefs 2024-03-29T04:19:09.877 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-03-29T04:19:09.877 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-03-29T04:19:09.878 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:19:09.878 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:19:09.878 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:19:09.878 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:19:09.878 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-03-29T04:19:09.878 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:19:09.878 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:19:09.878 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:19:09.878 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:19:09.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:09.879 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-03-29T04:19:09.927 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T04:19:09.927 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T04:19:09.972 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-03-29T04:19:09.985 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:19:10.035 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-03-29T04:19:10.035 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-03-29T04:19:10.035 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-03-29T04:19:10.035 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-03-29T04:19:10.036 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-03-29T04:19:10.036 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-03-29T04:19:10.036 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-03-29T04:19:10.036 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-03-29T04:19:10.036 INFO:teuthology.orchestra.run.smithi022.stdout:/run/lock 2024-03-29T04:19:10.036 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-03-29T04:19:10.036 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-03-29T04:19:10.036 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-03-29T04:19:10.036 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:19:10.036 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-03-29T04:19:10.036 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-03-29T04:19:10.036 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-03-29T04:19:10.036 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-03-29T04:19:10.036 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/fs/nfsd 2024-03-29T04:19:10.036 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-03-29T04:19:10.037 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-03-29T04:19:10.037 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysusers.service 2024-03-29T04:19:10.037 INFO:teuthology.orchestra.run.smithi022.stdout:/run/rpc_pipefs 2024-03-29T04:19:10.037 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-03-29T04:19:10.037 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-03-29T04:19:10.037 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:19:10.037 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:19:10.037 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:19:10.037 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:19:10.037 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-03-29T04:19:10.037 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:19:10.037 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:19:10.037 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:19:10.037 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:19:10.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:10.038 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-03-29T04:19:10.045 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:2024-03-29T04:19:10.063+0000 7f98be7a0580 -1 init, newargv = 0x55ac4a3fc9a0 newargc=15 2024-03-29T04:19:10.045 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[38693]: starting ceph client 2024-03-29T04:19:10.051 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[38693]: starting fuse 2024-03-29T04:19:11.150 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-03-29T04:19:11.165 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:19:11.213 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-03-29T04:19:11.213 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-03-29T04:19:11.213 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-03-29T04:19:11.213 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-03-29T04:19:11.213 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-03-29T04:19:11.213 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-03-29T04:19:11.213 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-03-29T04:19:11.213 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-03-29T04:19:11.214 INFO:teuthology.orchestra.run.smithi022.stdout:/run/lock 2024-03-29T04:19:11.214 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-03-29T04:19:11.214 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-03-29T04:19:11.214 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-03-29T04:19:11.214 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:19:11.214 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-03-29T04:19:11.214 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-03-29T04:19:11.214 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-03-29T04:19:11.214 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-03-29T04:19:11.214 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/fs/nfsd 2024-03-29T04:19:11.214 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-03-29T04:19:11.214 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-03-29T04:19:11.214 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysusers.service 2024-03-29T04:19:11.214 INFO:teuthology.orchestra.run.smithi022.stdout:/run/rpc_pipefs 2024-03-29T04:19:11.215 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-03-29T04:19:11.215 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-03-29T04:19:11.215 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:19:11.215 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:19:11.215 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:19:11.215 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:19:11.215 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-03-29T04:19:11.215 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:19:11.215 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:19:11.215 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:19:11.215 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:19:11.215 INFO:teuthology.orchestra.run.smithi022.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-29T04:19:11.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:11.216 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-03-29T04:19:11.259 INFO:teuthology.orchestra.run.smithi022.stdout:45 2024-03-29T04:19:11.260 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-03-29T04:19:11.260 DEBUG:teuthology.orchestra.run.smithi022:> sudo stdin-killer -- python3 -c ' 2024-03-29T04:19:11.260 DEBUG:teuthology.orchestra.run.smithi022:> import glob 2024-03-29T04:19:11.260 DEBUG:teuthology.orchestra.run.smithi022:> import re 2024-03-29T04:19:11.260 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-03-29T04:19:11.260 DEBUG:teuthology.orchestra.run.smithi022:> import subprocess 2024-03-29T04:19:11.260 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-29T04:19:11.260 DEBUG:teuthology.orchestra.run.smithi022:> def _find_admin_socket(client_name): 2024-03-29T04:19:11.260 DEBUG:teuthology.orchestra.run.smithi022:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T04:19:11.261 DEBUG:teuthology.orchestra.run.smithi022:> files = glob.glob(asok_path) 2024-03-29T04:19:11.261 DEBUG:teuthology.orchestra.run.smithi022:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T04:19:11.261 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-29T04:19:11.261 DEBUG:teuthology.orchestra.run.smithi022:> # Given a non-glob path, it better be there 2024-03-29T04:19:11.261 DEBUG:teuthology.orchestra.run.smithi022:> if "*" not in asok_path: 2024-03-29T04:19:11.261 DEBUG:teuthology.orchestra.run.smithi022:> assert(len(files) == 1) 2024-03-29T04:19:11.261 DEBUG:teuthology.orchestra.run.smithi022:> return files[0] 2024-03-29T04:19:11.261 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-29T04:19:11.261 DEBUG:teuthology.orchestra.run.smithi022:> for f in files: 2024-03-29T04:19:11.261 DEBUG:teuthology.orchestra.run.smithi022:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T04:19:11.261 DEBUG:teuthology.orchestra.run.smithi022:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T04:19:11.261 DEBUG:teuthology.orchestra.run.smithi022:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T04:19:11.261 DEBUG:teuthology.orchestra.run.smithi022:> contents = proc_f.read() 2024-03-29T04:19:11.261 DEBUG:teuthology.orchestra.run.smithi022:> if mountpoint in contents: 2024-03-29T04:19:11.261 DEBUG:teuthology.orchestra.run.smithi022:> return f 2024-03-29T04:19:11.261 DEBUG:teuthology.orchestra.run.smithi022:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T04:19:11.262 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-29T04:19:11.262 DEBUG:teuthology.orchestra.run.smithi022:> print(_find_admin_socket("client.0")) 2024-03-29T04:19:11.262 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-03-29T04:19:11.385 INFO:teuthology.orchestra.run.smithi022.stdout:/var/run/ceph/ceph-client.0.38693.asok 2024-03-29T04:19:11.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:19:11.398 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38693.asok 2024-03-29T04:19:11.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:11.398 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38693.asok status 2024-03-29T04:19:11.553 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-29T04:19:11.553 INFO:teuthology.orchestra.run.smithi022.stdout: "metadata": { 2024-03-29T04:19:11.553 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_sha1": "69e2cd73c52284dfd5096c66b60c6e6b26a3c447", 2024-03-29T04:19:11.553 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 19.0.0-2520-g69e2cd73 (69e2cd73c52284dfd5096c66b60c6e6b26a3c447) squid (dev)", 2024-03-29T04:19:11.553 INFO:teuthology.orchestra.run.smithi022.stdout: "entity_id": "0", 2024-03-29T04:19:11.554 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": "smithi022", 2024-03-29T04:19:11.554 INFO:teuthology.orchestra.run.smithi022.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T04:19:11.554 INFO:teuthology.orchestra.run.smithi022.stdout: "pid": "38693", 2024-03-29T04:19:11.554 INFO:teuthology.orchestra.run.smithi022.stdout: "root": "/" 2024-03-29T04:19:11.554 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:19:11.554 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_count": 0, 2024-03-29T04:19:11.554 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_pinned_count": 0, 2024-03-29T04:19:11.554 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 5742, 2024-03-29T04:19:11.554 INFO:teuthology.orchestra.run.smithi022.stdout: "inst": { 2024-03-29T04:19:11.555 INFO:teuthology.orchestra.run.smithi022.stdout: "name": { 2024-03-29T04:19:11.555 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "client", 2024-03-29T04:19:11.555 INFO:teuthology.orchestra.run.smithi022.stdout: "num": 5742 2024-03-29T04:19:11.555 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:19:11.555 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2024-03-29T04:19:11.555 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-03-29T04:19:11.555 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:19:11.555 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3683711404 2024-03-29T04:19:11.555 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:19:11.555 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:19:11.555 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2024-03-29T04:19:11.555 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-03-29T04:19:11.555 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:19:11.555 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3683711404 2024-03-29T04:19:11.555 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:19:11.556 INFO:teuthology.orchestra.run.smithi022.stdout: "inst_str": "client.5742 192.168.0.1:0/3683711404", 2024-03-29T04:19:11.556 INFO:teuthology.orchestra.run.smithi022.stdout: "addr_str": "192.168.0.1:0/3683711404", 2024-03-29T04:19:11.556 INFO:teuthology.orchestra.run.smithi022.stdout: "inode_count": 1, 2024-03-29T04:19:11.556 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_epoch": 18, 2024-03-29T04:19:11.556 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch": 40, 2024-03-29T04:19:11.556 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch_barrier": 0, 2024-03-29T04:19:11.556 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklisted": false, 2024-03-29T04:19:11.556 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs" 2024-03-29T04:19:11.556 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-29T04:19:11.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:11.566 DEBUG:teuthology.orchestra.run.smithi022:> stat --file-system '--printf=%T 2024-03-29T04:19:11.567 DEBUG:teuthology.orchestra.run.smithi022:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T04:19:11.572 INFO:teuthology.orchestra.run.smithi022.stdout:fuseblk 2024-03-29T04:19:11.572 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T04:19:11.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:11.572 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T04:19:11.625 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:19:11.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.811+0000 7f4ce5dd6640 1 Processor -- start 2024-03-29T04:19:11.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.811+0000 7f4ce5dd6640 1 -- start start 2024-03-29T04:19:11.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.815+0000 7f4ce5dd6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f4ce015e600 0x7f4ce00b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:11.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.815+0000 7f4ce5dd6640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f4ce00d1100 0x7f4ce00d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:11.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.815+0000 7f4ce5dd6640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ce00d3e60 con 0x7f4ce00b8160 2024-03-29T04:19:11.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.815+0000 7f4ce5dd6640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f4ce00b7450 con 0x7f4ce015e600 2024-03-29T04:19:11.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.815+0000 7f4ce5dd6640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f4ce00d1aa0 con 0x7f4ce00d1100 2024-03-29T04:19:11.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.815+0000 7f4cdf7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f4ce015e600 0x7f4ce00b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:11.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.815+0000 7f4cdf7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f4ce015e600 0x7f4ce00b9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35954/0 (socket says 172.21.15.22:35954) 2024-03-29T04:19:11.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.815+0000 7f4cdffff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f4ce00b8160 0x7f4ce00b85b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57058/0 (socket says 172.21.15.22:57058) 2024-03-29T04:19:11.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.815+0000 7f4cdf7fe640 1 -- 172.21.15.22:0/544275142 learned_addr learned my addr 172.21.15.22:0/544275142 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:11.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.815+0000 7f4cdeffd640 1 --2- 172.21.15.22:0/544275142 >> v2:172.21.15.151:3300/0 conn(0x7f4ce00d1100 0x7f4ce00d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:11.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.815+0000 7f4cdf7fe640 1 -- 172.21.15.22:0/544275142 >> v2:172.21.15.151:3300/0 conn(0x7f4ce00d1100 msgr2=0x7f4ce00d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:11.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.815+0000 7f4cdf7fe640 1 --2- 172.21.15.22:0/544275142 >> v2:172.21.15.151:3300/0 conn(0x7f4ce00d1100 0x7f4ce00d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:11.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.815+0000 7f4cdf7fe640 1 -- 172.21.15.22:0/544275142 >> v1:172.21.15.22:6789/0 conn(0x7f4ce00b8160 legacy=0x7f4ce00b85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:11.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.815+0000 7f4cdf7fe640 1 -- 172.21.15.22:0/544275142 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce00b7680 con 0x7f4ce015e600 2024-03-29T04:19:11.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.815+0000 7f4cdf7fe640 1 --2- 172.21.15.22:0/544275142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ce015e600 0x7f4ce00b9f00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00d1df0 tx=0x7f4cd001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcb1e641946b250 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:11.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.815+0000 7f4cde7fc640 1 -- 172.21.15.22:0/544275142 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd00a57a0 con 0x7f4ce015e600 2024-03-29T04:19:11.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.815+0000 7f4cde7fc640 1 -- 172.21.15.22:0/544275142 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cd00a5940 con 0x7f4ce015e600 2024-03-29T04:19:11.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.815+0000 7f4cde7fc640 1 -- 172.21.15.22:0/544275142 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd00a5d10 con 0x7f4ce015e600 2024-03-29T04:19:11.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4ce5dd6640 1 -- 172.21.15.22:0/544275142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ce015e600 msgr2=0x7f4ce00b9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:11.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4ce5dd6640 1 --2- 172.21.15.22:0/544275142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ce015e600 0x7f4ce00b9f00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00d1df0 tx=0x7f4cd001c9f0 comp rx=0 tx=0).stop 2024-03-29T04:19:11.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4ce5dd6640 1 -- 172.21.15.22:0/544275142 shutdown_connections 2024-03-29T04:19:11.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4ce5dd6640 1 --2- 172.21.15.22:0/544275142 >> v2:172.21.15.151:3300/0 conn(0x7f4ce00d1100 0x7f4ce00d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:11.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4ce5dd6640 1 --2- 172.21.15.22:0/544275142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ce015e600 0x7f4ce00b9f00 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:11.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4ce5dd6640 1 -- 172.21.15.22:0/544275142 >> 172.21.15.22:0/544275142 conn(0x7f4ce00b6230 msgr2=0x7f4ce00b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:11.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4ce5dd6640 1 -- 172.21.15.22:0/544275142 shutdown_connections 2024-03-29T04:19:11.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4ce5dd6640 1 -- 172.21.15.22:0/544275142 wait complete. 2024-03-29T04:19:11.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4ce5dd6640 1 Processor -- start 2024-03-29T04:19:11.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4ce5dd6640 1 -- start start 2024-03-29T04:19:11.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4ce5dd6640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4ce00b8160 0x7f4ce0161400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:11.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4ce5dd6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ce00d1100 0x7f4ce013f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:11.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4ce5dd6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ce015e600 0x7f4ce014e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:11.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4ce5dd6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce016b150 con 0x7f4ce015e600 2024-03-29T04:19:11.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4ce5dd6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce016afd0 con 0x7f4ce00d1100 2024-03-29T04:19:11.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4ce5dd6640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4ce016b2d0 con 0x7f4ce00b8160 2024-03-29T04:19:11.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4cdffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ce015e600 0x7f4ce014e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:11.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4cdffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ce015e600 0x7f4ce014e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35960/0 (socket says 172.21.15.22:35960) 2024-03-29T04:19:11.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4cdffff640 1 -- 172.21.15.22:0/2696446729 learned_addr learned my addr 172.21.15.22:0/2696446729 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:11.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4cdeffd640 1 --2- 172.21.15.22:0/2696446729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ce00d1100 0x7f4ce013f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:11.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4cdf7fe640 1 --2- 172.21.15.22:0/2696446729 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4ce00b8160 0x7f4ce0161400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:11.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4cdffff640 1 -- 172.21.15.22:0/2696446729 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4ce00b8160 msgr2=0x7f4ce0161400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:11.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4cdffff640 1 --2- 172.21.15.22:0/2696446729 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4ce00b8160 0x7f4ce0161400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:11.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4cdffff640 1 -- 172.21.15.22:0/2696446729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ce00d1100 msgr2=0x7f4ce013f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:11.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4cdffff640 1 --2- 172.21.15.22:0/2696446729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ce00d1100 0x7f4ce013f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:11.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4cdffff640 1 -- 172.21.15.22:0/2696446729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce014d900 con 0x7f4ce015e600 2024-03-29T04:19:11.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4cdffff640 1 --2- 172.21.15.22:0/2696446729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ce015e600 0x7f4ce014e3c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800c3c0 tx=0x7f4cd800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:11.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4cd7fff640 1 -- 172.21.15.22:0/2696446729 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd80074e0 con 0x7f4ce015e600 2024-03-29T04:19:11.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4cd7fff640 1 -- 172.21.15.22:0/2696446729 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cd8007680 con 0x7f4ce015e600 2024-03-29T04:19:11.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4cd7fff640 1 -- 172.21.15.22:0/2696446729 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8004a10 con 0x7f4ce015e600 2024-03-29T04:19:11.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4ce5dd6640 1 -- 172.21.15.22:0/2696446729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce015f040 con 0x7f4ce015e600 2024-03-29T04:19:11.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.819+0000 7f4ce5dd6640 1 -- 172.21.15.22:0/2696446729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce0161940 con 0x7f4ce015e600 2024-03-29T04:19:11.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.823+0000 7f4ce5dd6640 1 -- 172.21.15.22:0/2696446729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ce0068890 con 0x7f4ce015e600 2024-03-29T04:19:11.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.823+0000 7f4cd7fff640 1 -- 172.21.15.22:0/2696446729 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f4cd801a020 con 0x7f4ce015e600 2024-03-29T04:19:11.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.823+0000 7f4cd7fff640 1 --2- 172.21.15.22:0/2696446729 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4cc0067fc0 0x7f4cc006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:11.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.823+0000 7f4cd7fff640 1 -- 172.21.15.22:0/2696446729 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f4cd8085d30 con 0x7f4ce015e600 2024-03-29T04:19:11.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.823+0000 7f4cdf7fe640 1 --2- 172.21.15.22:0/2696446729 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4cc0067fc0 0x7f4cc006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:11.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.823+0000 7f4cdf7fe640 1 --2- 172.21.15.22:0/2696446729 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4cc0067fc0 0x7f4cc006a480 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00ae000 tx=0x7f4cd00a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:11.827+0000 7f4cd7fff640 1 -- 172.21.15.22:0/2696446729 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f4cd8056350 con 0x7f4ce015e600 2024-03-29T04:19:12.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:12.023+0000 7f4ce5dd6640 1 -- 172.21.15.22:0/2696446729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4ce015cdd0 con 0x7f4ce015e600 2024-03-29T04:19:12.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:12.023+0000 7f4cd7fff640 1 -- 172.21.15.22:0/2696446729 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f4cd8055c50 con 0x7f4ce015e600 2024-03-29T04:19:12.004 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T04:19:12.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:12.023+0000 7f4ce5dd6640 1 -- 172.21.15.22:0/2696446729 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4cc0067fc0 msgr2=0x7f4cc006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:12.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:12.023+0000 7f4ce5dd6640 1 --2- 172.21.15.22:0/2696446729 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4cc0067fc0 0x7f4cc006a480 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00ae000 tx=0x7f4cd00a7070 comp rx=0 tx=0).stop 2024-03-29T04:19:12.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:12.023+0000 7f4ce5dd6640 1 -- 172.21.15.22:0/2696446729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ce015e600 msgr2=0x7f4ce014e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:12.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:12.023+0000 7f4ce5dd6640 1 --2- 172.21.15.22:0/2696446729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ce015e600 0x7f4ce014e3c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800c3c0 tx=0x7f4cd800c880 comp rx=0 tx=0).stop 2024-03-29T04:19:12.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:12.023+0000 7f4ce5dd6640 1 -- 172.21.15.22:0/2696446729 shutdown_connections 2024-03-29T04:19:12.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:12.023+0000 7f4ce5dd6640 1 --2- 172.21.15.22:0/2696446729 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4cc0067fc0 0x7f4cc006a480 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:12.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:12.023+0000 7f4ce5dd6640 1 --2- 172.21.15.22:0/2696446729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ce015e600 0x7f4ce014e3c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:12.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:12.023+0000 7f4ce5dd6640 1 --2- 172.21.15.22:0/2696446729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ce00d1100 0x7f4ce013f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:12.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:12.023+0000 7f4ce5dd6640 1 --2- 172.21.15.22:0/2696446729 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4ce00b8160 0x7f4ce0161400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:12.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:12.023+0000 7f4ce5dd6640 1 -- 172.21.15.22:0/2696446729 >> 172.21.15.22:0/2696446729 conn(0x7f4ce00b6230 msgr2=0x7f4ce015bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:12.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:12.023+0000 7f4ce5dd6640 1 -- 172.21.15.22:0/2696446729 shutdown_connections 2024-03-29T04:19:12.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:12.023+0000 7f4ce5dd6640 1 -- 172.21.15.22:0/2696446729 wait complete. 2024-03-29T04:19:12.021 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:19:12.021 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:19:12.021 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi151.front.sepia.ceph.com 2024-03-29T04:19:12.021 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-29T04:19:12.021 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-29T04:19:12.021 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:19:12.021 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:19:12.021 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:19:12.022 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:19:12.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:12.022 DEBUG:teuthology.orchestra.run.smithi151:> ip addr 2024-03-29T04:19:12.029 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:19:12.029 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:19:12.029 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:19:12.029 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:19:12.029 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-03-29T04:19:12.029 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:19:12.029 INFO:teuthology.orchestra.run.smithi151.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T04:19:12.029 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0c brd ff:ff:ff:ff:ff:ff 2024-03-29T04:19:12.029 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f0 2024-03-29T04:19:12.030 INFO:teuthology.orchestra.run.smithi151.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T04:19:12.030 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0d brd ff:ff:ff:ff:ff:ff 2024-03-29T04:19:12.030 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f1 2024-03-29T04:19:12.030 INFO:teuthology.orchestra.run.smithi151.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T04:19:12.030 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:62 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:19:12.030 INFO:teuthology.orchestra.run.smithi151.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:19:12.030 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:63 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:19:12.030 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.21.15.151/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-29T04:19:12.030 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft 42407sec preferred_lft 42407sec 2024-03-29T04:19:12.030 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::ec4:7aff:fe88:6f63/64 scope link 2024-03-29T04:19:12.030 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:19:12.030 INFO:teuthology.orchestra.run.smithi151.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T04:19:12.030 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 02:42:63:20:ce:96 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:19:12.031 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T04:19:12.031 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:19:12.031 INFO:teuthology.orchestra.run.smithi151.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:19:12.031 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-29T04:19:12.031 INFO:teuthology.orchestra.run.smithi151.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T04:19:12.031 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:19:12.031 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-29T04:19:12.031 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:19:12.031 INFO:teuthology.orchestra.run.smithi151.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:19:12.031 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:19:12.031 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-29T04:19:12.031 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:19:12.031 INFO:teuthology.orchestra.run.smithi151.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:19:12.031 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:19:12.032 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-29T04:19:12.032 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:19:12.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:12.032 DEBUG:teuthology.orchestra.run.smithi151:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:19:12.085 INFO:teuthology.orchestra.run.smithi151.stdout:1 2024-03-29T04:19:12.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:12.087 DEBUG:teuthology.orchestra.run.smithi151:> route 2024-03-29T04:19:12.140 INFO:teuthology.orchestra.run.smithi151.stdout:Kernel IP routing table 2024-03-29T04:19:12.140 INFO:teuthology.orchestra.run.smithi151.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:19:12.140 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-29T04:19:12.140 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:19:12.140 INFO:teuthology.orchestra.run.smithi151.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-29T04:19:12.141 INFO:teuthology.orchestra.run.smithi151.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T04:19:12.141 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:19:12.141 INFO:teuthology.orchestra.run.smithi151.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-29T04:19:12.141 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-29T04:19:12.141 INFO:teuthology.orchestra.run.smithi151.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-29T04:19:12.141 INFO:teuthology.orchestra.run.smithi151.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:19:12.142 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:19:12.142 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-03-29T04:19:12.142 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:19:12.143 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:19:12.143 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:19:12.143 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-03-29T04:19:12.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:19:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:19:12.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:19:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:19:12.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:12.263 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-03-29T04:19:12.269 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-29T04:19:12.269 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-29T04:19:12.269 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:19:12.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:12.269 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set brx.0 up 2024-03-29T04:19:12.325 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-03-29T04:19:12.325 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:19:12.325 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-29T04:19:12.374 INFO:teuthology.orchestra.run.smithi151.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:19:12.374 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:19:12.374 DEBUG:teuthology.orchestra.run.smithi151:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-29T04:19:12.422 DEBUG:teuthology.orchestra.run.smithi151:> sudo modprobe fuse 2024-03-29T04:19:12.478 DEBUG:teuthology.orchestra.run.smithi151:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:19:12.527 INFO:teuthology.orchestra.run.smithi151.stdout:/sys 2024-03-29T04:19:12.527 INFO:teuthology.orchestra.run.smithi151.stdout:/proc 2024-03-29T04:19:12.527 INFO:teuthology.orchestra.run.smithi151.stdout:/dev 2024-03-29T04:19:12.527 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/pts 2024-03-29T04:19:12.527 INFO:teuthology.orchestra.run.smithi151.stdout:/run 2024-03-29T04:19:12.527 INFO:teuthology.orchestra.run.smithi151.stdout:/ 2024-03-29T04:19:12.527 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/security 2024-03-29T04:19:12.527 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/shm 2024-03-29T04:19:12.527 INFO:teuthology.orchestra.run.smithi151.stdout:/run/lock 2024-03-29T04:19:12.527 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/cgroup 2024-03-29T04:19:12.527 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/pstore 2024-03-29T04:19:12.528 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/bpf 2024-03-29T04:19:12.528 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:19:12.528 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/hugepages 2024-03-29T04:19:12.528 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/mqueue 2024-03-29T04:19:12.528 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/debug 2024-03-29T04:19:12.528 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/tracing 2024-03-29T04:19:12.528 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/fs/nfsd 2024-03-29T04:19:12.528 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/fuse/connections 2024-03-29T04:19:12.528 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/config 2024-03-29T04:19:12.528 INFO:teuthology.orchestra.run.smithi151.stdout:/run/credentials/systemd-sysusers.service 2024-03-29T04:19:12.528 INFO:teuthology.orchestra.run.smithi151.stdout:/run/rpc_pipefs 2024-03-29T04:19:12.528 INFO:teuthology.orchestra.run.smithi151.stdout:/run/user/1000 2024-03-29T04:19:12.528 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph 2024-03-29T04:19:12.528 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-4 2024-03-29T04:19:12.529 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-5 2024-03-29T04:19:12.529 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-6 2024-03-29T04:19:12.529 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-7 2024-03-29T04:19:12.529 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns 2024-03-29T04:19:12.529 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:19:12.529 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:19:12.529 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:19:12.529 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:19:12.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:12.530 DEBUG:teuthology.orchestra.run.smithi151:> ls /sys/fs/fuse/connections 2024-03-29T04:19:12.578 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T04:19:12.578 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-03-29T04:19:12.627 DEBUG:teuthology.orchestra.run.smithi151:> sudo modprobe fuse 2024-03-29T04:19:12.642 DEBUG:teuthology.orchestra.run.smithi151:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:19:12.690 INFO:teuthology.orchestra.run.smithi151.stdout:/sys 2024-03-29T04:19:12.690 INFO:teuthology.orchestra.run.smithi151.stdout:/proc 2024-03-29T04:19:12.690 INFO:teuthology.orchestra.run.smithi151.stdout:/dev 2024-03-29T04:19:12.690 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/pts 2024-03-29T04:19:12.690 INFO:teuthology.orchestra.run.smithi151.stdout:/run 2024-03-29T04:19:12.690 INFO:teuthology.orchestra.run.smithi151.stdout:/ 2024-03-29T04:19:12.690 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/security 2024-03-29T04:19:12.690 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/shm 2024-03-29T04:19:12.691 INFO:teuthology.orchestra.run.smithi151.stdout:/run/lock 2024-03-29T04:19:12.691 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/cgroup 2024-03-29T04:19:12.691 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/pstore 2024-03-29T04:19:12.691 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/bpf 2024-03-29T04:19:12.691 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:19:12.691 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/hugepages 2024-03-29T04:19:12.691 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/mqueue 2024-03-29T04:19:12.691 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/debug 2024-03-29T04:19:12.691 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/tracing 2024-03-29T04:19:12.691 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/fs/nfsd 2024-03-29T04:19:12.691 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/fuse/connections 2024-03-29T04:19:12.691 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/config 2024-03-29T04:19:12.691 INFO:teuthology.orchestra.run.smithi151.stdout:/run/credentials/systemd-sysusers.service 2024-03-29T04:19:12.691 INFO:teuthology.orchestra.run.smithi151.stdout:/run/rpc_pipefs 2024-03-29T04:19:12.692 INFO:teuthology.orchestra.run.smithi151.stdout:/run/user/1000 2024-03-29T04:19:12.692 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph 2024-03-29T04:19:12.692 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-4 2024-03-29T04:19:12.692 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-5 2024-03-29T04:19:12.692 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-6 2024-03-29T04:19:12.692 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-7 2024-03-29T04:19:12.692 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns 2024-03-29T04:19:12.692 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:19:12.692 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:19:12.692 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:19:12.692 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:19:12.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:12.693 DEBUG:teuthology.orchestra.run.smithi151:> ls /sys/fs/fuse/connections 2024-03-29T04:19:12.708 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi151.stderr:2024-03-29T04:19:12.726+0000 7fcc97e32580 -1 init, newargv = 0x555a3631c890 newargc=15 2024-03-29T04:19:12.708 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi151.stderr:ceph-fuse[34129]: starting ceph client 2024-03-29T04:19:12.715 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi151.stderr:ceph-fuse[34129]: starting fuse 2024-03-29T04:19:13.814 DEBUG:teuthology.orchestra.run.smithi151:> sudo modprobe fuse 2024-03-29T04:19:13.830 DEBUG:teuthology.orchestra.run.smithi151:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:19:13.880 INFO:teuthology.orchestra.run.smithi151.stdout:/sys 2024-03-29T04:19:13.880 INFO:teuthology.orchestra.run.smithi151.stdout:/proc 2024-03-29T04:19:13.880 INFO:teuthology.orchestra.run.smithi151.stdout:/dev 2024-03-29T04:19:13.880 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/pts 2024-03-29T04:19:13.880 INFO:teuthology.orchestra.run.smithi151.stdout:/run 2024-03-29T04:19:13.881 INFO:teuthology.orchestra.run.smithi151.stdout:/ 2024-03-29T04:19:13.881 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/security 2024-03-29T04:19:13.881 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/shm 2024-03-29T04:19:13.881 INFO:teuthology.orchestra.run.smithi151.stdout:/run/lock 2024-03-29T04:19:13.881 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/cgroup 2024-03-29T04:19:13.881 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/pstore 2024-03-29T04:19:13.881 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/bpf 2024-03-29T04:19:13.881 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:19:13.881 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/hugepages 2024-03-29T04:19:13.881 INFO:teuthology.orchestra.run.smithi151.stdout:/dev/mqueue 2024-03-29T04:19:13.881 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/debug 2024-03-29T04:19:13.881 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/tracing 2024-03-29T04:19:13.881 INFO:teuthology.orchestra.run.smithi151.stdout:/proc/fs/nfsd 2024-03-29T04:19:13.881 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/fs/fuse/connections 2024-03-29T04:19:13.881 INFO:teuthology.orchestra.run.smithi151.stdout:/sys/kernel/config 2024-03-29T04:19:13.882 INFO:teuthology.orchestra.run.smithi151.stdout:/run/credentials/systemd-sysusers.service 2024-03-29T04:19:13.882 INFO:teuthology.orchestra.run.smithi151.stdout:/run/rpc_pipefs 2024-03-29T04:19:13.882 INFO:teuthology.orchestra.run.smithi151.stdout:/run/user/1000 2024-03-29T04:19:13.882 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph 2024-03-29T04:19:13.882 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-4 2024-03-29T04:19:13.882 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-5 2024-03-29T04:19:13.882 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-6 2024-03-29T04:19:13.882 INFO:teuthology.orchestra.run.smithi151.stdout:/var/lib/ceph/osd/ceph-7 2024-03-29T04:19:13.882 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns 2024-03-29T04:19:13.882 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:19:13.882 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:19:13.882 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:19:13.882 INFO:teuthology.orchestra.run.smithi151.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:19:13.882 INFO:teuthology.orchestra.run.smithi151.stdout:/home/ubuntu/cephtest/mnt.1 2024-03-29T04:19:13.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:13.883 DEBUG:teuthology.orchestra.run.smithi151:> ls /sys/fs/fuse/connections 2024-03-29T04:19:13.930 INFO:teuthology.orchestra.run.smithi151.stdout:45 2024-03-29T04:19:13.930 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-03-29T04:19:13.930 DEBUG:teuthology.orchestra.run.smithi151:> sudo stdin-killer -- python3 -c ' 2024-03-29T04:19:13.931 DEBUG:teuthology.orchestra.run.smithi151:> import glob 2024-03-29T04:19:13.931 DEBUG:teuthology.orchestra.run.smithi151:> import re 2024-03-29T04:19:13.931 DEBUG:teuthology.orchestra.run.smithi151:> import os 2024-03-29T04:19:13.931 DEBUG:teuthology.orchestra.run.smithi151:> import subprocess 2024-03-29T04:19:13.931 DEBUG:teuthology.orchestra.run.smithi151:> 2024-03-29T04:19:13.931 DEBUG:teuthology.orchestra.run.smithi151:> def _find_admin_socket(client_name): 2024-03-29T04:19:13.931 DEBUG:teuthology.orchestra.run.smithi151:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-03-29T04:19:13.931 DEBUG:teuthology.orchestra.run.smithi151:> files = glob.glob(asok_path) 2024-03-29T04:19:13.931 DEBUG:teuthology.orchestra.run.smithi151:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-03-29T04:19:13.931 DEBUG:teuthology.orchestra.run.smithi151:> 2024-03-29T04:19:13.931 DEBUG:teuthology.orchestra.run.smithi151:> # Given a non-glob path, it better be there 2024-03-29T04:19:13.931 DEBUG:teuthology.orchestra.run.smithi151:> if "*" not in asok_path: 2024-03-29T04:19:13.931 DEBUG:teuthology.orchestra.run.smithi151:> assert(len(files) == 1) 2024-03-29T04:19:13.931 DEBUG:teuthology.orchestra.run.smithi151:> return files[0] 2024-03-29T04:19:13.931 DEBUG:teuthology.orchestra.run.smithi151:> 2024-03-29T04:19:13.931 DEBUG:teuthology.orchestra.run.smithi151:> for f in files: 2024-03-29T04:19:13.931 DEBUG:teuthology.orchestra.run.smithi151:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T04:19:13.932 DEBUG:teuthology.orchestra.run.smithi151:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T04:19:13.932 DEBUG:teuthology.orchestra.run.smithi151:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T04:19:13.932 DEBUG:teuthology.orchestra.run.smithi151:> contents = proc_f.read() 2024-03-29T04:19:13.932 DEBUG:teuthology.orchestra.run.smithi151:> if mountpoint in contents: 2024-03-29T04:19:13.932 DEBUG:teuthology.orchestra.run.smithi151:> return f 2024-03-29T04:19:13.932 DEBUG:teuthology.orchestra.run.smithi151:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T04:19:13.932 DEBUG:teuthology.orchestra.run.smithi151:> 2024-03-29T04:19:13.932 DEBUG:teuthology.orchestra.run.smithi151:> print(_find_admin_socket("client.1")) 2024-03-29T04:19:13.932 DEBUG:teuthology.orchestra.run.smithi151:> ' 2024-03-29T04:19:14.052 INFO:teuthology.orchestra.run.smithi151.stdout:/var/run/ceph/ceph-client.1.34129.asok 2024-03-29T04:19:14.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-29T04:19:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:19:14.064 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34129.asok 2024-03-29T04:19:14.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:14.065 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34129.asok status 2024-03-29T04:19:14.213 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-03-29T04:19:14.214 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": { 2024-03-29T04:19:14.214 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_sha1": "69e2cd73c52284dfd5096c66b60c6e6b26a3c447", 2024-03-29T04:19:14.214 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_version": "ceph version 19.0.0-2520-g69e2cd73 (69e2cd73c52284dfd5096c66b60c6e6b26a3c447) squid (dev)", 2024-03-29T04:19:14.214 INFO:teuthology.orchestra.run.smithi151.stdout: "entity_id": "1", 2024-03-29T04:19:14.214 INFO:teuthology.orchestra.run.smithi151.stdout: "hostname": "smithi151", 2024-03-29T04:19:14.214 INFO:teuthology.orchestra.run.smithi151.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-03-29T04:19:14.214 INFO:teuthology.orchestra.run.smithi151.stdout: "pid": "34129", 2024-03-29T04:19:14.214 INFO:teuthology.orchestra.run.smithi151.stdout: "root": "/" 2024-03-29T04:19:14.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-03-29T04:19:14.214 INFO:teuthology.orchestra.run.smithi151.stdout: "dentry_count": 0, 2024-03-29T04:19:14.214 INFO:teuthology.orchestra.run.smithi151.stdout: "dentry_pinned_count": 0, 2024-03-29T04:19:14.214 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 5497, 2024-03-29T04:19:14.215 INFO:teuthology.orchestra.run.smithi151.stdout: "inst": { 2024-03-29T04:19:14.215 INFO:teuthology.orchestra.run.smithi151.stdout: "name": { 2024-03-29T04:19:14.215 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "client", 2024-03-29T04:19:14.215 INFO:teuthology.orchestra.run.smithi151.stdout: "num": 5497 2024-03-29T04:19:14.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-03-29T04:19:14.215 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": { 2024-03-29T04:19:14.215 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v1", 2024-03-29T04:19:14.215 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:19:14.215 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 2041372969 2024-03-29T04:19:14.215 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-03-29T04:19:14.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-03-29T04:19:14.215 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": { 2024-03-29T04:19:14.215 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v1", 2024-03-29T04:19:14.215 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:19:14.216 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 2041372969 2024-03-29T04:19:14.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-03-29T04:19:14.216 INFO:teuthology.orchestra.run.smithi151.stdout: "inst_str": "client.5497 192.168.0.1:0/2041372969", 2024-03-29T04:19:14.216 INFO:teuthology.orchestra.run.smithi151.stdout: "addr_str": "192.168.0.1:0/2041372969", 2024-03-29T04:19:14.216 INFO:teuthology.orchestra.run.smithi151.stdout: "inode_count": 1, 2024-03-29T04:19:14.216 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_epoch": 18, 2024-03-29T04:19:14.216 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_epoch": 40, 2024-03-29T04:19:14.216 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_epoch_barrier": 0, 2024-03-29T04:19:14.216 INFO:teuthology.orchestra.run.smithi151.stdout: "blocklisted": false, 2024-03-29T04:19:14.216 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs" 2024-03-29T04:19:14.216 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-03-29T04:19:14.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:14.227 DEBUG:teuthology.orchestra.run.smithi151:> stat --file-system '--printf=%T 2024-03-29T04:19:14.227 DEBUG:teuthology.orchestra.run.smithi151:> ' -- /home/ubuntu/cephtest/mnt.1 2024-03-29T04:19:14.233 INFO:teuthology.orchestra.run.smithi151.stdout:fuseblk 2024-03-29T04:19:14.234 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-03-29T04:19:14.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:14.234 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-29T04:19:14.291 DEBUG:tasks.cephfs.test_uninlining:setting inline_data:1 2024-03-29T04:19:14.291 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs inline_data 1 --yes-i-really-really-mean-it 2024-03-29T04:19:14.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.447+0000 7f408df0d640 1 Processor -- start 2024-03-29T04:19:14.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.447+0000 7f408df0d640 1 -- start start 2024-03-29T04:19:14.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.447+0000 7f408df0d640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f408813f030 0x7f408806ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:14.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.447+0000 7f408df0d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f408815cf10 0x7f40880cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:14.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.447+0000 7f408df0d640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4088142640 con 0x7f408806f110 2024-03-29T04:19:14.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.447+0000 7f408df0d640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f408813f410 con 0x7f408815cf10 2024-03-29T04:19:14.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.447+0000 7f408df0d640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f408815d5a0 con 0x7f408813f030 2024-03-29T04:19:14.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.447+0000 7f4087fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f408815cf10 0x7f40880cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:14.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.447+0000 7f4086ffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f408806f110 0x7f408815c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:57060/0 (socket says 172.21.15.22:57060) 2024-03-29T04:19:14.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.447+0000 7f4086ffd640 1 -- 172.21.15.22:0/1467611407 learned_addr learned my addr 172.21.15.22:0/1467611407 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:19:14.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.451+0000 7f40877fe640 1 --2- 172.21.15.22:0/1467611407 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f408813f030 0x7f408806ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:14.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.451+0000 7f40867fc640 1 -- 172.21.15.22:0/1467611407 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2858891268 0 0) 0x7f4088142640 con 0x7f408806f110 2024-03-29T04:19:14.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.451+0000 7f4087fff640 1 -- 172.21.15.22:0/1467611407 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f408813f030 msgr2=0x7f408806ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:14.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.451+0000 7f4087fff640 1 --2- 172.21.15.22:0/1467611407 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f408813f030 0x7f408806ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:14.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.451+0000 7f4087fff640 1 -- 172.21.15.22:0/1467611407 >> v1:172.21.15.22:6789/0 conn(0x7f408806f110 legacy=0x7f408815c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:14.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.451+0000 7f4087fff640 1 -- 172.21.15.22:0/1467611407 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4088141880 con 0x7f408815cf10 2024-03-29T04:19:14.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.451+0000 7f4087fff640 1 --2- 172.21.15.22:0/1467611407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f408815cf10 0x7f40880cf910 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4080009d90 tx=0x7f408001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7937606f605cb350 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:14.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.451+0000 7f40867fc640 1 -- 172.21.15.22:0/1467611407 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40800b2070 con 0x7f408815cf10 2024-03-29T04:19:14.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.451+0000 7f40867fc640 1 -- 172.21.15.22:0/1467611407 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40800a57a0 con 0x7f408815cf10 2024-03-29T04:19:14.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.451+0000 7f40867fc640 1 -- 172.21.15.22:0/1467611407 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40800a5b70 con 0x7f408815cf10 2024-03-29T04:19:14.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.455+0000 7f408df0d640 1 -- 172.21.15.22:0/1467611407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f408815cf10 msgr2=0x7f40880cf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:14.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.455+0000 7f408df0d640 1 --2- 172.21.15.22:0/1467611407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f408815cf10 0x7f40880cf910 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4080009d90 tx=0x7f408001c8e0 comp rx=0 tx=0).stop 2024-03-29T04:19:14.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.455+0000 7f408df0d640 1 -- 172.21.15.22:0/1467611407 shutdown_connections 2024-03-29T04:19:14.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.455+0000 7f408df0d640 1 --2- 172.21.15.22:0/1467611407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f408815cf10 0x7f40880cf910 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:14.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.455+0000 7f408df0d640 1 --2- 172.21.15.22:0/1467611407 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f408813f030 0x7f408806ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:14.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.455+0000 7f408df0d640 1 -- 172.21.15.22:0/1467611407 >> 172.21.15.22:0/1467611407 conn(0x7f40880b3460 msgr2=0x7f408806e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:14.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.455+0000 7f408df0d640 1 -- 172.21.15.22:0/1467611407 shutdown_connections 2024-03-29T04:19:14.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.455+0000 7f408df0d640 1 -- 172.21.15.22:0/1467611407 wait complete. 2024-03-29T04:19:14.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.455+0000 7f408df0d640 1 Processor -- start 2024-03-29T04:19:14.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.455+0000 7f408df0d640 1 -- start start 2024-03-29T04:19:14.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f408df0d640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f408806f110 0x7f4088140e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:14.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f408df0d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f408813f030 0x7f4088166400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:14.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f408df0d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f408815cf10 0x7f4088129f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:14.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f408df0d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f40880d0a30 con 0x7f408815cf10 2024-03-29T04:19:14.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f408df0d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f40880d08b0 con 0x7f408813f030 2024-03-29T04:19:14.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f408df0d640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f408806f510 con 0x7f408806f110 2024-03-29T04:19:14.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f4087fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f408815cf10 0x7f4088129f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:14.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f4087fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f408815cf10 0x7f4088129f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35974/0 (socket says 172.21.15.22:35974) 2024-03-29T04:19:14.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f4087fff640 1 -- 172.21.15.22:0/1094591983 learned_addr learned my addr 172.21.15.22:0/1094591983 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:14.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f40877fe640 1 --2- 172.21.15.22:0/1094591983 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f408806f110 0x7f4088140e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:14.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f4086ffd640 1 --2- 172.21.15.22:0/1094591983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f408813f030 0x7f4088166400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:14.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f4087fff640 1 -- 172.21.15.22:0/1094591983 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f408806f110 msgr2=0x7f4088140e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:14.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f4087fff640 1 --2- 172.21.15.22:0/1094591983 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f408806f110 0x7f4088140e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:14.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f4087fff640 1 -- 172.21.15.22:0/1094591983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f408813f030 msgr2=0x7f4088166400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:14.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f4087fff640 1 --2- 172.21.15.22:0/1094591983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f408813f030 0x7f4088166400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:14.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f4087fff640 1 -- 172.21.15.22:0/1094591983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40881413e0 con 0x7f408815cf10 2024-03-29T04:19:14.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f40877fe640 1 --2- 172.21.15.22:0/1094591983 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f408806f110 0x7f4088140e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:19:14.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f4086ffd640 1 --2- 172.21.15.22:0/1094591983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f408813f030 0x7f4088166400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:19:14.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f4087fff640 1 --2- 172.21.15.22:0/1094591983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f408815cf10 0x7f4088129f30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f40800b0040 tx=0x7f4080019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:14.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f407ffff640 1 -- 172.21.15.22:0/1094591983 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40800bd030 con 0x7f408815cf10 2024-03-29T04:19:14.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f407ffff640 1 -- 172.21.15.22:0/1094591983 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40800a55a0 con 0x7f408815cf10 2024-03-29T04:19:14.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f407ffff640 1 -- 172.21.15.22:0/1094591983 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40800b2040 con 0x7f408815cf10 2024-03-29T04:19:14.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f408df0d640 1 -- 172.21.15.22:0/1094591983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408806ba20 con 0x7f408815cf10 2024-03-29T04:19:14.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f408df0d640 1 -- 172.21.15.22:0/1094591983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f408806bfe0 con 0x7f408815cf10 2024-03-29T04:19:14.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.459+0000 7f408df0d640 1 -- 172.21.15.22:0/1094591983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40880687f0 con 0x7f408815cf10 2024-03-29T04:19:14.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.463+0000 7f407ffff640 1 -- 172.21.15.22:0/1094591983 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f40800c0070 con 0x7f408815cf10 2024-03-29T04:19:14.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.463+0000 7f407ffff640 1 --2- 172.21.15.22:0/1094591983 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4068067fc0 0x7f406806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:14.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.463+0000 7f407ffff640 1 -- 172.21.15.22:0/1094591983 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f40801214a0 con 0x7f408815cf10 2024-03-29T04:19:14.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.463+0000 7f40877fe640 1 --2- 172.21.15.22:0/1094591983 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4068067fc0 0x7f406806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:14.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.463+0000 7f40877fe640 1 --2- 172.21.15.22:0/1094591983 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4068067fc0 0x7f406806a480 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f40780097a0 tx=0x7f4078008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:14.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.467+0000 7f407ffff640 1 -- 172.21.15.22:0/1094591983 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f4080016560 con 0x7f408815cf10 2024-03-29T04:19:14.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.667+0000 7f408df0d640 1 -- 172.21.15.22:0/1094591983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "1", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4088153260 con 0x7f408815cf10 2024-03-29T04:19:14.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.799+0000 7f407ffff640 1 -- 172.21.15.22:0/1094591983 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "1", "yes_i_really_really_mean_it": true}]=0 inline data enabled v19) ==== 165+0+0 (secure 0 0 0) 0x7f40800a5770 con 0x7f408815cf10 2024-03-29T04:19:14.781 INFO:teuthology.orchestra.run.smithi022.stderr:inline data enabled 2024-03-29T04:19:14.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.803+0000 7f408df0d640 1 -- 172.21.15.22:0/1094591983 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4068067fc0 msgr2=0x7f406806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:14.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.803+0000 7f408df0d640 1 --2- 172.21.15.22:0/1094591983 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4068067fc0 0x7f406806a480 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f40780097a0 tx=0x7f4078008040 comp rx=0 tx=0).stop 2024-03-29T04:19:14.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.803+0000 7f408df0d640 1 -- 172.21.15.22:0/1094591983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f408815cf10 msgr2=0x7f4088129f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:14.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.803+0000 7f408df0d640 1 --2- 172.21.15.22:0/1094591983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f408815cf10 0x7f4088129f30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f40800b0040 tx=0x7f4080019680 comp rx=0 tx=0).stop 2024-03-29T04:19:14.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.803+0000 7f408df0d640 1 -- 172.21.15.22:0/1094591983 shutdown_connections 2024-03-29T04:19:14.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.803+0000 7f408df0d640 1 --2- 172.21.15.22:0/1094591983 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4068067fc0 0x7f406806a480 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:14.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.803+0000 7f408df0d640 1 --2- 172.21.15.22:0/1094591983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f408815cf10 0x7f4088129f30 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:14.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.803+0000 7f408df0d640 1 --2- 172.21.15.22:0/1094591983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f408813f030 0x7f4088166400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:14.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.803+0000 7f408df0d640 1 --2- 172.21.15.22:0/1094591983 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f408806f110 0x7f4088140e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:14.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.803+0000 7f408df0d640 1 -- 172.21.15.22:0/1094591983 >> 172.21.15.22:0/1094591983 conn(0x7f40880b3460 msgr2=0x7f408815adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:14.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.803+0000 7f408df0d640 1 -- 172.21.15.22:0/1094591983 shutdown_connections 2024-03-29T04:19:14.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:14.803+0000 7f408df0d640 1 -- 172.21.15.22:0/1094591983 wait complete. 2024-03-29T04:19:14.799 DEBUG:tasks.cephfs.test_uninlining:fetching tarball 2024-03-29T04:19:14.799 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && wget http://download.ceph.com/qa/linux-5.4.tar.gz 2024-03-29T04:19:14.827 INFO:teuthology.orchestra.run.smithi022.stderr:--2024-03-29 04:19:14-- http://download.ceph.com/qa/linux-5.4.tar.gz 2024-03-29T04:19:14.828 INFO:teuthology.orchestra.run.smithi022.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-03-29T04:19:14.850 INFO:teuthology.orchestra.run.smithi022.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-03-29T04:19:14.870 INFO:teuthology.orchestra.run.smithi022.stderr:HTTP request sent, awaiting response... 200 OK 2024-03-29T04:19:14.871 INFO:teuthology.orchestra.run.smithi022.stderr:Length: 172616875 (165M) [application/octet-stream] 2024-03-29T04:19:14.879 INFO:teuthology.orchestra.run.smithi022.stderr:Saving to: 'linux-5.4.tar.gz' 2024-03-29T04:19:14.884 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-03-29T04:19:14.913 INFO:teuthology.orchestra.run.smithi022.stderr: 0K .......... .......... .......... .......... .......... 0% 1.46M 1m53s 2024-03-29T04:19:14.933 INFO:teuthology.orchestra.run.smithi022.stderr: 50K .......... .......... .......... .......... .......... 0% 2.47M 90s 2024-03-29T04:19:14.938 INFO:teuthology.orchestra.run.smithi022.stderr: 100K .......... .......... .......... .......... .......... 0% 9.16M 66s 2024-03-29T04:19:14.956 INFO:teuthology.orchestra.run.smithi022.stderr: 150K .......... .......... .......... .......... .......... 0% 2.69M 65s 2024-03-29T04:19:14.959 INFO:teuthology.orchestra.run.smithi022.stderr: 200K .......... .......... .......... .......... .......... 0% 14.3M 54s 2024-03-29T04:19:14.963 INFO:teuthology.orchestra.run.smithi022.stderr: 250K .......... .......... .......... .......... .......... 0% 15.0M 47s 2024-03-29T04:19:14.974 INFO:teuthology.orchestra.run.smithi022.stderr: 300K .......... .......... .......... .......... .......... 0% 4.44M 45s 2024-03-29T04:19:14.983 INFO:teuthology.orchestra.run.smithi022.stderr: 350K .......... .......... .......... .......... .......... 0% 10.5M 42s 2024-03-29T04:19:14.983 INFO:teuthology.orchestra.run.smithi022.stderr: 400K .......... .......... .......... .......... .......... 0% 13.0M 38s 2024-03-29T04:19:14.985 INFO:teuthology.orchestra.run.smithi022.stderr: 450K .......... .......... .......... .......... .......... 0% 16.1M 36s 2024-03-29T04:19:14.988 INFO:teuthology.orchestra.run.smithi022.stderr: 500K .......... .......... .......... .......... .......... 0% 16.3M 33s 2024-03-29T04:19:14.991 INFO:teuthology.orchestra.run.smithi022.stderr: 550K .......... .......... .......... .......... .......... 0% 16.1M 31s 2024-03-29T04:19:14.997 INFO:teuthology.orchestra.run.smithi022.stderr: 600K .......... .......... .......... .......... .......... 0% 8.92M 30s 2024-03-29T04:19:15.002 INFO:teuthology.orchestra.run.smithi022.stderr: 650K .......... .......... .......... .......... .......... 0% 10.1M 29s 2024-03-29T04:19:15.006 INFO:teuthology.orchestra.run.smithi022.stderr: 700K .......... .......... .......... .......... .......... 0% 10.8M 28s 2024-03-29T04:19:15.012 INFO:teuthology.orchestra.run.smithi022.stderr: 750K .......... .......... .......... .......... .......... 0% 9.15M 28s 2024-03-29T04:19:15.016 INFO:teuthology.orchestra.run.smithi022.stderr: 800K .......... .......... .......... .......... .......... 0% 10.2M 27s 2024-03-29T04:19:15.021 INFO:teuthology.orchestra.run.smithi022.stderr: 850K .......... .......... .......... .......... .......... 0% 10.5M 26s 2024-03-29T04:19:15.026 INFO:teuthology.orchestra.run.smithi022.stderr: 900K .......... .......... .......... .......... .......... 0% 10.6M 26s 2024-03-29T04:19:15.031 INFO:teuthology.orchestra.run.smithi022.stderr: 950K .......... .......... .......... .......... .......... 0% 8.72M 25s 2024-03-29T04:19:15.036 INFO:teuthology.orchestra.run.smithi022.stderr: 1000K .......... .......... .......... .......... .......... 0% 10.3M 25s 2024-03-29T04:19:15.041 INFO:teuthology.orchestra.run.smithi022.stderr: 1050K .......... .......... .......... .......... .......... 0% 10.1M 25s 2024-03-29T04:19:15.046 INFO:teuthology.orchestra.run.smithi022.stderr: 1100K .......... .......... .......... .......... .......... 0% 9.84M 24s 2024-03-29T04:19:15.050 INFO:teuthology.orchestra.run.smithi022.stderr: 1150K .......... .......... .......... .......... .......... 0% 11.5M 24s 2024-03-29T04:19:15.053 INFO:teuthology.orchestra.run.smithi022.stderr: 1200K .......... .......... .......... .......... .......... 0% 14.4M 23s 2024-03-29T04:19:15.057 INFO:teuthology.orchestra.run.smithi022.stderr: 1250K .......... .......... .......... .......... .......... 0% 12.9M 23s 2024-03-29T04:19:15.061 INFO:teuthology.orchestra.run.smithi022.stderr: 1300K .......... .......... .......... .......... .......... 0% 12.6M 22s 2024-03-29T04:19:15.065 INFO:teuthology.orchestra.run.smithi022.stderr: 1350K .......... .......... .......... .......... .......... 0% 10.7M 22s 2024-03-29T04:19:15.069 INFO:teuthology.orchestra.run.smithi022.stderr: 1400K .......... .......... .......... .......... .......... 0% 11.7M 22s 2024-03-29T04:19:15.073 INFO:teuthology.orchestra.run.smithi022.stderr: 1450K .......... .......... .......... .......... .......... 0% 12.4M 22s 2024-03-29T04:19:15.077 INFO:teuthology.orchestra.run.smithi022.stderr: 1500K .......... .......... .......... .......... .......... 0% 11.8M 21s 2024-03-29T04:19:15.082 INFO:teuthology.orchestra.run.smithi022.stderr: 1550K .......... .......... .......... .......... .......... 0% 10.9M 21s 2024-03-29T04:19:15.086 INFO:teuthology.orchestra.run.smithi022.stderr: 1600K .......... .......... .......... .......... .......... 0% 12.3M 21s 2024-03-29T04:19:15.090 INFO:teuthology.orchestra.run.smithi022.stderr: 1650K .......... .......... .......... .......... .......... 1% 11.4M 21s 2024-03-29T04:19:15.094 INFO:teuthology.orchestra.run.smithi022.stderr: 1700K .......... .......... .......... .......... .......... 1% 12.7M 20s 2024-03-29T04:19:15.099 INFO:teuthology.orchestra.run.smithi022.stderr: 1750K .......... .......... .......... .......... .......... 1% 10.6M 20s 2024-03-29T04:19:15.102 INFO:teuthology.orchestra.run.smithi022.stderr: 1800K .......... .......... .......... .......... .......... 1% 14.0M 20s 2024-03-29T04:19:15.106 INFO:teuthology.orchestra.run.smithi022.stderr: 1850K .......... .......... .......... .......... .......... 1% 14.2M 20s 2024-03-29T04:19:15.109 INFO:teuthology.orchestra.run.smithi022.stderr: 1900K .......... .......... .......... .......... .......... 1% 13.8M 20s 2024-03-29T04:19:15.113 INFO:teuthology.orchestra.run.smithi022.stderr: 1950K .......... .......... .......... .......... .......... 1% 12.0M 19s 2024-03-29T04:19:15.117 INFO:teuthology.orchestra.run.smithi022.stderr: 2000K .......... .......... .......... .......... .......... 1% 15.1M 19s 2024-03-29T04:19:15.120 INFO:teuthology.orchestra.run.smithi022.stderr: 2050K .......... .......... .......... .......... .......... 1% 13.0M 19s 2024-03-29T04:19:15.124 INFO:teuthology.orchestra.run.smithi022.stderr: 2100K .......... .......... .......... .......... .......... 1% 14.4M 19s 2024-03-29T04:19:15.127 INFO:teuthology.orchestra.run.smithi022.stderr: 2150K .......... .......... .......... .......... .......... 1% 13.3M 19s 2024-03-29T04:19:15.131 INFO:teuthology.orchestra.run.smithi022.stderr: 2200K .......... .......... .......... .......... .......... 1% 15.3M 19s 2024-03-29T04:19:15.134 INFO:teuthology.orchestra.run.smithi022.stderr: 2250K .......... .......... .......... .......... .......... 1% 14.9M 18s 2024-03-29T04:19:15.137 INFO:teuthology.orchestra.run.smithi022.stderr: 2300K .......... .......... .......... .......... .......... 1% 14.5M 18s 2024-03-29T04:19:15.141 INFO:teuthology.orchestra.run.smithi022.stderr: 2350K .......... .......... .......... .......... .......... 1% 13.2M 18s 2024-03-29T04:19:15.144 INFO:teuthology.orchestra.run.smithi022.stderr: 2400K .......... .......... .......... .......... .......... 1% 14.5M 18s 2024-03-29T04:19:15.147 INFO:teuthology.orchestra.run.smithi022.stderr: 2450K .......... .......... .......... .......... .......... 1% 16.3M 18s 2024-03-29T04:19:15.150 INFO:teuthology.orchestra.run.smithi022.stderr: 2500K .......... .......... .......... .......... .......... 1% 15.9M 18s 2024-03-29T04:19:15.154 INFO:teuthology.orchestra.run.smithi022.stderr: 2550K .......... .......... .......... .......... .......... 1% 13.4M 18s 2024-03-29T04:19:15.157 INFO:teuthology.orchestra.run.smithi022.stderr: 2600K .......... .......... .......... .......... .......... 1% 16.8M 17s 2024-03-29T04:19:15.160 INFO:teuthology.orchestra.run.smithi022.stderr: 2650K .......... .......... .......... .......... .......... 1% 15.1M 17s 2024-03-29T04:19:15.163 INFO:teuthology.orchestra.run.smithi022.stderr: 2700K .......... .......... .......... .......... .......... 1% 15.9M 17s 2024-03-29T04:19:15.168 INFO:teuthology.orchestra.run.smithi022.stderr: 2750K .......... .......... .......... .......... .......... 1% 11.2M 17s 2024-03-29T04:19:15.171 INFO:teuthology.orchestra.run.smithi022.stderr: 2800K .......... .......... .......... .......... .......... 1% 13.6M 17s 2024-03-29T04:19:15.175 INFO:teuthology.orchestra.run.smithi022.stderr: 2850K .......... .......... .......... .......... .......... 1% 12.5M 17s 2024-03-29T04:19:15.179 INFO:teuthology.orchestra.run.smithi022.stderr: 2900K .......... .......... .......... .......... .......... 1% 12.7M 17s 2024-03-29T04:19:15.187 INFO:teuthology.orchestra.run.smithi022.stderr: 2950K .......... .......... .......... .......... .......... 1% 10.6M 17s 2024-03-29T04:19:15.187 INFO:teuthology.orchestra.run.smithi022.stderr: 3000K .......... .......... .......... .......... .......... 1% 12.5M 17s 2024-03-29T04:19:15.191 INFO:teuthology.orchestra.run.smithi022.stderr: 3050K .......... .......... .......... .......... .......... 1% 12.7M 17s 2024-03-29T04:19:15.195 INFO:teuthology.orchestra.run.smithi022.stderr: 3100K .......... .......... .......... .......... .......... 1% 12.0M 17s 2024-03-29T04:19:15.200 INFO:teuthology.orchestra.run.smithi022.stderr: 3150K .......... .......... .......... .......... .......... 1% 10.7M 17s 2024-03-29T04:19:15.204 INFO:teuthology.orchestra.run.smithi022.stderr: 3200K .......... .......... .......... .......... .......... 1% 13.2M 16s 2024-03-29T04:19:15.208 INFO:teuthology.orchestra.run.smithi022.stderr: 3250K .......... .......... .......... .......... .......... 1% 12.4M 16s 2024-03-29T04:19:15.211 INFO:teuthology.orchestra.run.smithi022.stderr: 3300K .......... .......... .......... .......... .......... 1% 12.4M 16s 2024-03-29T04:19:15.216 INFO:teuthology.orchestra.run.smithi022.stderr: 3350K .......... .......... .......... .......... .......... 2% 11.6M 16s 2024-03-29T04:19:15.220 INFO:teuthology.orchestra.run.smithi022.stderr: 3400K .......... .......... .......... .......... .......... 2% 12.9M 16s 2024-03-29T04:19:15.223 INFO:teuthology.orchestra.run.smithi022.stderr: 3450K .......... .......... .......... .......... .......... 2% 13.4M 16s 2024-03-29T04:19:15.227 INFO:teuthology.orchestra.run.smithi022.stderr: 3500K .......... .......... .......... .......... .......... 2% 13.4M 16s 2024-03-29T04:19:15.232 INFO:teuthology.orchestra.run.smithi022.stderr: 3550K .......... .......... .......... .......... .......... 2% 10.6M 16s 2024-03-29T04:19:15.236 INFO:teuthology.orchestra.run.smithi022.stderr: 3600K .......... .......... .......... .......... .......... 2% 10.9M 16s 2024-03-29T04:19:15.240 INFO:teuthology.orchestra.run.smithi022.stderr: 3650K .......... .......... .......... .......... .......... 2% 11.0M 16s 2024-03-29T04:19:15.245 INFO:teuthology.orchestra.run.smithi022.stderr: 3700K .......... .......... .......... .......... .......... 2% 10.7M 16s 2024-03-29T04:19:15.250 INFO:teuthology.orchestra.run.smithi022.stderr: 3750K .......... .......... .......... .......... .......... 2% 9.40M 16s 2024-03-29T04:19:15.255 INFO:teuthology.orchestra.run.smithi022.stderr: 3800K .......... .......... .......... .......... .......... 2% 11.0M 16s 2024-03-29T04:19:15.259 INFO:teuthology.orchestra.run.smithi022.stderr: 3850K .......... .......... .......... .......... .......... 2% 11.2M 16s 2024-03-29T04:19:15.263 INFO:teuthology.orchestra.run.smithi022.stderr: 3900K .......... .......... .......... .......... .......... 2% 11.1M 16s 2024-03-29T04:19:15.269 INFO:teuthology.orchestra.run.smithi022.stderr: 3950K .......... .......... .......... .......... .......... 2% 10.0M 16s 2024-03-29T04:19:15.272 INFO:teuthology.orchestra.run.smithi022.stderr: 4000K .......... .......... .......... .......... .......... 2% 11.9M 16s 2024-03-29T04:19:15.276 INFO:teuthology.orchestra.run.smithi022.stderr: 4050K .......... .......... .......... .......... .......... 2% 13.5M 16s 2024-03-29T04:19:15.279 INFO:teuthology.orchestra.run.smithi022.stderr: 4100K .......... .......... .......... .......... .......... 2% 18.4M 16s 2024-03-29T04:19:15.281 INFO:teuthology.orchestra.run.smithi022.stderr: 4150K .......... .......... .......... .......... .......... 2% 16.4M 16s 2024-03-29T04:19:15.285 INFO:teuthology.orchestra.run.smithi022.stderr: 4200K .......... .......... .......... .......... .......... 2% 18.0M 16s 2024-03-29T04:19:15.287 INFO:teuthology.orchestra.run.smithi022.stderr: 4250K .......... .......... .......... .......... .......... 2% 18.4M 16s 2024-03-29T04:19:15.290 INFO:teuthology.orchestra.run.smithi022.stderr: 4300K .......... .......... .......... .......... .......... 2% 17.3M 15s 2024-03-29T04:19:15.293 INFO:teuthology.orchestra.run.smithi022.stderr: 4350K .......... .......... .......... .......... .......... 2% 14.5M 15s 2024-03-29T04:19:15.295 INFO:teuthology.orchestra.run.smithi022.stderr: 4400K .......... .......... .......... .......... .......... 2% 18.1M 15s 2024-03-29T04:19:15.298 INFO:teuthology.orchestra.run.smithi022.stderr: 4450K .......... .......... .......... .......... .......... 2% 17.8M 15s 2024-03-29T04:19:15.302 INFO:teuthology.orchestra.run.smithi022.stderr: 4500K .......... .......... .......... .......... .......... 2% 15.7M 15s 2024-03-29T04:19:15.306 INFO:teuthology.orchestra.run.smithi022.stderr: 4550K .......... .......... .......... .......... .......... 2% 11.8M 15s 2024-03-29T04:19:15.309 INFO:teuthology.orchestra.run.smithi022.stderr: 4600K .......... .......... .......... .......... .......... 2% 14.5M 15s 2024-03-29T04:19:15.312 INFO:teuthology.orchestra.run.smithi022.stderr: 4650K .......... .......... .......... .......... .......... 2% 13.8M 15s 2024-03-29T04:19:15.316 INFO:teuthology.orchestra.run.smithi022.stderr: 4700K .......... .......... .......... .......... .......... 2% 15.1M 15s 2024-03-29T04:19:15.319 INFO:teuthology.orchestra.run.smithi022.stderr: 4750K .......... .......... .......... .......... .......... 2% 13.2M 15s 2024-03-29T04:19:15.323 INFO:teuthology.orchestra.run.smithi022.stderr: 4800K .......... .......... .......... .......... .......... 2% 15.6M 15s 2024-03-29T04:19:15.326 INFO:teuthology.orchestra.run.smithi022.stderr: 4850K .......... .......... .......... .......... .......... 2% 15.3M 15s 2024-03-29T04:19:15.331 INFO:teuthology.orchestra.run.smithi022.stderr: 4900K .......... .......... .......... .......... .......... 2% 15.5M 15s 2024-03-29T04:19:15.332 INFO:teuthology.orchestra.run.smithi022.stderr: 4950K .......... .......... .......... .......... .......... 2% 13.4M 15s 2024-03-29T04:19:15.336 INFO:teuthology.orchestra.run.smithi022.stderr: 5000K .......... .......... .......... .......... .......... 2% 15.4M 15s 2024-03-29T04:19:15.339 INFO:teuthology.orchestra.run.smithi022.stderr: 5050K .......... .......... .......... .......... .......... 3% 15.1M 15s 2024-03-29T04:19:15.343 INFO:teuthology.orchestra.run.smithi022.stderr: 5100K .......... .......... .......... .......... .......... 3% 12.8M 15s 2024-03-29T04:19:15.347 INFO:teuthology.orchestra.run.smithi022.stderr: 5150K .......... .......... .......... .......... .......... 3% 12.4M 15s 2024-03-29T04:19:15.350 INFO:teuthology.orchestra.run.smithi022.stderr: 5200K .......... .......... .......... .......... .......... 3% 16.6M 15s 2024-03-29T04:19:15.352 INFO:teuthology.orchestra.run.smithi022.stderr: 5250K .......... .......... .......... .......... .......... 3% 16.7M 15s 2024-03-29T04:19:15.355 INFO:teuthology.orchestra.run.smithi022.stderr: 5300K .......... .......... .......... .......... .......... 3% 16.5M 15s 2024-03-29T04:19:15.359 INFO:teuthology.orchestra.run.smithi022.stderr: 5350K .......... .......... .......... .......... .......... 3% 14.0M 14s 2024-03-29T04:19:15.362 INFO:teuthology.orchestra.run.smithi022.stderr: 5400K .......... .......... .......... .......... .......... 3% 16.3M 14s 2024-03-29T04:19:15.365 INFO:teuthology.orchestra.run.smithi022.stderr: 5450K .......... .......... .......... .......... .......... 3% 16.3M 14s 2024-03-29T04:19:15.368 INFO:teuthology.orchestra.run.smithi022.stderr: 5500K .......... .......... .......... .......... .......... 3% 16.0M 14s 2024-03-29T04:19:15.373 INFO:teuthology.orchestra.run.smithi022.stderr: 5550K .......... .......... .......... .......... .......... 3% 10.7M 14s 2024-03-29T04:19:15.377 INFO:teuthology.orchestra.run.smithi022.stderr: 5600K .......... .......... .......... .......... .......... 3% 12.3M 14s 2024-03-29T04:19:15.381 INFO:teuthology.orchestra.run.smithi022.stderr: 5650K .......... .......... .......... .......... .......... 3% 12.0M 14s 2024-03-29T04:19:15.385 INFO:teuthology.orchestra.run.smithi022.stderr: 5700K .......... .......... .......... .......... .......... 3% 11.7M 14s 2024-03-29T04:19:15.390 INFO:teuthology.orchestra.run.smithi022.stderr: 5750K .......... .......... .......... .......... .......... 3% 10.2M 14s 2024-03-29T04:19:15.394 INFO:teuthology.orchestra.run.smithi022.stderr: 5800K .......... .......... .......... .......... .......... 3% 11.8M 14s 2024-03-29T04:19:15.398 INFO:teuthology.orchestra.run.smithi022.stderr: 5850K .......... .......... .......... .......... .......... 3% 11.9M 14s 2024-03-29T04:19:15.402 INFO:teuthology.orchestra.run.smithi022.stderr: 5900K .......... .......... .......... .......... .......... 3% 13.1M 14s 2024-03-29T04:19:15.406 INFO:teuthology.orchestra.run.smithi022.stderr: 5950K .......... .......... .......... .......... .......... 3% 11.5M 14s 2024-03-29T04:19:15.409 INFO:teuthology.orchestra.run.smithi022.stderr: 6000K .......... .......... .......... .......... .......... 3% 14.5M 14s 2024-03-29T04:19:15.413 INFO:teuthology.orchestra.run.smithi022.stderr: 6050K .......... .......... .......... .......... .......... 3% 14.5M 14s 2024-03-29T04:19:15.416 INFO:teuthology.orchestra.run.smithi022.stderr: 6100K .......... .......... .......... .......... .......... 3% 13.4M 14s 2024-03-29T04:19:15.421 INFO:teuthology.orchestra.run.smithi022.stderr: 6150K .......... .......... .......... .......... .......... 3% 11.0M 14s 2024-03-29T04:19:15.425 INFO:teuthology.orchestra.run.smithi022.stderr: 6200K .......... .......... .......... .......... .......... 3% 11.6M 14s 2024-03-29T04:19:15.430 INFO:teuthology.orchestra.run.smithi022.stderr: 6250K .......... .......... .......... .......... .......... 3% 11.0M 14s 2024-03-29T04:19:15.434 INFO:teuthology.orchestra.run.smithi022.stderr: 6300K .......... .......... .......... .......... .......... 3% 10.9M 14s 2024-03-29T04:19:15.438 INFO:teuthology.orchestra.run.smithi022.stderr: 6350K .......... .......... .......... .......... .......... 3% 10.9M 14s 2024-03-29T04:19:15.443 INFO:teuthology.orchestra.run.smithi022.stderr: 6400K .......... .......... .......... .......... .......... 3% 11.1M 14s 2024-03-29T04:19:15.448 INFO:teuthology.orchestra.run.smithi022.stderr: 6450K .......... .......... .......... .......... .......... 3% 10.2M 14s 2024-03-29T04:19:15.453 INFO:teuthology.orchestra.run.smithi022.stderr: 6500K .......... .......... .......... .......... .......... 3% 9.93M 14s 2024-03-29T04:19:15.458 INFO:teuthology.orchestra.run.smithi022.stderr: 6550K .......... .......... .......... .......... .......... 3% 8.62M 14s 2024-03-29T04:19:15.463 INFO:teuthology.orchestra.run.smithi022.stderr: 6600K .......... .......... .......... .......... .......... 3% 10.4M 14s 2024-03-29T04:19:15.467 INFO:teuthology.orchestra.run.smithi022.stderr: 6650K .......... .......... .......... .......... .......... 3% 11.0M 14s 2024-03-29T04:19:15.470 INFO:teuthology.orchestra.run.smithi022.stderr: 6700K .......... .......... .......... .......... .......... 4% 15.8M 14s 2024-03-29T04:19:15.474 INFO:teuthology.orchestra.run.smithi022.stderr: 6750K .......... .......... .......... .......... .......... 4% 12.8M 14s 2024-03-29T04:19:15.478 INFO:teuthology.orchestra.run.smithi022.stderr: 6800K .......... .......... .......... .......... .......... 4% 14.3M 14s 2024-03-29T04:19:15.481 INFO:teuthology.orchestra.run.smithi022.stderr: 6850K .......... .......... .......... .......... .......... 4% 16.0M 14s 2024-03-29T04:19:15.484 INFO:teuthology.orchestra.run.smithi022.stderr: 6900K .......... .......... .......... .......... .......... 4% 15.7M 14s 2024-03-29T04:19:15.488 INFO:teuthology.orchestra.run.smithi022.stderr: 6950K .......... .......... .......... .......... .......... 4% 12.6M 14s 2024-03-29T04:19:15.491 INFO:teuthology.orchestra.run.smithi022.stderr: 7000K .......... .......... .......... .......... .......... 4% 14.5M 14s 2024-03-29T04:19:15.494 INFO:teuthology.orchestra.run.smithi022.stderr: 7050K .......... .......... .......... .......... .......... 4% 15.4M 14s 2024-03-29T04:19:15.497 INFO:teuthology.orchestra.run.smithi022.stderr: 7100K .......... .......... .......... .......... .......... 4% 14.9M 14s 2024-03-29T04:19:15.501 INFO:teuthology.orchestra.run.smithi022.stderr: 7150K .......... .......... .......... .......... .......... 4% 12.8M 14s 2024-03-29T04:19:15.504 INFO:teuthology.orchestra.run.smithi022.stderr: 7200K .......... .......... .......... .......... .......... 4% 14.9M 14s 2024-03-29T04:19:15.508 INFO:teuthology.orchestra.run.smithi022.stderr: 7250K .......... .......... .......... .......... .......... 4% 14.9M 14s 2024-03-29T04:19:15.511 INFO:teuthology.orchestra.run.smithi022.stderr: 7300K .......... .......... .......... .......... .......... 4% 14.2M 14s 2024-03-29T04:19:15.515 INFO:teuthology.orchestra.run.smithi022.stderr: 7350K .......... .......... .......... .......... .......... 4% 12.5M 14s 2024-03-29T04:19:15.518 INFO:teuthology.orchestra.run.smithi022.stderr: 7400K .......... .......... .......... .......... .......... 4% 17.1M 14s 2024-03-29T04:19:15.521 INFO:teuthology.orchestra.run.smithi022.stderr: 7450K .......... .......... .......... .......... .......... 4% 15.9M 14s 2024-03-29T04:19:15.524 INFO:teuthology.orchestra.run.smithi022.stderr: 7500K .......... .......... .......... .......... .......... 4% 15.7M 14s 2024-03-29T04:19:15.528 INFO:teuthology.orchestra.run.smithi022.stderr: 7550K .......... .......... .......... .......... .......... 4% 13.6M 14s 2024-03-29T04:19:15.531 INFO:teuthology.orchestra.run.smithi022.stderr: 7600K .......... .......... .......... .......... .......... 4% 15.1M 14s 2024-03-29T04:19:15.534 INFO:teuthology.orchestra.run.smithi022.stderr: 7650K .......... .......... .......... .......... .......... 4% 15.7M 14s 2024-03-29T04:19:15.537 INFO:teuthology.orchestra.run.smithi022.stderr: 7700K .......... .......... .......... .......... .......... 4% 15.3M 14s 2024-03-29T04:19:15.541 INFO:teuthology.orchestra.run.smithi022.stderr: 7750K .......... .......... .......... .......... .......... 4% 12.2M 14s 2024-03-29T04:19:15.546 INFO:teuthology.orchestra.run.smithi022.stderr: 7800K .......... .......... .......... .......... .......... 4% 14.2M 14s 2024-03-29T04:19:15.548 INFO:teuthology.orchestra.run.smithi022.stderr: 7850K .......... .......... .......... .......... .......... 4% 15.3M 14s 2024-03-29T04:19:15.551 INFO:teuthology.orchestra.run.smithi022.stderr: 7900K .......... .......... .......... .......... .......... 4% 15.6M 14s 2024-03-29T04:19:15.555 INFO:teuthology.orchestra.run.smithi022.stderr: 7950K .......... .......... .......... .......... .......... 4% 13.5M 14s 2024-03-29T04:19:15.558 INFO:teuthology.orchestra.run.smithi022.stderr: 8000K .......... .......... .......... .......... .......... 4% 13.7M 14s 2024-03-29T04:19:15.561 INFO:teuthology.orchestra.run.smithi022.stderr: 8050K .......... .......... .......... .......... .......... 4% 15.1M 14s 2024-03-29T04:19:15.564 INFO:teuthology.orchestra.run.smithi022.stderr: 8100K .......... .......... .......... .......... .......... 4% 17.2M 13s 2024-03-29T04:19:15.568 INFO:teuthology.orchestra.run.smithi022.stderr: 8150K .......... .......... .......... .......... .......... 4% 15.4M 13s 2024-03-29T04:19:15.570 INFO:teuthology.orchestra.run.smithi022.stderr: 8200K .......... .......... .......... .......... .......... 4% 19.8M 13s 2024-03-29T04:19:15.572 INFO:teuthology.orchestra.run.smithi022.stderr: 8250K .......... .......... .......... .......... .......... 4% 21.3M 13s 2024-03-29T04:19:15.574 INFO:teuthology.orchestra.run.smithi022.stderr: 8300K .......... .......... .......... .......... .......... 4% 21.9M 13s 2024-03-29T04:19:15.577 INFO:teuthology.orchestra.run.smithi022.stderr: 8350K .......... .......... .......... .......... .......... 4% 18.9M 13s 2024-03-29T04:19:15.579 INFO:teuthology.orchestra.run.smithi022.stderr: 8400K .......... .......... .......... .......... .......... 5% 21.3M 13s 2024-03-29T04:19:15.581 INFO:teuthology.orchestra.run.smithi022.stderr: 8450K .......... .......... .......... .......... .......... 5% 22.1M 13s 2024-03-29T04:19:15.583 INFO:teuthology.orchestra.run.smithi022.stderr: 8500K .......... .......... .......... .......... .......... 5% 21.3M 13s 2024-03-29T04:19:15.586 INFO:teuthology.orchestra.run.smithi022.stderr: 8550K .......... .......... .......... .......... .......... 5% 18.8M 13s 2024-03-29T04:19:15.588 INFO:teuthology.orchestra.run.smithi022.stderr: 8600K .......... .......... .......... .......... .......... 5% 21.2M 13s 2024-03-29T04:19:15.595 INFO:teuthology.orchestra.run.smithi022.stderr: 8650K .......... .......... .......... .......... .......... 5% 19.9M 13s 2024-03-29T04:19:15.595 INFO:teuthology.orchestra.run.smithi022.stderr: 8700K .......... .......... .......... .......... .......... 5% 17.3M 13s 2024-03-29T04:19:15.598 INFO:teuthology.orchestra.run.smithi022.stderr: 8750K .......... .......... .......... .......... .......... 5% 13.4M 13s 2024-03-29T04:19:15.601 INFO:teuthology.orchestra.run.smithi022.stderr: 8800K .......... .......... .......... .......... .......... 5% 14.4M 13s 2024-03-29T04:19:15.604 INFO:teuthology.orchestra.run.smithi022.stderr: 8850K .......... .......... .......... .......... .......... 5% 13.7M 13s 2024-03-29T04:19:15.606 INFO:teuthology.orchestra.run.smithi022.stderr: 8900K .......... .......... .......... .......... .......... 5% 23.3M 13s 2024-03-29T04:19:15.609 INFO:teuthology.orchestra.run.smithi022.stderr: 8950K .......... .......... .......... .......... .......... 5% 20.6M 13s 2024-03-29T04:19:15.611 INFO:teuthology.orchestra.run.smithi022.stderr: 9000K .......... .......... .......... .......... .......... 5% 24.4M 13s 2024-03-29T04:19:15.613 INFO:teuthology.orchestra.run.smithi022.stderr: 9050K .......... .......... .......... .......... .......... 5% 24.1M 13s 2024-03-29T04:19:15.615 INFO:teuthology.orchestra.run.smithi022.stderr: 9100K .......... .......... .......... .......... .......... 5% 24.1M 13s 2024-03-29T04:19:15.617 INFO:teuthology.orchestra.run.smithi022.stderr: 9150K .......... .......... .......... .......... .......... 5% 23.3M 13s 2024-03-29T04:19:15.619 INFO:teuthology.orchestra.run.smithi022.stderr: 9200K .......... .......... .......... .......... .......... 5% 27.3M 13s 2024-03-29T04:19:15.620 INFO:teuthology.orchestra.run.smithi022.stderr: 9250K .......... .......... .......... .......... .......... 5% 26.1M 13s 2024-03-29T04:19:15.622 INFO:teuthology.orchestra.run.smithi022.stderr: 9300K .......... .......... .......... .......... .......... 5% 26.5M 13s 2024-03-29T04:19:15.625 INFO:teuthology.orchestra.run.smithi022.stderr: 9350K .......... .......... .......... .......... .......... 5% 22.0M 13s 2024-03-29T04:19:15.626 INFO:teuthology.orchestra.run.smithi022.stderr: 9400K .......... .......... .......... .......... .......... 5% 25.8M 13s 2024-03-29T04:19:15.628 INFO:teuthology.orchestra.run.smithi022.stderr: 9450K .......... .......... .......... .......... .......... 5% 25.6M 13s 2024-03-29T04:19:15.630 INFO:teuthology.orchestra.run.smithi022.stderr: 9500K .......... .......... .......... .......... .......... 5% 26.6M 13s 2024-03-29T04:19:15.632 INFO:teuthology.orchestra.run.smithi022.stderr: 9550K .......... .......... .......... .......... .......... 5% 22.7M 12s 2024-03-29T04:19:15.634 INFO:teuthology.orchestra.run.smithi022.stderr: 9600K .......... .......... .......... .......... .......... 5% 22.3M 12s 2024-03-29T04:19:15.636 INFO:teuthology.orchestra.run.smithi022.stderr: 9650K .......... .......... .......... .......... .......... 5% 24.4M 12s 2024-03-29T04:19:15.639 INFO:teuthology.orchestra.run.smithi022.stderr: 9700K .......... .......... .......... .......... .......... 5% 23.5M 12s 2024-03-29T04:19:15.641 INFO:teuthology.orchestra.run.smithi022.stderr: 9750K .......... .......... .......... .......... .......... 5% 21.7M 12s 2024-03-29T04:19:15.643 INFO:teuthology.orchestra.run.smithi022.stderr: 9800K .......... .......... .......... .......... .......... 5% 24.3M 12s 2024-03-29T04:19:15.645 INFO:teuthology.orchestra.run.smithi022.stderr: 9850K .......... .......... .......... .......... .......... 5% 22.7M 12s 2024-03-29T04:19:15.647 INFO:teuthology.orchestra.run.smithi022.stderr: 9900K .......... .......... .......... .......... .......... 5% 23.8M 12s 2024-03-29T04:19:15.649 INFO:teuthology.orchestra.run.smithi022.stderr: 9950K .......... .......... .......... .......... .......... 5% 21.0M 12s 2024-03-29T04:19:15.651 INFO:teuthology.orchestra.run.smithi022.stderr: 10000K .......... .......... .......... .......... .......... 5% 25.1M 12s 2024-03-29T04:19:15.653 INFO:teuthology.orchestra.run.smithi022.stderr: 10050K .......... .......... .......... .......... .......... 5% 26.3M 12s 2024-03-29T04:19:15.655 INFO:teuthology.orchestra.run.smithi022.stderr: 10100K .......... .......... .......... .......... .......... 6% 26.7M 12s 2024-03-29T04:19:15.657 INFO:teuthology.orchestra.run.smithi022.stderr: 10150K .......... .......... .......... .......... .......... 6% 21.8M 12s 2024-03-29T04:19:15.659 INFO:teuthology.orchestra.run.smithi022.stderr: 10200K .......... .......... .......... .......... .......... 6% 25.7M 12s 2024-03-29T04:19:15.661 INFO:teuthology.orchestra.run.smithi022.stderr: 10250K .......... .......... .......... .......... .......... 6% 24.7M 12s 2024-03-29T04:19:15.664 INFO:teuthology.orchestra.run.smithi022.stderr: 10300K .......... .......... .......... .......... .......... 6% 17.2M 12s 2024-03-29T04:19:15.667 INFO:teuthology.orchestra.run.smithi022.stderr: 10350K .......... .......... .......... .......... .......... 6% 16.7M 12s 2024-03-29T04:19:15.670 INFO:teuthology.orchestra.run.smithi022.stderr: 10400K .......... .......... .......... .......... .......... 6% 18.9M 12s 2024-03-29T04:19:15.672 INFO:teuthology.orchestra.run.smithi022.stderr: 10450K .......... .......... .......... .......... .......... 6% 18.4M 12s 2024-03-29T04:19:15.675 INFO:teuthology.orchestra.run.smithi022.stderr: 10500K .......... .......... .......... .......... .......... 6% 18.4M 12s 2024-03-29T04:19:15.678 INFO:teuthology.orchestra.run.smithi022.stderr: 10550K .......... .......... .......... .......... .......... 6% 15.9M 12s 2024-03-29T04:19:15.681 INFO:teuthology.orchestra.run.smithi022.stderr: 10600K .......... .......... .......... .......... .......... 6% 18.9M 12s 2024-03-29T04:19:15.683 INFO:teuthology.orchestra.run.smithi022.stderr: 10650K .......... .......... .......... .......... .......... 6% 18.6M 12s 2024-03-29T04:19:15.686 INFO:teuthology.orchestra.run.smithi022.stderr: 10700K .......... .......... .......... .......... .......... 6% 18.9M 12s 2024-03-29T04:19:15.689 INFO:teuthology.orchestra.run.smithi022.stderr: 10750K .......... .......... .......... .......... .......... 6% 15.6M 12s 2024-03-29T04:19:15.692 INFO:teuthology.orchestra.run.smithi022.stderr: 10800K .......... .......... .......... .......... .......... 6% 18.0M 12s 2024-03-29T04:19:15.694 INFO:teuthology.orchestra.run.smithi022.stderr: 10850K .......... .......... .......... .......... .......... 6% 19.3M 12s 2024-03-29T04:19:15.697 INFO:teuthology.orchestra.run.smithi022.stderr: 10900K .......... .......... .......... .......... .......... 6% 17.3M 12s 2024-03-29T04:19:15.700 INFO:teuthology.orchestra.run.smithi022.stderr: 10950K .......... .......... .......... .......... .......... 6% 14.2M 12s 2024-03-29T04:19:15.704 INFO:teuthology.orchestra.run.smithi022.stderr: 11000K .......... .......... .......... .......... .......... 6% 16.3M 12s 2024-03-29T04:19:15.706 INFO:teuthology.orchestra.run.smithi022.stderr: 11050K .......... .......... .......... .......... .......... 6% 20.6M 12s 2024-03-29T04:19:15.708 INFO:teuthology.orchestra.run.smithi022.stderr: 11100K .......... .......... .......... .......... .......... 6% 21.3M 12s 2024-03-29T04:19:15.711 INFO:teuthology.orchestra.run.smithi022.stderr: 11150K .......... .......... .......... .......... .......... 6% 17.5M 12s 2024-03-29T04:19:15.713 INFO:teuthology.orchestra.run.smithi022.stderr: 11200K .......... .......... .......... .......... .......... 6% 19.9M 12s 2024-03-29T04:19:15.716 INFO:teuthology.orchestra.run.smithi022.stderr: 11250K .......... .......... .......... .......... .......... 6% 20.1M 12s 2024-03-29T04:19:15.719 INFO:teuthology.orchestra.run.smithi022.stderr: 11300K .......... .......... .......... .......... .......... 6% 18.4M 12s 2024-03-29T04:19:15.722 INFO:teuthology.orchestra.run.smithi022.stderr: 11350K .......... .......... .......... .......... .......... 6% 14.6M 12s 2024-03-29T04:19:15.725 INFO:teuthology.orchestra.run.smithi022.stderr: 11400K .......... .......... .......... .......... .......... 6% 18.3M 12s 2024-03-29T04:19:15.728 INFO:teuthology.orchestra.run.smithi022.stderr: 11450K .......... .......... .......... .......... .......... 6% 15.4M 12s 2024-03-29T04:19:15.730 INFO:teuthology.orchestra.run.smithi022.stderr: 11500K .......... .......... .......... .......... .......... 6% 18.3M 12s 2024-03-29T04:19:15.734 INFO:teuthology.orchestra.run.smithi022.stderr: 11550K .......... .......... .......... .......... .......... 6% 14.0M 12s 2024-03-29T04:19:15.737 INFO:teuthology.orchestra.run.smithi022.stderr: 11600K .......... .......... .......... .......... .......... 6% 17.2M 12s 2024-03-29T04:19:15.740 INFO:teuthology.orchestra.run.smithi022.stderr: 11650K .......... .......... .......... .......... .......... 6% 15.7M 12s 2024-03-29T04:19:15.742 INFO:teuthology.orchestra.run.smithi022.stderr: 11700K .......... .......... .......... .......... .......... 6% 22.0M 12s 2024-03-29T04:19:15.745 INFO:teuthology.orchestra.run.smithi022.stderr: 11750K .......... .......... .......... .......... .......... 7% 15.9M 12s 2024-03-29T04:19:15.748 INFO:teuthology.orchestra.run.smithi022.stderr: 11800K .......... .......... .......... .......... .......... 7% 15.4M 11s 2024-03-29T04:19:15.751 INFO:teuthology.orchestra.run.smithi022.stderr: 11850K .......... .......... .......... .......... .......... 7% 15.9M 11s 2024-03-29T04:19:15.754 INFO:teuthology.orchestra.run.smithi022.stderr: 11900K .......... .......... .......... .......... .......... 7% 17.3M 11s 2024-03-29T04:19:15.758 INFO:teuthology.orchestra.run.smithi022.stderr: 11950K .......... .......... .......... .......... .......... 7% 13.2M 11s 2024-03-29T04:19:15.761 INFO:teuthology.orchestra.run.smithi022.stderr: 12000K .......... .......... .......... .......... .......... 7% 14.6M 11s 2024-03-29T04:19:15.764 INFO:teuthology.orchestra.run.smithi022.stderr: 12050K .......... .......... .......... .......... .......... 7% 17.4M 11s 2024-03-29T04:19:15.767 INFO:teuthology.orchestra.run.smithi022.stderr: 12100K .......... .......... .......... .......... .......... 7% 17.5M 11s 2024-03-29T04:19:15.770 INFO:teuthology.orchestra.run.smithi022.stderr: 12150K .......... .......... .......... .......... .......... 7% 15.7M 11s 2024-03-29T04:19:15.772 INFO:teuthology.orchestra.run.smithi022.stderr: 12200K .......... .......... .......... .......... .......... 7% 17.6M 11s 2024-03-29T04:19:15.775 INFO:teuthology.orchestra.run.smithi022.stderr: 12250K .......... .......... .......... .......... .......... 7% 20.5M 11s 2024-03-29T04:19:15.777 INFO:teuthology.orchestra.run.smithi022.stderr: 12300K .......... .......... .......... .......... .......... 7% 19.6M 11s 2024-03-29T04:19:15.779 INFO:teuthology.orchestra.run.smithi022.stderr: 12350K .......... .......... .......... .......... .......... 7% 22.9M 11s 2024-03-29T04:19:15.782 INFO:teuthology.orchestra.run.smithi022.stderr: 12400K .......... .......... .......... .......... .......... 7% 23.9M 11s 2024-03-29T04:19:15.784 INFO:teuthology.orchestra.run.smithi022.stderr: 12450K .......... .......... .......... .......... .......... 7% 18.5M 11s 2024-03-29T04:19:15.786 INFO:teuthology.orchestra.run.smithi022.stderr: 12500K .......... .......... .......... .......... .......... 7% 21.8M 11s 2024-03-29T04:19:15.790 INFO:teuthology.orchestra.run.smithi022.stderr: 12550K .......... .......... .......... .......... .......... 7% 14.9M 11s 2024-03-29T04:19:15.792 INFO:teuthology.orchestra.run.smithi022.stderr: 12600K .......... .......... .......... .......... .......... 7% 23.9M 11s 2024-03-29T04:19:15.793 INFO:teuthology.orchestra.run.smithi022.stderr: 12650K .......... .......... .......... .......... .......... 7% 29.1M 11s 2024-03-29T04:19:15.795 INFO:teuthology.orchestra.run.smithi022.stderr: 12700K .......... .......... .......... .......... .......... 7% 27.6M 11s 2024-03-29T04:19:15.797 INFO:teuthology.orchestra.run.smithi022.stderr: 12750K .......... .......... .......... .......... .......... 7% 23.5M 11s 2024-03-29T04:19:15.803 INFO:teuthology.orchestra.run.smithi022.stderr: 12800K .......... .......... .......... .......... .......... 7% 27.5M 11s 2024-03-29T04:19:15.803 INFO:teuthology.orchestra.run.smithi022.stderr: 12850K .......... .......... .......... .......... .......... 7% 29.6M 11s 2024-03-29T04:19:15.803 INFO:teuthology.orchestra.run.smithi022.stderr: 12900K .......... .......... .......... .......... .......... 7% 32.4M 11s 2024-03-29T04:19:15.804 INFO:teuthology.orchestra.run.smithi022.stderr: 12950K .......... .......... .......... .......... .......... 7% 27.4M 11s 2024-03-29T04:19:15.805 INFO:teuthology.orchestra.run.smithi022.stderr: 13000K .......... .......... .......... .......... .......... 7% 31.8M 11s 2024-03-29T04:19:15.807 INFO:teuthology.orchestra.run.smithi022.stderr: 13050K .......... .......... .......... .......... .......... 7% 31.4M 11s 2024-03-29T04:19:15.808 INFO:teuthology.orchestra.run.smithi022.stderr: 13100K .......... .......... .......... .......... .......... 7% 31.7M 11s 2024-03-29T04:19:15.810 INFO:teuthology.orchestra.run.smithi022.stderr: 13150K .......... .......... .......... .......... .......... 7% 25.6M 11s 2024-03-29T04:19:15.812 INFO:teuthology.orchestra.run.smithi022.stderr: 13200K .......... .......... .......... .......... .......... 7% 29.4M 11s 2024-03-29T04:19:15.814 INFO:teuthology.orchestra.run.smithi022.stderr: 13250K .......... .......... .......... .......... .......... 7% 31.3M 11s 2024-03-29T04:19:15.815 INFO:teuthology.orchestra.run.smithi022.stderr: 13300K .......... .......... .......... .......... .......... 7% 27.9M 11s 2024-03-29T04:19:15.817 INFO:teuthology.orchestra.run.smithi022.stderr: 13350K .......... .......... .......... .......... .......... 7% 23.9M 11s 2024-03-29T04:19:15.819 INFO:teuthology.orchestra.run.smithi022.stderr: 13400K .......... .......... .......... .......... .......... 7% 28.1M 11s 2024-03-29T04:19:15.821 INFO:teuthology.orchestra.run.smithi022.stderr: 13450K .......... .......... .......... .......... .......... 8% 27.6M 11s 2024-03-29T04:19:15.823 INFO:teuthology.orchestra.run.smithi022.stderr: 13500K .......... .......... .......... .......... .......... 8% 26.5M 11s 2024-03-29T04:19:15.825 INFO:teuthology.orchestra.run.smithi022.stderr: 13550K .......... .......... .......... .......... .......... 8% 23.2M 11s 2024-03-29T04:19:15.827 INFO:teuthology.orchestra.run.smithi022.stderr: 13600K .......... .......... .......... .......... .......... 8% 25.2M 11s 2024-03-29T04:19:15.829 INFO:teuthology.orchestra.run.smithi022.stderr: 13650K .......... .......... .......... .......... .......... 8% 25.6M 11s 2024-03-29T04:19:15.831 INFO:teuthology.orchestra.run.smithi022.stderr: 13700K .......... .......... .......... .......... .......... 8% 25.1M 11s 2024-03-29T04:19:15.833 INFO:teuthology.orchestra.run.smithi022.stderr: 13750K .......... .......... .......... .......... .......... 8% 21.7M 11s 2024-03-29T04:19:15.835 INFO:teuthology.orchestra.run.smithi022.stderr: 13800K .......... .......... .......... .......... .......... 8% 23.8M 11s 2024-03-29T04:19:15.837 INFO:teuthology.orchestra.run.smithi022.stderr: 13850K .......... .......... .......... .......... .......... 8% 25.0M 11s 2024-03-29T04:19:15.839 INFO:teuthology.orchestra.run.smithi022.stderr: 13900K .......... .......... .......... .......... .......... 8% 23.4M 11s 2024-03-29T04:19:15.841 INFO:teuthology.orchestra.run.smithi022.stderr: 13950K .......... .......... .......... .......... .......... 8% 22.6M 11s 2024-03-29T04:19:15.843 INFO:teuthology.orchestra.run.smithi022.stderr: 14000K .......... .......... .......... .......... .......... 8% 24.9M 11s 2024-03-29T04:19:15.845 INFO:teuthology.orchestra.run.smithi022.stderr: 14050K .......... .......... .......... .......... .......... 8% 24.9M 11s 2024-03-29T04:19:15.847 INFO:teuthology.orchestra.run.smithi022.stderr: 14100K .......... .......... .......... .......... .......... 8% 25.2M 11s 2024-03-29T04:19:15.849 INFO:teuthology.orchestra.run.smithi022.stderr: 14150K .......... .......... .......... .......... .......... 8% 21.9M 11s 2024-03-29T04:19:15.851 INFO:teuthology.orchestra.run.smithi022.stderr: 14200K .......... .......... .......... .......... .......... 8% 25.4M 11s 2024-03-29T04:19:15.853 INFO:teuthology.orchestra.run.smithi022.stderr: 14250K .......... .......... .......... .......... .......... 8% 25.6M 11s 2024-03-29T04:19:15.855 INFO:teuthology.orchestra.run.smithi022.stderr: 14300K .......... .......... .......... .......... .......... 8% 25.4M 10s 2024-03-29T04:19:15.857 INFO:teuthology.orchestra.run.smithi022.stderr: 14350K .......... .......... .......... .......... .......... 8% 21.7M 10s 2024-03-29T04:19:15.859 INFO:teuthology.orchestra.run.smithi022.stderr: 14400K .......... .......... .......... .......... .......... 8% 24.9M 10s 2024-03-29T04:19:15.861 INFO:teuthology.orchestra.run.smithi022.stderr: 14450K .......... .......... .......... .......... .......... 8% 25.3M 10s 2024-03-29T04:19:15.863 INFO:teuthology.orchestra.run.smithi022.stderr: 14500K .......... .......... .......... .......... .......... 8% 23.8M 10s 2024-03-29T04:19:15.866 INFO:teuthology.orchestra.run.smithi022.stderr: 14550K .......... .......... .......... .......... .......... 8% 20.3M 10s 2024-03-29T04:19:15.868 INFO:teuthology.orchestra.run.smithi022.stderr: 14600K .......... .......... .......... .......... .......... 8% 18.8M 10s 2024-03-29T04:19:15.871 INFO:teuthology.orchestra.run.smithi022.stderr: 14650K .......... .......... .......... .......... .......... 8% 20.3M 10s 2024-03-29T04:19:15.873 INFO:teuthology.orchestra.run.smithi022.stderr: 14700K .......... .......... .......... .......... .......... 8% 20.5M 10s 2024-03-29T04:19:15.876 INFO:teuthology.orchestra.run.smithi022.stderr: 14750K .......... .......... .......... .......... .......... 8% 17.3M 10s 2024-03-29T04:19:15.878 INFO:teuthology.orchestra.run.smithi022.stderr: 14800K .......... .......... .......... .......... .......... 8% 20.4M 10s 2024-03-29T04:19:15.881 INFO:teuthology.orchestra.run.smithi022.stderr: 14850K .......... .......... .......... .......... .......... 8% 21.2M 10s 2024-03-29T04:19:15.883 INFO:teuthology.orchestra.run.smithi022.stderr: 14900K .......... .......... .......... .......... .......... 8% 23.9M 10s 2024-03-29T04:19:15.885 INFO:teuthology.orchestra.run.smithi022.stderr: 14950K .......... .......... .......... .......... .......... 8% 21.6M 10s 2024-03-29T04:19:15.887 INFO:teuthology.orchestra.run.smithi022.stderr: 15000K .......... .......... .......... .......... .......... 8% 24.7M 10s 2024-03-29T04:19:15.889 INFO:teuthology.orchestra.run.smithi022.stderr: 15050K .......... .......... .......... .......... .......... 8% 23.8M 10s 2024-03-29T04:19:15.891 INFO:teuthology.orchestra.run.smithi022.stderr: 15100K .......... .......... .......... .......... .......... 8% 23.4M 10s 2024-03-29T04:19:15.893 INFO:teuthology.orchestra.run.smithi022.stderr: 15150K .......... .......... .......... .......... .......... 9% 21.1M 10s 2024-03-29T04:19:15.895 INFO:teuthology.orchestra.run.smithi022.stderr: 15200K .......... .......... .......... .......... .......... 9% 27.0M 10s 2024-03-29T04:19:15.897 INFO:teuthology.orchestra.run.smithi022.stderr: 15250K .......... .......... .......... .......... .......... 9% 26.2M 10s 2024-03-29T04:19:15.899 INFO:teuthology.orchestra.run.smithi022.stderr: 15300K .......... .......... .......... .......... .......... 9% 26.1M 10s 2024-03-29T04:19:15.901 INFO:teuthology.orchestra.run.smithi022.stderr: 15350K .......... .......... .......... .......... .......... 9% 21.4M 10s 2024-03-29T04:19:15.903 INFO:teuthology.orchestra.run.smithi022.stderr: 15400K .......... .......... .......... .......... .......... 9% 23.4M 10s 2024-03-29T04:19:15.905 INFO:teuthology.orchestra.run.smithi022.stderr: 15450K .......... .......... .......... .......... .......... 9% 22.9M 10s 2024-03-29T04:19:15.908 INFO:teuthology.orchestra.run.smithi022.stderr: 15500K .......... .......... .......... .......... .......... 9% 22.4M 10s 2024-03-29T04:19:15.911 INFO:teuthology.orchestra.run.smithi022.stderr: 15550K .......... .......... .......... .......... .......... 9% 17.6M 10s 2024-03-29T04:19:15.913 INFO:teuthology.orchestra.run.smithi022.stderr: 15600K .......... .......... .......... .......... .......... 9% 18.8M 10s 2024-03-29T04:19:15.916 INFO:teuthology.orchestra.run.smithi022.stderr: 15650K .......... .......... .......... .......... .......... 9% 19.2M 10s 2024-03-29T04:19:15.918 INFO:teuthology.orchestra.run.smithi022.stderr: 15700K .......... .......... .......... .......... .......... 9% 18.6M 10s 2024-03-29T04:19:15.921 INFO:teuthology.orchestra.run.smithi022.stderr: 15750K .......... .......... .......... .......... .......... 9% 15.9M 10s 2024-03-29T04:19:15.924 INFO:teuthology.orchestra.run.smithi022.stderr: 15800K .......... .......... .......... .......... .......... 9% 18.5M 10s 2024-03-29T04:19:15.927 INFO:teuthology.orchestra.run.smithi022.stderr: 15850K .......... .......... .......... .......... .......... 9% 18.6M 10s 2024-03-29T04:19:15.929 INFO:teuthology.orchestra.run.smithi022.stderr: 15900K .......... .......... .......... .......... .......... 9% 19.0M 10s 2024-03-29T04:19:15.932 INFO:teuthology.orchestra.run.smithi022.stderr: 15950K .......... .......... .......... .......... .......... 9% 16.3M 10s 2024-03-29T04:19:15.935 INFO:teuthology.orchestra.run.smithi022.stderr: 16000K .......... .......... .......... .......... .......... 9% 18.6M 10s 2024-03-29T04:19:15.937 INFO:teuthology.orchestra.run.smithi022.stderr: 16050K .......... .......... .......... .......... .......... 9% 19.3M 10s 2024-03-29T04:19:15.940 INFO:teuthology.orchestra.run.smithi022.stderr: 16100K .......... .......... .......... .......... .......... 9% 18.4M 10s 2024-03-29T04:19:15.943 INFO:teuthology.orchestra.run.smithi022.stderr: 16150K .......... .......... .......... .......... .......... 9% 15.7M 10s 2024-03-29T04:19:15.946 INFO:teuthology.orchestra.run.smithi022.stderr: 16200K .......... .......... .......... .......... .......... 9% 18.2M 10s 2024-03-29T04:19:15.948 INFO:teuthology.orchestra.run.smithi022.stderr: 16250K .......... .......... .......... .......... .......... 9% 22.2M 10s 2024-03-29T04:19:15.950 INFO:teuthology.orchestra.run.smithi022.stderr: 16300K .......... .......... .......... .......... .......... 9% 22.4M 10s 2024-03-29T04:19:15.953 INFO:teuthology.orchestra.run.smithi022.stderr: 16350K .......... .......... .......... .......... .......... 9% 18.3M 10s 2024-03-29T04:19:15.955 INFO:teuthology.orchestra.run.smithi022.stderr: 16400K .......... .......... .......... .......... .......... 9% 27.4M 10s 2024-03-29T04:19:15.956 INFO:teuthology.orchestra.run.smithi022.stderr: 16450K .......... .......... .......... .......... .......... 9% 27.4M 10s 2024-03-29T04:19:15.958 INFO:teuthology.orchestra.run.smithi022.stderr: 16500K .......... .......... .......... .......... .......... 9% 27.7M 10s 2024-03-29T04:19:15.960 INFO:teuthology.orchestra.run.smithi022.stderr: 16550K .......... .......... .......... .......... .......... 9% 22.6M 10s 2024-03-29T04:19:15.962 INFO:teuthology.orchestra.run.smithi022.stderr: 16600K .......... .......... .......... .......... .......... 9% 27.0M 10s 2024-03-29T04:19:15.964 INFO:teuthology.orchestra.run.smithi022.stderr: 16650K .......... .......... .......... .......... .......... 9% 25.7M 10s 2024-03-29T04:19:15.966 INFO:teuthology.orchestra.run.smithi022.stderr: 16700K .......... .......... .......... .......... .......... 9% 25.2M 10s 2024-03-29T04:19:15.968 INFO:teuthology.orchestra.run.smithi022.stderr: 16750K .......... .......... .......... .......... .......... 9% 24.6M 10s 2024-03-29T04:19:15.969 INFO:teuthology.orchestra.run.smithi022.stderr: 16800K .......... .......... .......... .......... .......... 9% 30.7M 10s 2024-03-29T04:19:15.971 INFO:teuthology.orchestra.run.smithi022.stderr: 16850K .......... .......... .......... .......... .......... 10% 29.7M 10s 2024-03-29T04:19:15.973 INFO:teuthology.orchestra.run.smithi022.stderr: 16900K .......... .......... .......... .......... .......... 10% 28.6M 10s 2024-03-29T04:19:15.975 INFO:teuthology.orchestra.run.smithi022.stderr: 16950K .......... .......... .......... .......... .......... 10% 25.8M 10s 2024-03-29T04:19:15.976 INFO:teuthology.orchestra.run.smithi022.stderr: 17000K .......... .......... .......... .......... .......... 10% 31.0M 10s 2024-03-29T04:19:15.978 INFO:teuthology.orchestra.run.smithi022.stderr: 17050K .......... .......... .......... .......... .......... 10% 31.4M 10s 2024-03-29T04:19:15.979 INFO:teuthology.orchestra.run.smithi022.stderr: 17100K .......... .......... .......... .......... .......... 10% 29.8M 10s 2024-03-29T04:19:15.981 INFO:teuthology.orchestra.run.smithi022.stderr: 17150K .......... .......... .......... .......... .......... 10% 25.4M 10s 2024-03-29T04:19:15.984 INFO:teuthology.orchestra.run.smithi022.stderr: 17200K .......... .......... .......... .......... .......... 10% 21.0M 10s 2024-03-29T04:19:15.986 INFO:teuthology.orchestra.run.smithi022.stderr: 17250K .......... .......... .......... .......... .......... 10% 22.9M 10s 2024-03-29T04:19:15.988 INFO:teuthology.orchestra.run.smithi022.stderr: 17300K .......... .......... .......... .......... .......... 10% 22.7M 10s 2024-03-29T04:19:15.990 INFO:teuthology.orchestra.run.smithi022.stderr: 17350K .......... .......... .......... .......... .......... 10% 20.2M 10s 2024-03-29T04:19:15.993 INFO:teuthology.orchestra.run.smithi022.stderr: 17400K .......... .......... .......... .......... .......... 10% 18.9M 10s 2024-03-29T04:19:15.996 INFO:teuthology.orchestra.run.smithi022.stderr: 17450K .......... .......... .......... .......... .......... 10% 17.5M 10s 2024-03-29T04:19:15.998 INFO:teuthology.orchestra.run.smithi022.stderr: 17500K .......... .......... .......... .......... .......... 10% 18.4M 10s 2024-03-29T04:19:16.003 INFO:teuthology.orchestra.run.smithi022.stderr: 17550K .......... .......... .......... .......... .......... 10% 16.7M 10s 2024-03-29T04:19:16.004 INFO:teuthology.orchestra.run.smithi022.stderr: 17600K .......... .......... .......... .......... .......... 10% 19.4M 10s 2024-03-29T04:19:16.007 INFO:teuthology.orchestra.run.smithi022.stderr: 17650K .......... .......... .......... .......... .......... 10% 18.4M 10s 2024-03-29T04:19:16.010 INFO:teuthology.orchestra.run.smithi022.stderr: 17700K .......... .......... .......... .......... .......... 10% 18.4M 10s 2024-03-29T04:19:16.012 INFO:teuthology.orchestra.run.smithi022.stderr: 17750K .......... .......... .......... .......... .......... 10% 16.0M 10s 2024-03-29T04:19:16.015 INFO:teuthology.orchestra.run.smithi022.stderr: 17800K .......... .......... .......... .......... .......... 10% 18.2M 10s 2024-03-29T04:19:16.018 INFO:teuthology.orchestra.run.smithi022.stderr: 17850K .......... .......... .......... .......... .......... 10% 17.7M 10s 2024-03-29T04:19:16.020 INFO:teuthology.orchestra.run.smithi022.stderr: 17900K .......... .......... .......... .......... .......... 10% 17.8M 10s 2024-03-29T04:19:16.024 INFO:teuthology.orchestra.run.smithi022.stderr: 17950K .......... .......... .......... .......... .......... 10% 15.3M 10s 2024-03-29T04:19:16.026 INFO:teuthology.orchestra.run.smithi022.stderr: 18000K .......... .......... .......... .......... .......... 10% 17.9M 10s 2024-03-29T04:19:16.030 INFO:teuthology.orchestra.run.smithi022.stderr: 18050K .......... .......... .......... .......... .......... 10% 15.8M 10s 2024-03-29T04:19:16.033 INFO:teuthology.orchestra.run.smithi022.stderr: 18100K .......... .......... .......... .......... .......... 10% 15.4M 10s 2024-03-29T04:19:16.037 INFO:teuthology.orchestra.run.smithi022.stderr: 18150K .......... .......... .......... .......... .......... 10% 14.3M 10s 2024-03-29T04:19:16.039 INFO:teuthology.orchestra.run.smithi022.stderr: 18200K .......... .......... .......... .......... .......... 10% 18.7M 10s 2024-03-29T04:19:16.042 INFO:teuthology.orchestra.run.smithi022.stderr: 18250K .......... .......... .......... .......... .......... 10% 15.1M 10s 2024-03-29T04:19:16.045 INFO:teuthology.orchestra.run.smithi022.stderr: 18300K .......... .......... .......... .......... .......... 10% 16.8M 10s 2024-03-29T04:19:16.048 INFO:teuthology.orchestra.run.smithi022.stderr: 18350K .......... .......... .......... .......... .......... 10% 14.6M 10s 2024-03-29T04:19:16.050 INFO:teuthology.orchestra.run.smithi022.stderr: 18400K .......... .......... .......... .......... .......... 10% 22.5M 10s 2024-03-29T04:19:16.052 INFO:teuthology.orchestra.run.smithi022.stderr: 18450K .......... .......... .......... .......... .......... 10% 22.8M 10s 2024-03-29T04:19:16.055 INFO:teuthology.orchestra.run.smithi022.stderr: 18500K .......... .......... .......... .......... .......... 11% 22.9M 10s 2024-03-29T04:19:16.057 INFO:teuthology.orchestra.run.smithi022.stderr: 18550K .......... .......... .......... .......... .......... 11% 19.3M 9s 2024-03-29T04:19:16.059 INFO:teuthology.orchestra.run.smithi022.stderr: 18600K .......... .......... .......... .......... .......... 11% 21.5M 9s 2024-03-29T04:19:16.062 INFO:teuthology.orchestra.run.smithi022.stderr: 18650K .......... .......... .......... .......... .......... 11% 18.3M 9s 2024-03-29T04:19:16.065 INFO:teuthology.orchestra.run.smithi022.stderr: 18700K .......... .......... .......... .......... .......... 11% 19.3M 9s 2024-03-29T04:19:16.067 INFO:teuthology.orchestra.run.smithi022.stderr: 18750K .......... .......... .......... .......... .......... 11% 17.8M 9s 2024-03-29T04:19:16.070 INFO:teuthology.orchestra.run.smithi022.stderr: 18800K .......... .......... .......... .......... .......... 11% 17.2M 9s 2024-03-29T04:19:16.073 INFO:teuthology.orchestra.run.smithi022.stderr: 18850K .......... .......... .......... .......... .......... 11% 17.0M 9s 2024-03-29T04:19:16.076 INFO:teuthology.orchestra.run.smithi022.stderr: 18900K .......... .......... .......... .......... .......... 11% 15.8M 9s 2024-03-29T04:19:16.080 INFO:teuthology.orchestra.run.smithi022.stderr: 18950K .......... .......... .......... .......... .......... 11% 14.7M 9s 2024-03-29T04:19:16.082 INFO:teuthology.orchestra.run.smithi022.stderr: 19000K .......... .......... .......... .......... .......... 11% 19.4M 9s 2024-03-29T04:19:16.085 INFO:teuthology.orchestra.run.smithi022.stderr: 19050K .......... .......... .......... .......... .......... 11% 18.0M 9s 2024-03-29T04:19:16.088 INFO:teuthology.orchestra.run.smithi022.stderr: 19100K .......... .......... .......... .......... .......... 11% 16.6M 9s 2024-03-29T04:19:16.091 INFO:teuthology.orchestra.run.smithi022.stderr: 19150K .......... .......... .......... .......... .......... 11% 16.4M 9s 2024-03-29T04:19:16.093 INFO:teuthology.orchestra.run.smithi022.stderr: 19200K .......... .......... .......... .......... .......... 11% 19.2M 9s 2024-03-29T04:19:16.097 INFO:teuthology.orchestra.run.smithi022.stderr: 19250K .......... .......... .......... .......... .......... 11% 13.8M 9s 2024-03-29T04:19:16.100 INFO:teuthology.orchestra.run.smithi022.stderr: 19300K .......... .......... .......... .......... .......... 11% 15.6M 9s 2024-03-29T04:19:16.104 INFO:teuthology.orchestra.run.smithi022.stderr: 19350K .......... .......... .......... .......... .......... 11% 11.6M 9s 2024-03-29T04:19:16.108 INFO:teuthology.orchestra.run.smithi022.stderr: 19400K .......... .......... .......... .......... .......... 11% 12.9M 9s 2024-03-29T04:19:16.111 INFO:teuthology.orchestra.run.smithi022.stderr: 19450K .......... .......... .......... .......... .......... 11% 13.4M 9s 2024-03-29T04:19:16.115 INFO:teuthology.orchestra.run.smithi022.stderr: 19500K .......... .......... .......... .......... .......... 11% 14.4M 9s 2024-03-29T04:19:16.119 INFO:teuthology.orchestra.run.smithi022.stderr: 19550K .......... .......... .......... .......... .......... 11% 12.0M 9s 2024-03-29T04:19:16.123 INFO:teuthology.orchestra.run.smithi022.stderr: 19600K .......... .......... .......... .......... .......... 11% 13.1M 9s 2024-03-29T04:19:16.126 INFO:teuthology.orchestra.run.smithi022.stderr: 19650K .......... .......... .......... .......... .......... 11% 12.8M 9s 2024-03-29T04:19:16.130 INFO:teuthology.orchestra.run.smithi022.stderr: 19700K .......... .......... .......... .......... .......... 11% 14.9M 9s 2024-03-29T04:19:16.134 INFO:teuthology.orchestra.run.smithi022.stderr: 19750K .......... .......... .......... .......... .......... 11% 12.6M 9s 2024-03-29T04:19:16.137 INFO:teuthology.orchestra.run.smithi022.stderr: 19800K .......... .......... .......... .......... .......... 11% 15.2M 9s 2024-03-29T04:19:16.141 INFO:teuthology.orchestra.run.smithi022.stderr: 19850K .......... .......... .......... .......... .......... 11% 12.0M 9s 2024-03-29T04:19:16.145 INFO:teuthology.orchestra.run.smithi022.stderr: 19900K .......... .......... .......... .......... .......... 11% 11.1M 9s 2024-03-29T04:19:16.150 INFO:teuthology.orchestra.run.smithi022.stderr: 19950K .......... .......... .......... .......... .......... 11% 9.84M 9s 2024-03-29T04:19:16.155 INFO:teuthology.orchestra.run.smithi022.stderr: 20000K .......... .......... .......... .......... .......... 11% 11.3M 9s 2024-03-29T04:19:16.158 INFO:teuthology.orchestra.run.smithi022.stderr: 20050K .......... .......... .......... .......... .......... 11% 13.1M 9s 2024-03-29T04:19:16.161 INFO:teuthology.orchestra.run.smithi022.stderr: 20100K .......... .......... .......... .......... .......... 11% 16.4M 9s 2024-03-29T04:19:16.165 INFO:teuthology.orchestra.run.smithi022.stderr: 20150K .......... .......... .......... .......... .......... 11% 13.2M 9s 2024-03-29T04:19:16.168 INFO:teuthology.orchestra.run.smithi022.stderr: 20200K .......... .......... .......... .......... .......... 12% 15.7M 9s 2024-03-29T04:19:16.171 INFO:teuthology.orchestra.run.smithi022.stderr: 20250K .......... .......... .......... .......... .......... 12% 14.7M 9s 2024-03-29T04:19:16.174 INFO:teuthology.orchestra.run.smithi022.stderr: 20300K .......... .......... .......... .......... .......... 12% 17.3M 9s 2024-03-29T04:19:16.177 INFO:teuthology.orchestra.run.smithi022.stderr: 20350K .......... .......... .......... .......... .......... 12% 15.3M 9s 2024-03-29T04:19:16.180 INFO:teuthology.orchestra.run.smithi022.stderr: 20400K .......... .......... .......... .......... .......... 12% 16.8M 9s 2024-03-29T04:19:16.184 INFO:teuthology.orchestra.run.smithi022.stderr: 20450K .......... .......... .......... .......... .......... 12% 15.2M 9s 2024-03-29T04:19:16.186 INFO:teuthology.orchestra.run.smithi022.stderr: 20500K .......... .......... .......... .......... .......... 12% 19.9M 9s 2024-03-29T04:19:16.189 INFO:teuthology.orchestra.run.smithi022.stderr: 20550K .......... .......... .......... .......... .......... 12% 15.7M 9s 2024-03-29T04:19:16.192 INFO:teuthology.orchestra.run.smithi022.stderr: 20600K .......... .......... .......... .......... .......... 12% 19.7M 9s 2024-03-29T04:19:16.195 INFO:teuthology.orchestra.run.smithi022.stderr: 20650K .......... .......... .......... .......... .......... 12% 16.1M 9s 2024-03-29T04:19:16.197 INFO:teuthology.orchestra.run.smithi022.stderr: 20700K .......... .......... .......... .......... .......... 12% 19.6M 9s 2024-03-29T04:19:16.200 INFO:teuthology.orchestra.run.smithi022.stderr: 20750K .......... .......... .......... .......... .......... 12% 19.0M 9s 2024-03-29T04:19:16.202 INFO:teuthology.orchestra.run.smithi022.stderr: 20800K .......... .......... .......... .......... .......... 12% 21.6M 9s 2024-03-29T04:19:16.204 INFO:teuthology.orchestra.run.smithi022.stderr: 20850K .......... .......... .......... .......... .......... 12% 23.9M 9s 2024-03-29T04:19:16.206 INFO:teuthology.orchestra.run.smithi022.stderr: 20900K .......... .......... .......... .......... .......... 12% 23.0M 9s 2024-03-29T04:19:16.208 INFO:teuthology.orchestra.run.smithi022.stderr: 20950K .......... .......... .......... .......... .......... 12% 19.8M 9s 2024-03-29T04:19:16.211 INFO:teuthology.orchestra.run.smithi022.stderr: 21000K .......... .......... .......... .......... .......... 12% 23.2M 9s 2024-03-29T04:19:16.213 INFO:teuthology.orchestra.run.smithi022.stderr: 21050K .......... .......... .......... .......... .......... 12% 23.1M 9s 2024-03-29T04:19:16.215 INFO:teuthology.orchestra.run.smithi022.stderr: 21100K .......... .......... .......... .......... .......... 12% 23.1M 9s 2024-03-29T04:19:16.217 INFO:teuthology.orchestra.run.smithi022.stderr: 21150K .......... .......... .......... .......... .......... 12% 21.4M 9s 2024-03-29T04:19:16.219 INFO:teuthology.orchestra.run.smithi022.stderr: 21200K .......... .......... .......... .......... .......... 12% 33.0M 9s 2024-03-29T04:19:16.220 INFO:teuthology.orchestra.run.smithi022.stderr: 21250K .......... .......... .......... .......... .......... 12% 33.1M 9s 2024-03-29T04:19:16.221 INFO:teuthology.orchestra.run.smithi022.stderr: 21300K .......... .......... .......... .......... .......... 12% 33.0M 9s 2024-03-29T04:19:16.223 INFO:teuthology.orchestra.run.smithi022.stderr: 21350K .......... .......... .......... .......... .......... 12% 28.8M 9s 2024-03-29T04:19:16.224 INFO:teuthology.orchestra.run.smithi022.stderr: 21400K .......... .......... .......... .......... .......... 12% 33.2M 9s 2024-03-29T04:19:16.226 INFO:teuthology.orchestra.run.smithi022.stderr: 21450K .......... .......... .......... .......... .......... 12% 34.4M 9s 2024-03-29T04:19:16.227 INFO:teuthology.orchestra.run.smithi022.stderr: 21500K .......... .......... .......... .......... .......... 12% 32.7M 9s 2024-03-29T04:19:16.229 INFO:teuthology.orchestra.run.smithi022.stderr: 21550K .......... .......... .......... .......... .......... 12% 30.1M 9s 2024-03-29T04:19:16.231 INFO:teuthology.orchestra.run.smithi022.stderr: 21600K .......... .......... .......... .......... .......... 12% 34.7M 9s 2024-03-29T04:19:16.232 INFO:teuthology.orchestra.run.smithi022.stderr: 21650K .......... .......... .......... .......... .......... 12% 31.8M 9s 2024-03-29T04:19:16.233 INFO:teuthology.orchestra.run.smithi022.stderr: 21700K .......... .......... .......... .......... .......... 12% 33.6M 9s 2024-03-29T04:19:16.235 INFO:teuthology.orchestra.run.smithi022.stderr: 21750K .......... .......... .......... .......... .......... 12% 27.9M 9s 2024-03-29T04:19:16.237 INFO:teuthology.orchestra.run.smithi022.stderr: 21800K .......... .......... .......... .......... .......... 12% 33.9M 9s 2024-03-29T04:19:16.238 INFO:teuthology.orchestra.run.smithi022.stderr: 21850K .......... .......... .......... .......... .......... 12% 33.9M 9s 2024-03-29T04:19:16.240 INFO:teuthology.orchestra.run.smithi022.stderr: 21900K .......... .......... .......... .......... .......... 13% 30.4M 9s 2024-03-29T04:19:16.241 INFO:teuthology.orchestra.run.smithi022.stderr: 21950K .......... .......... .......... .......... .......... 13% 26.9M 9s 2024-03-29T04:19:16.243 INFO:teuthology.orchestra.run.smithi022.stderr: 22000K .......... .......... .......... .......... .......... 13% 30.8M 9s 2024-03-29T04:19:16.245 INFO:teuthology.orchestra.run.smithi022.stderr: 22050K .......... .......... .......... .......... .......... 13% 31.0M 9s 2024-03-29T04:19:16.246 INFO:teuthology.orchestra.run.smithi022.stderr: 22100K .......... .......... .......... .......... .......... 13% 31.6M 9s 2024-03-29T04:19:16.248 INFO:teuthology.orchestra.run.smithi022.stderr: 22150K .......... .......... .......... .......... .......... 13% 25.8M 9s 2024-03-29T04:19:16.250 INFO:teuthology.orchestra.run.smithi022.stderr: 22200K .......... .......... .......... .......... .......... 13% 31.3M 9s 2024-03-29T04:19:16.251 INFO:teuthology.orchestra.run.smithi022.stderr: 22250K .......... .......... .......... .......... .......... 13% 30.6M 9s 2024-03-29T04:19:16.253 INFO:teuthology.orchestra.run.smithi022.stderr: 22300K .......... .......... .......... .......... .......... 13% 28.8M 9s 2024-03-29T04:19:16.255 INFO:teuthology.orchestra.run.smithi022.stderr: 22350K .......... .......... .......... .......... .......... 13% 24.9M 9s 2024-03-29T04:19:16.257 INFO:teuthology.orchestra.run.smithi022.stderr: 22400K .......... .......... .......... .......... .......... 13% 29.6M 9s 2024-03-29T04:19:16.258 INFO:teuthology.orchestra.run.smithi022.stderr: 22450K .......... .......... .......... .......... .......... 13% 30.5M 9s 2024-03-29T04:19:16.260 INFO:teuthology.orchestra.run.smithi022.stderr: 22500K .......... .......... .......... .......... .......... 13% 26.6M 9s 2024-03-29T04:19:16.262 INFO:teuthology.orchestra.run.smithi022.stderr: 22550K .......... .......... .......... .......... .......... 13% 23.2M 9s 2024-03-29T04:19:16.264 INFO:teuthology.orchestra.run.smithi022.stderr: 22600K .......... .......... .......... .......... .......... 13% 27.6M 9s 2024-03-29T04:19:16.266 INFO:teuthology.orchestra.run.smithi022.stderr: 22650K .......... .......... .......... .......... .......... 13% 27.8M 9s 2024-03-29T04:19:16.267 INFO:teuthology.orchestra.run.smithi022.stderr: 22700K .......... .......... .......... .......... .......... 13% 26.2M 9s 2024-03-29T04:19:16.269 INFO:teuthology.orchestra.run.smithi022.stderr: 22750K .......... .......... .......... .......... .......... 13% 24.7M 9s 2024-03-29T04:19:16.271 INFO:teuthology.orchestra.run.smithi022.stderr: 22800K .......... .......... .......... .......... .......... 13% 29.1M 9s 2024-03-29T04:19:16.273 INFO:teuthology.orchestra.run.smithi022.stderr: 22850K .......... .......... .......... .......... .......... 13% 28.3M 9s 2024-03-29T04:19:16.275 INFO:teuthology.orchestra.run.smithi022.stderr: 22900K .......... .......... .......... .......... .......... 13% 28.0M 9s 2024-03-29T04:19:16.277 INFO:teuthology.orchestra.run.smithi022.stderr: 22950K .......... .......... .......... .......... .......... 13% 24.1M 9s 2024-03-29T04:19:16.278 INFO:teuthology.orchestra.run.smithi022.stderr: 23000K .......... .......... .......... .......... .......... 13% 29.0M 9s 2024-03-29T04:19:16.280 INFO:teuthology.orchestra.run.smithi022.stderr: 23050K .......... .......... .......... .......... .......... 13% 28.1M 9s 2024-03-29T04:19:16.282 INFO:teuthology.orchestra.run.smithi022.stderr: 23100K .......... .......... .......... .......... .......... 13% 29.0M 9s 2024-03-29T04:19:16.284 INFO:teuthology.orchestra.run.smithi022.stderr: 23150K .......... .......... .......... .......... .......... 13% 23.7M 9s 2024-03-29T04:19:16.286 INFO:teuthology.orchestra.run.smithi022.stderr: 23200K .......... .......... .......... .......... .......... 13% 27.2M 9s 2024-03-29T04:19:16.287 INFO:teuthology.orchestra.run.smithi022.stderr: 23250K .......... .......... .......... .......... .......... 13% 26.8M 9s 2024-03-29T04:19:16.289 INFO:teuthology.orchestra.run.smithi022.stderr: 23300K .......... .......... .......... .......... .......... 13% 27.2M 9s 2024-03-29T04:19:16.291 INFO:teuthology.orchestra.run.smithi022.stderr: 23350K .......... .......... .......... .......... .......... 13% 23.9M 9s 2024-03-29T04:19:16.293 INFO:teuthology.orchestra.run.smithi022.stderr: 23400K .......... .......... .......... .......... .......... 13% 27.9M 9s 2024-03-29T04:19:16.295 INFO:teuthology.orchestra.run.smithi022.stderr: 23450K .......... .......... .......... .......... .......... 13% 20.5M 9s 2024-03-29T04:19:16.298 INFO:teuthology.orchestra.run.smithi022.stderr: 23500K .......... .......... .......... .......... .......... 13% 20.0M 9s 2024-03-29T04:19:16.301 INFO:teuthology.orchestra.run.smithi022.stderr: 23550K .......... .......... .......... .......... .......... 14% 17.1M 9s 2024-03-29T04:19:16.303 INFO:teuthology.orchestra.run.smithi022.stderr: 23600K .......... .......... .......... .......... .......... 14% 20.3M 9s 2024-03-29T04:19:16.305 INFO:teuthology.orchestra.run.smithi022.stderr: 23650K .......... .......... .......... .......... .......... 14% 22.0M 9s 2024-03-29T04:19:16.308 INFO:teuthology.orchestra.run.smithi022.stderr: 23700K .......... .......... .......... .......... .......... 14% 21.3M 9s 2024-03-29T04:19:16.310 INFO:teuthology.orchestra.run.smithi022.stderr: 23750K .......... .......... .......... .......... .......... 14% 19.4M 9s 2024-03-29T04:19:16.312 INFO:teuthology.orchestra.run.smithi022.stderr: 23800K .......... .......... .......... .......... .......... 14% 23.3M 9s 2024-03-29T04:19:16.314 INFO:teuthology.orchestra.run.smithi022.stderr: 23850K .......... .......... .......... .......... .......... 14% 22.3M 9s 2024-03-29T04:19:16.317 INFO:teuthology.orchestra.run.smithi022.stderr: 23900K .......... .......... .......... .......... .......... 14% 22.4M 9s 2024-03-29T04:19:16.319 INFO:teuthology.orchestra.run.smithi022.stderr: 23950K .......... .......... .......... .......... .......... 14% 19.1M 9s 2024-03-29T04:19:16.321 INFO:teuthology.orchestra.run.smithi022.stderr: 24000K .......... .......... .......... .......... .......... 14% 22.5M 9s 2024-03-29T04:19:16.324 INFO:teuthology.orchestra.run.smithi022.stderr: 24050K .......... .......... .......... .......... .......... 14% 20.6M 9s 2024-03-29T04:19:16.327 INFO:teuthology.orchestra.run.smithi022.stderr: 24100K .......... .......... .......... .......... .......... 14% 16.2M 9s 2024-03-29T04:19:16.331 INFO:teuthology.orchestra.run.smithi022.stderr: 24150K .......... .......... .......... .......... .......... 14% 13.3M 9s 2024-03-29T04:19:16.334 INFO:teuthology.orchestra.run.smithi022.stderr: 24200K .......... .......... .......... .......... .......... 14% 15.1M 9s 2024-03-29T04:19:16.337 INFO:teuthology.orchestra.run.smithi022.stderr: 24250K .......... .......... .......... .......... .......... 14% 15.6M 9s 2024-03-29T04:19:16.340 INFO:teuthology.orchestra.run.smithi022.stderr: 24300K .......... .......... .......... .......... .......... 14% 15.3M 9s 2024-03-29T04:19:16.344 INFO:teuthology.orchestra.run.smithi022.stderr: 24350K .......... .......... .......... .......... .......... 14% 13.0M 9s 2024-03-29T04:19:16.347 INFO:teuthology.orchestra.run.smithi022.stderr: 24400K .......... .......... .......... .......... .......... 14% 15.1M 9s 2024-03-29T04:19:16.350 INFO:teuthology.orchestra.run.smithi022.stderr: 24450K .......... .......... .......... .......... .......... 14% 18.1M 9s 2024-03-29T04:19:16.352 INFO:teuthology.orchestra.run.smithi022.stderr: 24500K .......... .......... .......... .......... .......... 14% 21.7M 9s 2024-03-29T04:19:16.355 INFO:teuthology.orchestra.run.smithi022.stderr: 24550K .......... .......... .......... .......... .......... 14% 17.7M 9s 2024-03-29T04:19:16.356 INFO:teuthology.orchestra.run.smithi022.stderr: 24600K .......... .......... .......... .......... .......... 14% 31.7M 9s 2024-03-29T04:19:16.358 INFO:teuthology.orchestra.run.smithi022.stderr: 24650K .......... .......... .......... .......... .......... 14% 29.7M 9s 2024-03-29T04:19:16.360 INFO:teuthology.orchestra.run.smithi022.stderr: 24700K .......... .......... .......... .......... .......... 14% 28.3M 9s 2024-03-29T04:19:16.362 INFO:teuthology.orchestra.run.smithi022.stderr: 24750K .......... .......... .......... .......... .......... 14% 22.8M 9s 2024-03-29T04:19:16.363 INFO:teuthology.orchestra.run.smithi022.stderr: 24800K .......... .......... .......... .......... .......... 14% 29.2M 9s 2024-03-29T04:19:16.365 INFO:teuthology.orchestra.run.smithi022.stderr: 24850K .......... .......... .......... .......... .......... 14% 32.9M 9s 2024-03-29T04:19:16.366 INFO:teuthology.orchestra.run.smithi022.stderr: 24900K .......... .......... .......... .......... .......... 14% 33.0M 9s 2024-03-29T04:19:16.368 INFO:teuthology.orchestra.run.smithi022.stderr: 24950K .......... .......... .......... .......... .......... 14% 27.8M 9s 2024-03-29T04:19:16.370 INFO:teuthology.orchestra.run.smithi022.stderr: 25000K .......... .......... .......... .......... .......... 14% 33.0M 9s 2024-03-29T04:19:16.371 INFO:teuthology.orchestra.run.smithi022.stderr: 25050K .......... .......... .......... .......... .......... 14% 30.1M 9s 2024-03-29T04:19:16.373 INFO:teuthology.orchestra.run.smithi022.stderr: 25100K .......... .......... .......... .......... .......... 14% 29.2M 9s 2024-03-29T04:19:16.375 INFO:teuthology.orchestra.run.smithi022.stderr: 25150K .......... .......... .......... .......... .......... 14% 25.0M 9s 2024-03-29T04:19:16.377 INFO:teuthology.orchestra.run.smithi022.stderr: 25200K .......... .......... .......... .......... .......... 14% 27.3M 9s 2024-03-29T04:19:16.378 INFO:teuthology.orchestra.run.smithi022.stderr: 25250K .......... .......... .......... .......... .......... 15% 27.1M 8s 2024-03-29T04:19:16.380 INFO:teuthology.orchestra.run.smithi022.stderr: 25300K .......... .......... .......... .......... .......... 15% 27.7M 8s 2024-03-29T04:19:16.382 INFO:teuthology.orchestra.run.smithi022.stderr: 25350K .......... .......... .......... .......... .......... 15% 24.3M 8s 2024-03-29T04:19:16.384 INFO:teuthology.orchestra.run.smithi022.stderr: 25400K .......... .......... .......... .......... .......... 15% 25.2M 8s 2024-03-29T04:19:16.386 INFO:teuthology.orchestra.run.smithi022.stderr: 25450K .......... .......... .......... .......... .......... 15% 27.1M 8s 2024-03-29T04:19:16.388 INFO:teuthology.orchestra.run.smithi022.stderr: 25500K .......... .......... .......... .......... .......... 15% 27.8M 8s 2024-03-29T04:19:16.390 INFO:teuthology.orchestra.run.smithi022.stderr: 25550K .......... .......... .......... .......... .......... 15% 23.9M 8s 2024-03-29T04:19:16.391 INFO:teuthology.orchestra.run.smithi022.stderr: 25600K .......... .......... .......... .......... .......... 15% 27.9M 8s 2024-03-29T04:19:16.393 INFO:teuthology.orchestra.run.smithi022.stderr: 25650K .......... .......... .......... .......... .......... 15% 28.6M 8s 2024-03-29T04:19:16.395 INFO:teuthology.orchestra.run.smithi022.stderr: 25700K .......... .......... .......... .......... .......... 15% 28.0M 8s 2024-03-29T04:19:16.397 INFO:teuthology.orchestra.run.smithi022.stderr: 25750K .......... .......... .......... .......... .......... 15% 23.2M 8s 2024-03-29T04:19:16.399 INFO:teuthology.orchestra.run.smithi022.stderr: 25800K .......... .......... .......... .......... .......... 15% 26.7M 8s 2024-03-29T04:19:16.401 INFO:teuthology.orchestra.run.smithi022.stderr: 25850K .......... .......... .......... .......... .......... 15% 27.6M 8s 2024-03-29T04:19:16.402 INFO:teuthology.orchestra.run.smithi022.stderr: 25900K .......... .......... .......... .......... .......... 15% 27.7M 8s 2024-03-29T04:19:16.404 INFO:teuthology.orchestra.run.smithi022.stderr: 25950K .......... .......... .......... .......... .......... 15% 22.8M 8s 2024-03-29T04:19:16.406 INFO:teuthology.orchestra.run.smithi022.stderr: 26000K .......... .......... .......... .......... .......... 15% 27.7M 8s 2024-03-29T04:19:16.411 INFO:teuthology.orchestra.run.smithi022.stderr: 26050K .......... .......... .......... .......... .......... 15% 27.7M 8s 2024-03-29T04:19:16.412 INFO:teuthology.orchestra.run.smithi022.stderr: 26100K .......... .......... .......... .......... .......... 15% 26.0M 8s 2024-03-29T04:19:16.412 INFO:teuthology.orchestra.run.smithi022.stderr: 26150K .......... .......... .......... .......... .......... 15% 22.8M 8s 2024-03-29T04:19:16.414 INFO:teuthology.orchestra.run.smithi022.stderr: 26200K .......... .......... .......... .......... .......... 15% 27.9M 8s 2024-03-29T04:19:16.416 INFO:teuthology.orchestra.run.smithi022.stderr: 26250K .......... .......... .......... .......... .......... 15% 26.4M 8s 2024-03-29T04:19:16.418 INFO:teuthology.orchestra.run.smithi022.stderr: 26300K .......... .......... .......... .......... .......... 15% 22.7M 8s 2024-03-29T04:19:16.421 INFO:teuthology.orchestra.run.smithi022.stderr: 26350K .......... .......... .......... .......... .......... 15% 18.4M 8s 2024-03-29T04:19:16.423 INFO:teuthology.orchestra.run.smithi022.stderr: 26400K .......... .......... .......... .......... .......... 15% 22.3M 8s 2024-03-29T04:19:16.425 INFO:teuthology.orchestra.run.smithi022.stderr: 26450K .......... .......... .......... .......... .......... 15% 22.0M 8s 2024-03-29T04:19:16.427 INFO:teuthology.orchestra.run.smithi022.stderr: 26500K .......... .......... .......... .......... .......... 15% 21.7M 8s 2024-03-29T04:19:16.430 INFO:teuthology.orchestra.run.smithi022.stderr: 26550K .......... .......... .......... .......... .......... 15% 18.6M 8s 2024-03-29T04:19:16.432 INFO:teuthology.orchestra.run.smithi022.stderr: 26600K .......... .......... .......... .......... .......... 15% 22.1M 8s 2024-03-29T04:19:16.434 INFO:teuthology.orchestra.run.smithi022.stderr: 26650K .......... .......... .......... .......... .......... 15% 23.7M 8s 2024-03-29T04:19:16.436 INFO:teuthology.orchestra.run.smithi022.stderr: 26700K .......... .......... .......... .......... .......... 15% 23.5M 8s 2024-03-29T04:19:16.439 INFO:teuthology.orchestra.run.smithi022.stderr: 26750K .......... .......... .......... .......... .......... 15% 19.4M 8s 2024-03-29T04:19:16.441 INFO:teuthology.orchestra.run.smithi022.stderr: 26800K .......... .......... .......... .......... .......... 15% 18.8M 8s 2024-03-29T04:19:16.443 INFO:teuthology.orchestra.run.smithi022.stderr: 26850K .......... .......... .......... .......... .......... 15% 21.5M 8s 2024-03-29T04:19:16.446 INFO:teuthology.orchestra.run.smithi022.stderr: 26900K .......... .......... .......... .......... .......... 15% 20.2M 8s 2024-03-29T04:19:16.449 INFO:teuthology.orchestra.run.smithi022.stderr: 26950K .......... .......... .......... .......... .......... 16% 16.5M 8s 2024-03-29T04:19:16.452 INFO:teuthology.orchestra.run.smithi022.stderr: 27000K .......... .......... .......... .......... .......... 16% 19.3M 8s 2024-03-29T04:19:16.454 INFO:teuthology.orchestra.run.smithi022.stderr: 27050K .......... .......... .......... .......... .......... 16% 19.7M 8s 2024-03-29T04:19:16.456 INFO:teuthology.orchestra.run.smithi022.stderr: 27100K .......... .......... .......... .......... .......... 16% 19.6M 8s 2024-03-29T04:19:16.460 INFO:teuthology.orchestra.run.smithi022.stderr: 27150K .......... .......... .......... .......... .......... 16% 15.8M 8s 2024-03-29T04:19:16.462 INFO:teuthology.orchestra.run.smithi022.stderr: 27200K .......... .......... .......... .......... .......... 16% 19.1M 8s 2024-03-29T04:19:16.464 INFO:teuthology.orchestra.run.smithi022.stderr: 27250K .......... .......... .......... .......... .......... 16% 20.3M 8s 2024-03-29T04:19:16.467 INFO:teuthology.orchestra.run.smithi022.stderr: 27300K .......... .......... .......... .......... .......... 16% 22.0M 8s 2024-03-29T04:19:16.469 INFO:teuthology.orchestra.run.smithi022.stderr: 27350K .......... .......... .......... .......... .......... 16% 19.0M 8s 2024-03-29T04:19:16.471 INFO:teuthology.orchestra.run.smithi022.stderr: 27400K .......... .......... .......... .......... .......... 16% 21.8M 8s 2024-03-29T04:19:16.474 INFO:teuthology.orchestra.run.smithi022.stderr: 27450K .......... .......... .......... .......... .......... 16% 21.8M 8s 2024-03-29T04:19:16.476 INFO:teuthology.orchestra.run.smithi022.stderr: 27500K .......... .......... .......... .......... .......... 16% 21.8M 8s 2024-03-29T04:19:16.479 INFO:teuthology.orchestra.run.smithi022.stderr: 27550K .......... .......... .......... .......... .......... 16% 18.7M 8s 2024-03-29T04:19:16.481 INFO:teuthology.orchestra.run.smithi022.stderr: 27600K .......... .......... .......... .......... .......... 16% 21.5M 8s 2024-03-29T04:19:16.483 INFO:teuthology.orchestra.run.smithi022.stderr: 27650K .......... .......... .......... .......... .......... 16% 21.5M 8s 2024-03-29T04:19:16.485 INFO:teuthology.orchestra.run.smithi022.stderr: 27700K .......... .......... .......... .......... .......... 16% 21.3M 8s 2024-03-29T04:19:16.488 INFO:teuthology.orchestra.run.smithi022.stderr: 27750K .......... .......... .......... .......... .......... 16% 17.6M 8s 2024-03-29T04:19:16.491 INFO:teuthology.orchestra.run.smithi022.stderr: 27800K .......... .......... .......... .......... .......... 16% 20.0M 8s 2024-03-29T04:19:16.493 INFO:teuthology.orchestra.run.smithi022.stderr: 27850K .......... .......... .......... .......... .......... 16% 21.0M 8s 2024-03-29T04:19:16.495 INFO:teuthology.orchestra.run.smithi022.stderr: 27900K .......... .......... .......... .......... .......... 16% 21.5M 8s 2024-03-29T04:19:16.498 INFO:teuthology.orchestra.run.smithi022.stderr: 27950K .......... .......... .......... .......... .......... 16% 17.6M 8s 2024-03-29T04:19:16.501 INFO:teuthology.orchestra.run.smithi022.stderr: 28000K .......... .......... .......... .......... .......... 16% 19.6M 8s 2024-03-29T04:19:16.503 INFO:teuthology.orchestra.run.smithi022.stderr: 28050K .......... .......... .......... .......... .......... 16% 19.2M 8s 2024-03-29T04:19:16.506 INFO:teuthology.orchestra.run.smithi022.stderr: 28100K .......... .......... .......... .......... .......... 16% 20.4M 8s 2024-03-29T04:19:16.508 INFO:teuthology.orchestra.run.smithi022.stderr: 28150K .......... .......... .......... .......... .......... 16% 18.1M 8s 2024-03-29T04:19:16.511 INFO:teuthology.orchestra.run.smithi022.stderr: 28200K .......... .......... .......... .......... .......... 16% 21.1M 8s 2024-03-29T04:19:16.513 INFO:teuthology.orchestra.run.smithi022.stderr: 28250K .......... .......... .......... .......... .......... 16% 20.7M 8s 2024-03-29T04:19:16.515 INFO:teuthology.orchestra.run.smithi022.stderr: 28300K .......... .......... .......... .......... .......... 16% 20.8M 8s 2024-03-29T04:19:16.518 INFO:teuthology.orchestra.run.smithi022.stderr: 28350K .......... .......... .......... .......... .......... 16% 18.1M 8s 2024-03-29T04:19:16.521 INFO:teuthology.orchestra.run.smithi022.stderr: 28400K .......... .......... .......... .......... .......... 16% 18.7M 8s 2024-03-29T04:19:16.523 INFO:teuthology.orchestra.run.smithi022.stderr: 28450K .......... .......... .......... .......... .......... 16% 20.5M 8s 2024-03-29T04:19:16.525 INFO:teuthology.orchestra.run.smithi022.stderr: 28500K .......... .......... .......... .......... .......... 16% 19.4M 8s 2024-03-29T04:19:16.529 INFO:teuthology.orchestra.run.smithi022.stderr: 28550K .......... .......... .......... .......... .......... 16% 15.2M 8s 2024-03-29T04:19:16.531 INFO:teuthology.orchestra.run.smithi022.stderr: 28600K .......... .......... .......... .......... .......... 16% 19.8M 8s 2024-03-29T04:19:16.534 INFO:teuthology.orchestra.run.smithi022.stderr: 28650K .......... .......... .......... .......... .......... 17% 18.4M 8s 2024-03-29T04:19:16.536 INFO:teuthology.orchestra.run.smithi022.stderr: 28700K .......... .......... .......... .......... .......... 17% 23.9M 8s 2024-03-29T04:19:16.538 INFO:teuthology.orchestra.run.smithi022.stderr: 28750K .......... .......... .......... .......... .......... 17% 19.8M 8s 2024-03-29T04:19:16.542 INFO:teuthology.orchestra.run.smithi022.stderr: 28800K .......... .......... .......... .......... .......... 17% 14.9M 8s 2024-03-29T04:19:16.543 INFO:teuthology.orchestra.run.smithi022.stderr: 28850K .......... .......... .......... .......... .......... 17% 24.3M 8s 2024-03-29T04:19:16.545 INFO:teuthology.orchestra.run.smithi022.stderr: 28900K .......... .......... .......... .......... .......... 17% 23.9M 8s 2024-03-29T04:19:16.547 INFO:teuthology.orchestra.run.smithi022.stderr: 28950K .......... .......... .......... .......... .......... 17% 23.3M 8s 2024-03-29T04:19:16.549 INFO:teuthology.orchestra.run.smithi022.stderr: 29000K .......... .......... .......... .......... .......... 17% 29.0M 8s 2024-03-29T04:19:16.551 INFO:teuthology.orchestra.run.smithi022.stderr: 29050K .......... .......... .......... .......... .......... 17% 32.0M 8s 2024-03-29T04:19:16.552 INFO:teuthology.orchestra.run.smithi022.stderr: 29100K .......... .......... .......... .......... .......... 17% 32.4M 8s 2024-03-29T04:19:16.554 INFO:teuthology.orchestra.run.smithi022.stderr: 29150K .......... .......... .......... .......... .......... 17% 26.7M 8s 2024-03-29T04:19:16.556 INFO:teuthology.orchestra.run.smithi022.stderr: 29200K .......... .......... .......... .......... .......... 17% 30.0M 8s 2024-03-29T04:19:16.557 INFO:teuthology.orchestra.run.smithi022.stderr: 29250K .......... .......... .......... .......... .......... 17% 31.5M 8s 2024-03-29T04:19:16.559 INFO:teuthology.orchestra.run.smithi022.stderr: 29300K .......... .......... .......... .......... .......... 17% 31.4M 8s 2024-03-29T04:19:16.561 INFO:teuthology.orchestra.run.smithi022.stderr: 29350K .......... .......... .......... .......... .......... 17% 27.7M 8s 2024-03-29T04:19:16.562 INFO:teuthology.orchestra.run.smithi022.stderr: 29400K .......... .......... .......... .......... .......... 17% 29.5M 8s 2024-03-29T04:19:16.564 INFO:teuthology.orchestra.run.smithi022.stderr: 29450K .......... .......... .......... .......... .......... 17% 29.4M 8s 2024-03-29T04:19:16.566 INFO:teuthology.orchestra.run.smithi022.stderr: 29500K .......... .......... .......... .......... .......... 17% 30.9M 8s 2024-03-29T04:19:16.568 INFO:teuthology.orchestra.run.smithi022.stderr: 29550K .......... .......... .......... .......... .......... 17% 26.0M 8s 2024-03-29T04:19:16.569 INFO:teuthology.orchestra.run.smithi022.stderr: 29600K .......... .......... .......... .......... .......... 17% 29.8M 8s 2024-03-29T04:19:16.571 INFO:teuthology.orchestra.run.smithi022.stderr: 29650K .......... .......... .......... .......... .......... 17% 29.8M 8s 2024-03-29T04:19:16.572 INFO:teuthology.orchestra.run.smithi022.stderr: 29700K .......... .......... .......... .......... .......... 17% 30.5M 8s 2024-03-29T04:19:16.574 INFO:teuthology.orchestra.run.smithi022.stderr: 29750K .......... .......... .......... .......... .......... 17% 25.8M 8s 2024-03-29T04:19:16.576 INFO:teuthology.orchestra.run.smithi022.stderr: 29800K .......... .......... .......... .......... .......... 17% 28.0M 8s 2024-03-29T04:19:16.578 INFO:teuthology.orchestra.run.smithi022.stderr: 29850K .......... .......... .......... .......... .......... 17% 28.5M 8s 2024-03-29T04:19:16.579 INFO:teuthology.orchestra.run.smithi022.stderr: 29900K .......... .......... .......... .......... .......... 17% 29.2M 8s 2024-03-29T04:19:16.581 INFO:teuthology.orchestra.run.smithi022.stderr: 29950K .......... .......... .......... .......... .......... 17% 24.6M 8s 2024-03-29T04:19:16.583 INFO:teuthology.orchestra.run.smithi022.stderr: 30000K .......... .......... .......... .......... .......... 17% 27.7M 8s 2024-03-29T04:19:16.585 INFO:teuthology.orchestra.run.smithi022.stderr: 30050K .......... .......... .......... .......... .......... 17% 28.2M 8s 2024-03-29T04:19:16.586 INFO:teuthology.orchestra.run.smithi022.stderr: 30100K .......... .......... .......... .......... .......... 17% 28.3M 8s 2024-03-29T04:19:16.589 INFO:teuthology.orchestra.run.smithi022.stderr: 30150K .......... .......... .......... .......... .......... 17% 22.7M 8s 2024-03-29T04:19:16.590 INFO:teuthology.orchestra.run.smithi022.stderr: 30200K .......... .......... .......... .......... .......... 17% 28.8M 8s 2024-03-29T04:19:16.592 INFO:teuthology.orchestra.run.smithi022.stderr: 30250K .......... .......... .......... .......... .......... 17% 28.1M 8s 2024-03-29T04:19:16.594 INFO:teuthology.orchestra.run.smithi022.stderr: 30300K .......... .......... .......... .......... .......... 18% 28.1M 8s 2024-03-29T04:19:16.596 INFO:teuthology.orchestra.run.smithi022.stderr: 30350K .......... .......... .......... .......... .......... 18% 22.4M 8s 2024-03-29T04:19:16.598 INFO:teuthology.orchestra.run.smithi022.stderr: 30400K .......... .......... .......... .......... .......... 18% 27.0M 8s 2024-03-29T04:19:16.600 INFO:teuthology.orchestra.run.smithi022.stderr: 30450K .......... .......... .......... .......... .......... 18% 26.9M 8s 2024-03-29T04:19:16.601 INFO:teuthology.orchestra.run.smithi022.stderr: 30500K .......... .......... .......... .......... .......... 18% 27.4M 8s 2024-03-29T04:19:16.604 INFO:teuthology.orchestra.run.smithi022.stderr: 30550K .......... .......... .......... .......... .......... 18% 21.7M 8s 2024-03-29T04:19:16.606 INFO:teuthology.orchestra.run.smithi022.stderr: 30600K .......... .......... .......... .......... .......... 18% 26.2M 8s 2024-03-29T04:19:16.607 INFO:teuthology.orchestra.run.smithi022.stderr: 30650K .......... .......... .......... .......... .......... 18% 26.6M 8s 2024-03-29T04:19:16.609 INFO:teuthology.orchestra.run.smithi022.stderr: 30700K .......... .......... .......... .......... .......... 18% 26.0M 8s 2024-03-29T04:19:16.615 INFO:teuthology.orchestra.run.smithi022.stderr: 30750K .......... .......... .......... .......... .......... 18% 22.5M 8s 2024-03-29T04:19:16.615 INFO:teuthology.orchestra.run.smithi022.stderr: 30800K .......... .......... .......... .......... .......... 18% 26.3M 8s 2024-03-29T04:19:16.615 INFO:teuthology.orchestra.run.smithi022.stderr: 30850K .......... .......... .......... .......... .......... 18% 26.3M 8s 2024-03-29T04:19:16.617 INFO:teuthology.orchestra.run.smithi022.stderr: 30900K .......... .......... .......... .......... .......... 18% 25.0M 8s 2024-03-29T04:19:16.619 INFO:teuthology.orchestra.run.smithi022.stderr: 30950K .......... .......... .......... .......... .......... 18% 22.1M 8s 2024-03-29T04:19:16.621 INFO:teuthology.orchestra.run.smithi022.stderr: 31000K .......... .......... .......... .......... .......... 18% 25.4M 8s 2024-03-29T04:19:16.623 INFO:teuthology.orchestra.run.smithi022.stderr: 31050K .......... .......... .......... .......... .......... 18% 23.8M 8s 2024-03-29T04:19:16.625 INFO:teuthology.orchestra.run.smithi022.stderr: 31100K .......... .......... .......... .......... .......... 18% 26.0M 8s 2024-03-29T04:19:16.627 INFO:teuthology.orchestra.run.smithi022.stderr: 31150K .......... .......... .......... .......... .......... 18% 22.3M 8s 2024-03-29T04:19:16.629 INFO:teuthology.orchestra.run.smithi022.stderr: 31200K .......... .......... .......... .......... .......... 18% 25.9M 8s 2024-03-29T04:19:16.631 INFO:teuthology.orchestra.run.smithi022.stderr: 31250K .......... .......... .......... .......... .......... 18% 24.7M 8s 2024-03-29T04:19:16.634 INFO:teuthology.orchestra.run.smithi022.stderr: 31300K .......... .......... .......... .......... .......... 18% 17.5M 8s 2024-03-29T04:19:16.638 INFO:teuthology.orchestra.run.smithi022.stderr: 31350K .......... .......... .......... .......... .......... 18% 13.7M 8s 2024-03-29T04:19:16.641 INFO:teuthology.orchestra.run.smithi022.stderr: 31400K .......... .......... .......... .......... .......... 18% 17.3M 8s 2024-03-29T04:19:16.644 INFO:teuthology.orchestra.run.smithi022.stderr: 31450K .......... .......... .......... .......... .......... 18% 16.6M 8s 2024-03-29T04:19:16.647 INFO:teuthology.orchestra.run.smithi022.stderr: 31500K .......... .......... .......... .......... .......... 18% 13.0M 8s 2024-03-29T04:19:16.652 INFO:teuthology.orchestra.run.smithi022.stderr: 31550K .......... .......... .......... .......... .......... 18% 10.0M 8s 2024-03-29T04:19:16.657 INFO:teuthology.orchestra.run.smithi022.stderr: 31600K .......... .......... .......... .......... .......... 18% 11.3M 8s 2024-03-29T04:19:16.661 INFO:teuthology.orchestra.run.smithi022.stderr: 31650K .......... .......... .......... .......... .......... 18% 11.6M 8s 2024-03-29T04:19:16.665 INFO:teuthology.orchestra.run.smithi022.stderr: 31700K .......... .......... .......... .......... .......... 18% 13.1M 8s 2024-03-29T04:19:16.668 INFO:teuthology.orchestra.run.smithi022.stderr: 31750K .......... .......... .......... .......... .......... 18% 12.9M 8s 2024-03-29T04:19:16.672 INFO:teuthology.orchestra.run.smithi022.stderr: 31800K .......... .......... .......... .......... .......... 18% 15.2M 8s 2024-03-29T04:19:16.675 INFO:teuthology.orchestra.run.smithi022.stderr: 31850K .......... .......... .......... .......... .......... 18% 15.3M 8s 2024-03-29T04:19:16.679 INFO:teuthology.orchestra.run.smithi022.stderr: 31900K .......... .......... .......... .......... .......... 18% 11.0M 8s 2024-03-29T04:19:16.685 INFO:teuthology.orchestra.run.smithi022.stderr: 31950K .......... .......... .......... .......... .......... 18% 9.57M 8s 2024-03-29T04:19:16.689 INFO:teuthology.orchestra.run.smithi022.stderr: 32000K .......... .......... .......... .......... .......... 19% 10.3M 8s 2024-03-29T04:19:16.693 INFO:teuthology.orchestra.run.smithi022.stderr: 32050K .......... .......... .......... .......... .......... 19% 10.9M 8s 2024-03-29T04:19:16.698 INFO:teuthology.orchestra.run.smithi022.stderr: 32100K .......... .......... .......... .......... .......... 19% 10.8M 8s 2024-03-29T04:19:16.703 INFO:teuthology.orchestra.run.smithi022.stderr: 32150K .......... .......... .......... .......... .......... 19% 8.85M 8s 2024-03-29T04:19:16.708 INFO:teuthology.orchestra.run.smithi022.stderr: 32200K .......... .......... .......... .......... .......... 19% 10.2M 8s 2024-03-29T04:19:16.712 INFO:teuthology.orchestra.run.smithi022.stderr: 32250K .......... .......... .......... .......... .......... 19% 12.6M 8s 2024-03-29T04:19:16.717 INFO:teuthology.orchestra.run.smithi022.stderr: 32300K .......... .......... .......... .......... .......... 19% 11.5M 8s 2024-03-29T04:19:16.722 INFO:teuthology.orchestra.run.smithi022.stderr: 32350K .......... .......... .......... .......... .......... 19% 8.38M 8s 2024-03-29T04:19:16.727 INFO:teuthology.orchestra.run.smithi022.stderr: 32400K .......... .......... .......... .......... .......... 19% 10.0M 8s 2024-03-29T04:19:16.732 INFO:teuthology.orchestra.run.smithi022.stderr: 32450K .......... .......... .......... .......... .......... 19% 11.1M 8s 2024-03-29T04:19:16.736 INFO:teuthology.orchestra.run.smithi022.stderr: 32500K .......... .......... .......... .......... .......... 19% 12.6M 8s 2024-03-29T04:19:16.740 INFO:teuthology.orchestra.run.smithi022.stderr: 32550K .......... .......... .......... .......... .......... 19% 11.3M 8s 2024-03-29T04:19:16.743 INFO:teuthology.orchestra.run.smithi022.stderr: 32600K .......... .......... .......... .......... .......... 19% 15.7M 8s 2024-03-29T04:19:16.746 INFO:teuthology.orchestra.run.smithi022.stderr: 32650K .......... .......... .......... .......... .......... 19% 14.8M 8s 2024-03-29T04:19:16.751 INFO:teuthology.orchestra.run.smithi022.stderr: 32700K .......... .......... .......... .......... .......... 19% 10.0M 8s 2024-03-29T04:19:16.756 INFO:teuthology.orchestra.run.smithi022.stderr: 32750K .......... .......... .......... .......... .......... 19% 9.95M 8s 2024-03-29T04:19:16.759 INFO:teuthology.orchestra.run.smithi022.stderr: 32800K .......... .......... .......... .......... .......... 19% 16.8M 8s 2024-03-29T04:19:16.761 INFO:teuthology.orchestra.run.smithi022.stderr: 32850K .......... .......... .......... .......... .......... 19% 20.2M 8s 2024-03-29T04:19:16.763 INFO:teuthology.orchestra.run.smithi022.stderr: 32900K .......... .......... .......... .......... .......... 19% 20.2M 8s 2024-03-29T04:19:16.766 INFO:teuthology.orchestra.run.smithi022.stderr: 32950K .......... .......... .......... .......... .......... 19% 17.9M 8s 2024-03-29T04:19:16.769 INFO:teuthology.orchestra.run.smithi022.stderr: 33000K .......... .......... .......... .......... .......... 19% 16.3M 8s 2024-03-29T04:19:16.773 INFO:teuthology.orchestra.run.smithi022.stderr: 33050K .......... .......... .......... .......... .......... 19% 15.2M 8s 2024-03-29T04:19:16.775 INFO:teuthology.orchestra.run.smithi022.stderr: 33100K .......... .......... .......... .......... .......... 19% 19.7M 8s 2024-03-29T04:19:16.777 INFO:teuthology.orchestra.run.smithi022.stderr: 33150K .......... .......... .......... .......... .......... 19% 23.5M 8s 2024-03-29T04:19:16.779 INFO:teuthology.orchestra.run.smithi022.stderr: 33200K .......... .......... .......... .......... .......... 19% 26.0M 8s 2024-03-29T04:19:16.780 INFO:teuthology.orchestra.run.smithi022.stderr: 33250K .......... .......... .......... .......... .......... 19% 28.0M 8s 2024-03-29T04:19:16.782 INFO:teuthology.orchestra.run.smithi022.stderr: 33300K .......... .......... .......... .......... .......... 19% 28.9M 8s 2024-03-29T04:19:16.785 INFO:teuthology.orchestra.run.smithi022.stderr: 33350K .......... .......... .......... .......... .......... 19% 20.2M 8s 2024-03-29T04:19:16.787 INFO:teuthology.orchestra.run.smithi022.stderr: 33400K .......... .......... .......... .......... .......... 19% 20.0M 8s 2024-03-29T04:19:16.791 INFO:teuthology.orchestra.run.smithi022.stderr: 33450K .......... .......... .......... .......... .......... 19% 12.0M 8s 2024-03-29T04:19:16.793 INFO:teuthology.orchestra.run.smithi022.stderr: 33500K .......... .......... .......... .......... .......... 19% 24.2M 8s 2024-03-29T04:19:16.796 INFO:teuthology.orchestra.run.smithi022.stderr: 33550K .......... .......... .......... .......... .......... 19% 18.9M 8s 2024-03-29T04:19:16.798 INFO:teuthology.orchestra.run.smithi022.stderr: 33600K .......... .......... .......... .......... .......... 19% 22.5M 8s 2024-03-29T04:19:16.800 INFO:teuthology.orchestra.run.smithi022.stderr: 33650K .......... .......... .......... .......... .......... 19% 23.5M 8s 2024-03-29T04:19:16.802 INFO:teuthology.orchestra.run.smithi022.stderr: 33700K .......... .......... .......... .......... .......... 20% 24.8M 8s 2024-03-29T04:19:16.804 INFO:teuthology.orchestra.run.smithi022.stderr: 33750K .......... .......... .......... .......... .......... 20% 19.8M 8s 2024-03-29T04:19:16.807 INFO:teuthology.orchestra.run.smithi022.stderr: 33800K .......... .......... .......... .......... .......... 20% 23.8M 8s 2024-03-29T04:19:16.809 INFO:teuthology.orchestra.run.smithi022.stderr: 33850K .......... .......... .......... .......... .......... 20% 23.2M 8s 2024-03-29T04:19:16.812 INFO:teuthology.orchestra.run.smithi022.stderr: 33900K .......... .......... .......... .......... .......... 20% 23.8M 8s 2024-03-29T04:19:16.813 INFO:teuthology.orchestra.run.smithi022.stderr: 33950K .......... .......... .......... .......... .......... 20% 19.8M 8s 2024-03-29T04:19:16.821 INFO:teuthology.orchestra.run.smithi022.stderr: 34000K .......... .......... .......... .......... .......... 20% 25.4M 8s 2024-03-29T04:19:16.821 INFO:teuthology.orchestra.run.smithi022.stderr: 34050K .......... .......... .......... .......... .......... 20% 29.7M 8s 2024-03-29T04:19:16.821 INFO:teuthology.orchestra.run.smithi022.stderr: 34100K .......... .......... .......... .......... .......... 20% 25.9M 8s 2024-03-29T04:19:16.822 INFO:teuthology.orchestra.run.smithi022.stderr: 34150K .......... .......... .......... .......... .......... 20% 19.7M 8s 2024-03-29T04:19:16.824 INFO:teuthology.orchestra.run.smithi022.stderr: 34200K .......... .......... .......... .......... .......... 20% 22.0M 8s 2024-03-29T04:19:16.825 INFO:teuthology.orchestra.run.smithi022.stderr: 34250K .......... .......... .......... .......... .......... 20% 22.8M 8s 2024-03-29T04:19:16.828 INFO:teuthology.orchestra.run.smithi022.stderr: 34300K .......... .......... .......... .......... .......... 20% 22.9M 8s 2024-03-29T04:19:16.830 INFO:teuthology.orchestra.run.smithi022.stderr: 34350K .......... .......... .......... .......... .......... 20% 18.4M 8s 2024-03-29T04:19:16.832 INFO:teuthology.orchestra.run.smithi022.stderr: 34400K .......... .......... .......... .......... .......... 20% 22.4M 8s 2024-03-29T04:19:16.835 INFO:teuthology.orchestra.run.smithi022.stderr: 34450K .......... .......... .......... .......... .......... 20% 22.4M 8s 2024-03-29T04:19:16.837 INFO:teuthology.orchestra.run.smithi022.stderr: 34500K .......... .......... .......... .......... .......... 20% 22.4M 8s 2024-03-29T04:19:16.839 INFO:teuthology.orchestra.run.smithi022.stderr: 34550K .......... .......... .......... .......... .......... 20% 19.0M 8s 2024-03-29T04:19:16.841 INFO:teuthology.orchestra.run.smithi022.stderr: 34600K .......... .......... .......... .......... .......... 20% 22.4M 8s 2024-03-29T04:19:16.845 INFO:teuthology.orchestra.run.smithi022.stderr: 34650K .......... .......... .......... .......... .......... 20% 20.6M 8s 2024-03-29T04:19:16.846 INFO:teuthology.orchestra.run.smithi022.stderr: 34700K .......... .......... .......... .......... .......... 20% 22.2M 8s 2024-03-29T04:19:16.849 INFO:teuthology.orchestra.run.smithi022.stderr: 34750K .......... .......... .......... .......... .......... 20% 23.1M 8s 2024-03-29T04:19:16.850 INFO:teuthology.orchestra.run.smithi022.stderr: 34800K .......... .......... .......... .......... .......... 20% 22.1M 8s 2024-03-29T04:19:16.853 INFO:teuthology.orchestra.run.smithi022.stderr: 34850K .......... .......... .......... .......... .......... 20% 20.4M 8s 2024-03-29T04:19:16.856 INFO:teuthology.orchestra.run.smithi022.stderr: 34900K .......... .......... .......... .......... .......... 20% 16.9M 8s 2024-03-29T04:19:16.859 INFO:teuthology.orchestra.run.smithi022.stderr: 34950K .......... .......... .......... .......... .......... 20% 15.8M 8s 2024-03-29T04:19:16.862 INFO:teuthology.orchestra.run.smithi022.stderr: 35000K .......... .......... .......... .......... .......... 20% 17.8M 8s 2024-03-29T04:19:16.865 INFO:teuthology.orchestra.run.smithi022.stderr: 35050K .......... .......... .......... .......... .......... 20% 17.1M 8s 2024-03-29T04:19:16.867 INFO:teuthology.orchestra.run.smithi022.stderr: 35100K .......... .......... .......... .......... .......... 20% 17.7M 8s 2024-03-29T04:19:16.870 INFO:teuthology.orchestra.run.smithi022.stderr: 35150K .......... .......... .......... .......... .......... 20% 14.8M 8s 2024-03-29T04:19:16.872 INFO:teuthology.orchestra.run.smithi022.stderr: 35200K .......... .......... .......... .......... .......... 20% 24.8M 8s 2024-03-29T04:19:16.874 INFO:teuthology.orchestra.run.smithi022.stderr: 35250K .......... .......... .......... .......... .......... 20% 25.8M 8s 2024-03-29T04:19:16.877 INFO:teuthology.orchestra.run.smithi022.stderr: 35300K .......... .......... .......... .......... .......... 20% 24.0M 8s 2024-03-29T04:19:16.879 INFO:teuthology.orchestra.run.smithi022.stderr: 35350K .......... .......... .......... .......... .......... 21% 20.6M 8s 2024-03-29T04:19:16.881 INFO:teuthology.orchestra.run.smithi022.stderr: 35400K .......... .......... .......... .......... .......... 21% 24.1M 8s 2024-03-29T04:19:16.883 INFO:teuthology.orchestra.run.smithi022.stderr: 35450K .......... .......... .......... .......... .......... 21% 24.0M 8s 2024-03-29T04:19:16.885 INFO:teuthology.orchestra.run.smithi022.stderr: 35500K .......... .......... .......... .......... .......... 21% 24.1M 8s 2024-03-29T04:19:16.887 INFO:teuthology.orchestra.run.smithi022.stderr: 35550K .......... .......... .......... .......... .......... 21% 20.9M 8s 2024-03-29T04:19:16.889 INFO:teuthology.orchestra.run.smithi022.stderr: 35600K .......... .......... .......... .......... .......... 21% 24.0M 7s 2024-03-29T04:19:16.891 INFO:teuthology.orchestra.run.smithi022.stderr: 35650K .......... .......... .......... .......... .......... 21% 22.8M 7s 2024-03-29T04:19:16.893 INFO:teuthology.orchestra.run.smithi022.stderr: 35700K .......... .......... .......... .......... .......... 21% 24.0M 7s 2024-03-29T04:19:16.896 INFO:teuthology.orchestra.run.smithi022.stderr: 35750K .......... .......... .......... .......... .......... 21% 20.2M 7s 2024-03-29T04:19:16.898 INFO:teuthology.orchestra.run.smithi022.stderr: 35800K .......... .......... .......... .......... .......... 21% 24.0M 7s 2024-03-29T04:19:16.900 INFO:teuthology.orchestra.run.smithi022.stderr: 35850K .......... .......... .......... .......... .......... 21% 24.1M 7s 2024-03-29T04:19:16.902 INFO:teuthology.orchestra.run.smithi022.stderr: 35900K .......... .......... .......... .......... .......... 21% 24.9M 7s 2024-03-29T04:19:16.904 INFO:teuthology.orchestra.run.smithi022.stderr: 35950K .......... .......... .......... .......... .......... 21% 21.2M 7s 2024-03-29T04:19:16.906 INFO:teuthology.orchestra.run.smithi022.stderr: 36000K .......... .......... .......... .......... .......... 21% 24.3M 7s 2024-03-29T04:19:16.908 INFO:teuthology.orchestra.run.smithi022.stderr: 36050K .......... .......... .......... .......... .......... 21% 23.9M 7s 2024-03-29T04:19:16.910 INFO:teuthology.orchestra.run.smithi022.stderr: 36100K .......... .......... .......... .......... .......... 21% 24.4M 7s 2024-03-29T04:19:16.913 INFO:teuthology.orchestra.run.smithi022.stderr: 36150K .......... .......... .......... .......... .......... 21% 19.6M 7s 2024-03-29T04:19:16.915 INFO:teuthology.orchestra.run.smithi022.stderr: 36200K .......... .......... .......... .......... .......... 21% 22.9M 7s 2024-03-29T04:19:16.919 INFO:teuthology.orchestra.run.smithi022.stderr: 36250K .......... .......... .......... .......... .......... 21% 22.7M 7s 2024-03-29T04:19:16.920 INFO:teuthology.orchestra.run.smithi022.stderr: 36300K .......... .......... .......... .......... .......... 21% 22.5M 7s 2024-03-29T04:19:16.922 INFO:teuthology.orchestra.run.smithi022.stderr: 36350K .......... .......... .......... .......... .......... 21% 19.7M 7s 2024-03-29T04:19:16.925 INFO:teuthology.orchestra.run.smithi022.stderr: 36400K .......... .......... .......... .......... .......... 21% 17.7M 7s 2024-03-29T04:19:16.927 INFO:teuthology.orchestra.run.smithi022.stderr: 36450K .......... .......... .......... .......... .......... 21% 19.9M 7s 2024-03-29T04:19:16.929 INFO:teuthology.orchestra.run.smithi022.stderr: 36500K .......... .......... .......... .......... .......... 21% 23.9M 7s 2024-03-29T04:19:16.932 INFO:teuthology.orchestra.run.smithi022.stderr: 36550K .......... .......... .......... .......... .......... 21% 21.3M 7s 2024-03-29T04:19:16.933 INFO:teuthology.orchestra.run.smithi022.stderr: 36600K .......... .......... .......... .......... .......... 21% 21.5M 7s 2024-03-29T04:19:16.936 INFO:teuthology.orchestra.run.smithi022.stderr: 36650K .......... .......... .......... .......... .......... 21% 19.2M 7s 2024-03-29T04:19:16.939 INFO:teuthology.orchestra.run.smithi022.stderr: 36700K .......... .......... .......... .......... .......... 21% 22.3M 7s 2024-03-29T04:19:16.942 INFO:teuthology.orchestra.run.smithi022.stderr: 36750K .......... .......... .......... .......... .......... 21% 19.4M 7s 2024-03-29T04:19:16.944 INFO:teuthology.orchestra.run.smithi022.stderr: 36800K .......... .......... .......... .......... .......... 21% 22.3M 7s 2024-03-29T04:19:16.946 INFO:teuthology.orchestra.run.smithi022.stderr: 36850K .......... .......... .......... .......... .......... 21% 16.2M 7s 2024-03-29T04:19:16.949 INFO:teuthology.orchestra.run.smithi022.stderr: 36900K .......... .......... .......... .......... .......... 21% 19.5M 7s 2024-03-29T04:19:16.952 INFO:teuthology.orchestra.run.smithi022.stderr: 36950K .......... .......... .......... .......... .......... 21% 25.9M 7s 2024-03-29T04:19:16.955 INFO:teuthology.orchestra.run.smithi022.stderr: 37000K .......... .......... .......... .......... .......... 21% 19.6M 7s 2024-03-29T04:19:16.958 INFO:teuthology.orchestra.run.smithi022.stderr: 37050K .......... .......... .......... .......... .......... 22% 19.4M 7s 2024-03-29T04:19:16.960 INFO:teuthology.orchestra.run.smithi022.stderr: 37100K .......... .......... .......... .......... .......... 22% 22.0M 7s 2024-03-29T04:19:16.963 INFO:teuthology.orchestra.run.smithi022.stderr: 37150K .......... .......... .......... .......... .......... 22% 19.1M 7s 2024-03-29T04:19:16.963 INFO:teuthology.orchestra.run.smithi022.stderr: 37200K .......... .......... .......... .......... .......... 22% 28.9M 7s 2024-03-29T04:19:16.966 INFO:teuthology.orchestra.run.smithi022.stderr: 37250K .......... .......... .......... .......... .......... 22% 21.2M 7s 2024-03-29T04:19:16.971 INFO:teuthology.orchestra.run.smithi022.stderr: 37300K .......... .......... .......... .......... .......... 22% 24.5M 7s 2024-03-29T04:19:16.976 INFO:teuthology.orchestra.run.smithi022.stderr: 37350K .......... .......... .......... .......... .......... 22% 21.1M 7s 2024-03-29T04:19:16.977 INFO:teuthology.orchestra.run.smithi022.stderr: 37400K .......... .......... .......... .......... .......... 22% 26.0M 7s 2024-03-29T04:19:16.977 INFO:teuthology.orchestra.run.smithi022.stderr: 37450K .......... .......... .......... .......... .......... 22% 25.1M 7s 2024-03-29T04:19:16.979 INFO:teuthology.orchestra.run.smithi022.stderr: 37500K .......... .......... .......... .......... .......... 22% 25.9M 7s 2024-03-29T04:19:16.984 INFO:teuthology.orchestra.run.smithi022.stderr: 37550K .......... .......... .......... .......... .......... 22% 21.8M 7s 2024-03-29T04:19:16.985 INFO:teuthology.orchestra.run.smithi022.stderr: 37600K .......... .......... .......... .......... .......... 22% 24.8M 7s 2024-03-29T04:19:16.985 INFO:teuthology.orchestra.run.smithi022.stderr: 37650K .......... .......... .......... .......... .......... 22% 25.7M 7s 2024-03-29T04:19:16.988 INFO:teuthology.orchestra.run.smithi022.stderr: 37700K .......... .......... .......... .......... .......... 22% 26.1M 7s 2024-03-29T04:19:16.989 INFO:teuthology.orchestra.run.smithi022.stderr: 37750K .......... .......... .......... .......... .......... 22% 21.8M 7s 2024-03-29T04:19:16.989 INFO:teuthology.orchestra.run.smithi022.stderr: 37800K .......... .......... .......... .......... .......... 22% 25.1M 7s 2024-03-29T04:19:16.992 INFO:teuthology.orchestra.run.smithi022.stderr: 37850K .......... .......... .......... .......... .......... 22% 25.2M 7s 2024-03-29T04:19:16.992 INFO:teuthology.orchestra.run.smithi022.stderr: 37900K .......... .......... .......... .......... .......... 22% 24.7M 7s 2024-03-29T04:19:16.993 INFO:teuthology.orchestra.run.smithi022.stderr: 37950K .......... .......... .......... .......... .......... 22% 21.3M 7s 2024-03-29T04:19:16.996 INFO:teuthology.orchestra.run.smithi022.stderr: 38000K .......... .......... .......... .......... .......... 22% 24.7M 7s 2024-03-29T04:19:16.997 INFO:teuthology.orchestra.run.smithi022.stderr: 38050K .......... .......... .......... .......... .......... 22% 24.0M 7s 2024-03-29T04:19:17.000 INFO:teuthology.orchestra.run.smithi022.stderr: 38100K .......... .......... .......... .......... .......... 22% 24.5M 7s 2024-03-29T04:19:17.003 INFO:teuthology.orchestra.run.smithi022.stderr: 38150K .......... .......... .......... .......... .......... 22% 21.0M 7s 2024-03-29T04:19:17.006 INFO:teuthology.orchestra.run.smithi022.stderr: 38200K .......... .......... .......... .......... .......... 22% 29.7M 7s 2024-03-29T04:19:17.007 INFO:teuthology.orchestra.run.smithi022.stderr: 38250K .......... .......... .......... .......... .......... 22% 29.9M 7s 2024-03-29T04:19:17.009 INFO:teuthology.orchestra.run.smithi022.stderr: 38300K .......... .......... .......... .......... .......... 22% 29.9M 7s 2024-03-29T04:19:17.010 INFO:teuthology.orchestra.run.smithi022.stderr: 38350K .......... .......... .......... .......... .......... 22% 26.1M 7s 2024-03-29T04:19:17.016 INFO:teuthology.orchestra.run.smithi022.stderr: 38400K .......... .......... .......... .......... .......... 22% 27.6M 7s 2024-03-29T04:19:17.044 INFO:teuthology.orchestra.run.smithi022.stderr: 38450K .......... .......... .......... .......... .......... 22% 30.1M 7s 2024-03-29T04:19:17.044 INFO:teuthology.orchestra.run.smithi022.stderr: 38500K .......... .......... .......... .......... .......... 22% 27.2M 7s 2024-03-29T04:19:17.044 INFO:teuthology.orchestra.run.smithi022.stderr: 38550K .......... .......... .......... .......... .......... 22% 18.6M 7s 2024-03-29T04:19:17.044 INFO:teuthology.orchestra.run.smithi022.stderr: 38600K .......... .......... .......... .......... .......... 22% 32.2M 7s 2024-03-29T04:19:17.044 INFO:teuthology.orchestra.run.smithi022.stderr: 38650K .......... .......... .......... .......... .......... 22% 31.7M 7s 2024-03-29T04:19:17.044 INFO:teuthology.orchestra.run.smithi022.stderr: 38700K .......... .......... .......... .......... .......... 22% 32.3M 7s 2024-03-29T04:19:17.044 INFO:teuthology.orchestra.run.smithi022.stderr: 38750K .......... .......... .......... .......... .......... 23% 27.3M 7s 2024-03-29T04:19:17.044 INFO:teuthology.orchestra.run.smithi022.stderr: 38800K .......... .......... .......... .......... .......... 23% 31.4M 7s 2024-03-29T04:19:17.045 INFO:teuthology.orchestra.run.smithi022.stderr: 38850K .......... .......... .......... .......... .......... 23% 28.1M 7s 2024-03-29T04:19:17.045 INFO:teuthology.orchestra.run.smithi022.stderr: 38900K .......... .......... .......... .......... .......... 23% 16.5M 7s 2024-03-29T04:19:17.045 INFO:teuthology.orchestra.run.smithi022.stderr: 38950K .......... .......... .......... .......... .......... 23% 13.6M 7s 2024-03-29T04:19:17.045 INFO:teuthology.orchestra.run.smithi022.stderr: 39000K .......... .......... .......... .......... .......... 23% 23.8M 7s 2024-03-29T04:19:17.045 INFO:teuthology.orchestra.run.smithi022.stderr: 39050K .......... .......... .......... .......... .......... 23% 22.5M 7s 2024-03-29T04:19:17.045 INFO:teuthology.orchestra.run.smithi022.stderr: 39100K .......... .......... .......... .......... .......... 23% 23.6M 7s 2024-03-29T04:19:17.045 INFO:teuthology.orchestra.run.smithi022.stderr: 39150K .......... .......... .......... .......... .......... 23% 19.9M 7s 2024-03-29T04:19:17.045 INFO:teuthology.orchestra.run.smithi022.stderr: 39200K .......... .......... .......... .......... .......... 23% 24.3M 7s 2024-03-29T04:19:17.048 INFO:teuthology.orchestra.run.smithi022.stderr: 39250K .......... .......... .......... .......... .......... 23% 22.3M 7s 2024-03-29T04:19:17.048 INFO:teuthology.orchestra.run.smithi022.stderr: 39300K .......... .......... .......... .......... .......... 23% 23.3M 7s 2024-03-29T04:19:17.050 INFO:teuthology.orchestra.run.smithi022.stderr: 39350K .......... .......... .......... .......... .......... 23% 20.6M 7s 2024-03-29T04:19:17.053 INFO:teuthology.orchestra.run.smithi022.stderr: 39400K .......... .......... .......... .......... .......... 23% 22.9M 7s 2024-03-29T04:19:17.055 INFO:teuthology.orchestra.run.smithi022.stderr: 39450K .......... .......... .......... .......... .......... 23% 23.4M 7s 2024-03-29T04:19:17.058 INFO:teuthology.orchestra.run.smithi022.stderr: 39500K .......... .......... .......... .......... .......... 23% 22.9M 7s 2024-03-29T04:19:17.061 INFO:teuthology.orchestra.run.smithi022.stderr: 39550K .......... .......... .......... .......... .......... 23% 20.1M 7s 2024-03-29T04:19:17.064 INFO:teuthology.orchestra.run.smithi022.stderr: 39600K .......... .......... .......... .......... .......... 23% 21.9M 7s 2024-03-29T04:19:17.070 INFO:teuthology.orchestra.run.smithi022.stderr: 39650K .......... .......... .......... .......... .......... 23% 20.3M 7s 2024-03-29T04:19:17.070 INFO:teuthology.orchestra.run.smithi022.stderr: 39700K .......... .......... .......... .......... .......... 23% 20.1M 7s 2024-03-29T04:19:17.070 INFO:teuthology.orchestra.run.smithi022.stderr: 39750K .......... .......... .......... .......... .......... 23% 17.6M 7s 2024-03-29T04:19:17.076 INFO:teuthology.orchestra.run.smithi022.stderr: 39800K .......... .......... .......... .......... .......... 23% 19.6M 7s 2024-03-29T04:19:17.076 INFO:teuthology.orchestra.run.smithi022.stderr: 39850K .......... .......... .......... .......... .......... 23% 19.8M 7s 2024-03-29T04:19:17.077 INFO:teuthology.orchestra.run.smithi022.stderr: 39900K .......... .......... .......... .......... .......... 23% 21.0M 7s 2024-03-29T04:19:17.087 INFO:teuthology.orchestra.run.smithi022.stderr: 39950K .......... .......... .......... .......... .......... 23% 14.4M 7s 2024-03-29T04:19:17.087 INFO:teuthology.orchestra.run.smithi022.stderr: 40000K .......... .......... .......... .......... .......... 23% 16.2M 7s 2024-03-29T04:19:17.087 INFO:teuthology.orchestra.run.smithi022.stderr: 40050K .......... .......... .......... .......... .......... 23% 16.1M 7s 2024-03-29T04:19:17.090 INFO:teuthology.orchestra.run.smithi022.stderr: 40100K .......... .......... .......... .......... .......... 23% 15.7M 7s 2024-03-29T04:19:17.093 INFO:teuthology.orchestra.run.smithi022.stderr: 40150K .......... .......... .......... .......... .......... 23% 11.5M 7s 2024-03-29T04:19:17.097 INFO:teuthology.orchestra.run.smithi022.stderr: 40200K .......... .......... .......... .......... .......... 23% 14.8M 7s 2024-03-29T04:19:17.098 INFO:teuthology.orchestra.run.smithi022.stderr: 40250K .......... .......... .......... .......... .......... 23% 14.7M 7s 2024-03-29T04:19:17.101 INFO:teuthology.orchestra.run.smithi022.stderr: 40300K .......... .......... .......... .......... .......... 23% 15.9M 7s 2024-03-29T04:19:17.105 INFO:teuthology.orchestra.run.smithi022.stderr: 40350K .......... .......... .......... .......... .......... 23% 12.7M 7s 2024-03-29T04:19:17.109 INFO:teuthology.orchestra.run.smithi022.stderr: 40400K .......... .......... .......... .......... .......... 23% 14.6M 7s 2024-03-29T04:19:17.112 INFO:teuthology.orchestra.run.smithi022.stderr: 40450K .......... .......... .......... .......... .......... 24% 14.7M 7s 2024-03-29T04:19:17.117 INFO:teuthology.orchestra.run.smithi022.stderr: 40500K .......... .......... .......... .......... .......... 24% 11.8M 7s 2024-03-29T04:19:17.121 INFO:teuthology.orchestra.run.smithi022.stderr: 40550K .......... .......... .......... .......... .......... 24% 10.0M 7s 2024-03-29T04:19:17.125 INFO:teuthology.orchestra.run.smithi022.stderr: 40600K .......... .......... .......... .......... .......... 24% 12.6M 7s 2024-03-29T04:19:17.129 INFO:teuthology.orchestra.run.smithi022.stderr: 40650K .......... .......... .......... .......... .......... 24% 13.4M 7s 2024-03-29T04:19:17.133 INFO:teuthology.orchestra.run.smithi022.stderr: 40700K .......... .......... .......... .......... .......... 24% 12.8M 7s 2024-03-29T04:19:17.137 INFO:teuthology.orchestra.run.smithi022.stderr: 40750K .......... .......... .......... .......... .......... 24% 11.4M 7s 2024-03-29T04:19:17.141 INFO:teuthology.orchestra.run.smithi022.stderr: 40800K .......... .......... .......... .......... .......... 24% 12.6M 7s 2024-03-29T04:19:17.144 INFO:teuthology.orchestra.run.smithi022.stderr: 40850K .......... .......... .......... .......... .......... 24% 13.0M 7s 2024-03-29T04:19:17.148 INFO:teuthology.orchestra.run.smithi022.stderr: 40900K .......... .......... .......... .......... .......... 24% 13.0M 7s 2024-03-29T04:19:17.152 INFO:teuthology.orchestra.run.smithi022.stderr: 40950K .......... .......... .......... .......... .......... 24% 13.2M 7s 2024-03-29T04:19:17.154 INFO:teuthology.orchestra.run.smithi022.stderr: 41000K .......... .......... .......... .......... .......... 24% 28.9M 7s 2024-03-29T04:19:17.155 INFO:teuthology.orchestra.run.smithi022.stderr: 41050K .......... .......... .......... .......... .......... 24% 32.4M 7s 2024-03-29T04:19:17.156 INFO:teuthology.orchestra.run.smithi022.stderr: 41100K .......... .......... .......... .......... .......... 24% 32.7M 7s 2024-03-29T04:19:17.158 INFO:teuthology.orchestra.run.smithi022.stderr: 41150K .......... .......... .......... .......... .......... 24% 27.4M 7s 2024-03-29T04:19:17.159 INFO:teuthology.orchestra.run.smithi022.stderr: 41200K .......... .......... .......... .......... .......... 24% 30.6M 7s 2024-03-29T04:19:17.161 INFO:teuthology.orchestra.run.smithi022.stderr: 41250K .......... .......... .......... .......... .......... 24% 31.6M 7s 2024-03-29T04:19:17.163 INFO:teuthology.orchestra.run.smithi022.stderr: 41300K .......... .......... .......... .......... .......... 24% 30.8M 7s 2024-03-29T04:19:17.166 INFO:teuthology.orchestra.run.smithi022.stderr: 41350K .......... .......... .......... .......... .......... 24% 27.5M 7s 2024-03-29T04:19:17.166 INFO:teuthology.orchestra.run.smithi022.stderr: 41400K .......... .......... .......... .......... .......... 24% 31.5M 7s 2024-03-29T04:19:17.167 INFO:teuthology.orchestra.run.smithi022.stderr: 41450K .......... .......... .......... .......... .......... 24% 30.9M 7s 2024-03-29T04:19:17.170 INFO:teuthology.orchestra.run.smithi022.stderr: 41500K .......... .......... .......... .......... .......... 24% 26.0M 7s 2024-03-29T04:19:17.172 INFO:teuthology.orchestra.run.smithi022.stderr: 41550K .......... .......... .......... .......... .......... 24% 21.1M 7s 2024-03-29T04:19:17.174 INFO:teuthology.orchestra.run.smithi022.stderr: 41600K .......... .......... .......... .......... .......... 24% 25.3M 7s 2024-03-29T04:19:17.176 INFO:teuthology.orchestra.run.smithi022.stderr: 41650K .......... .......... .......... .......... .......... 24% 24.9M 7s 2024-03-29T04:19:17.178 INFO:teuthology.orchestra.run.smithi022.stderr: 41700K .......... .......... .......... .......... .......... 24% 25.9M 7s 2024-03-29T04:19:17.180 INFO:teuthology.orchestra.run.smithi022.stderr: 41750K .......... .......... .......... .......... .......... 24% 19.7M 7s 2024-03-29T04:19:17.183 INFO:teuthology.orchestra.run.smithi022.stderr: 41800K .......... .......... .......... .......... .......... 24% 22.7M 7s 2024-03-29T04:19:17.184 INFO:teuthology.orchestra.run.smithi022.stderr: 41850K .......... .......... .......... .......... .......... 24% 30.6M 7s 2024-03-29T04:19:17.185 INFO:teuthology.orchestra.run.smithi022.stderr: 41900K .......... .......... .......... .......... .......... 24% 31.2M 7s 2024-03-29T04:19:17.187 INFO:teuthology.orchestra.run.smithi022.stderr: 41950K .......... .......... .......... .......... .......... 24% 27.0M 7s 2024-03-29T04:19:17.189 INFO:teuthology.orchestra.run.smithi022.stderr: 42000K .......... .......... .......... .......... .......... 24% 31.9M 7s 2024-03-29T04:19:17.190 INFO:teuthology.orchestra.run.smithi022.stderr: 42050K .......... .......... .......... .......... .......... 24% 31.4M 7s 2024-03-29T04:19:17.192 INFO:teuthology.orchestra.run.smithi022.stderr: 42100K .......... .......... .......... .......... .......... 25% 29.7M 7s 2024-03-29T04:19:17.194 INFO:teuthology.orchestra.run.smithi022.stderr: 42150K .......... .......... .......... .......... .......... 25% 26.3M 7s 2024-03-29T04:19:17.196 INFO:teuthology.orchestra.run.smithi022.stderr: 42200K .......... .......... .......... .......... .......... 25% 29.6M 7s 2024-03-29T04:19:17.197 INFO:teuthology.orchestra.run.smithi022.stderr: 42250K .......... .......... .......... .......... .......... 25% 28.6M 7s 2024-03-29T04:19:17.199 INFO:teuthology.orchestra.run.smithi022.stderr: 42300K .......... .......... .......... .......... .......... 25% 27.9M 7s 2024-03-29T04:19:17.201 INFO:teuthology.orchestra.run.smithi022.stderr: 42350K .......... .......... .......... .......... .......... 25% 24.0M 7s 2024-03-29T04:19:17.203 INFO:teuthology.orchestra.run.smithi022.stderr: 42400K .......... .......... .......... .......... .......... 25% 27.4M 7s 2024-03-29T04:19:17.205 INFO:teuthology.orchestra.run.smithi022.stderr: 42450K .......... .......... .......... .......... .......... 25% 26.9M 7s 2024-03-29T04:19:17.206 INFO:teuthology.orchestra.run.smithi022.stderr: 42500K .......... .......... .......... .......... .......... 25% 26.7M 7s 2024-03-29T04:19:17.208 INFO:teuthology.orchestra.run.smithi022.stderr: 42550K .......... .......... .......... .......... .......... 25% 22.5M 7s 2024-03-29T04:19:17.210 INFO:teuthology.orchestra.run.smithi022.stderr: 42600K .......... .......... .......... .......... .......... 25% 27.0M 7s 2024-03-29T04:19:17.213 INFO:teuthology.orchestra.run.smithi022.stderr: 42650K .......... .......... .......... .......... .......... 25% 25.2M 7s 2024-03-29T04:19:17.214 INFO:teuthology.orchestra.run.smithi022.stderr: 42700K .......... .......... .......... .......... .......... 25% 26.7M 7s 2024-03-29T04:19:17.217 INFO:teuthology.orchestra.run.smithi022.stderr: 42750K .......... .......... .......... .......... .......... 25% 22.3M 7s 2024-03-29T04:19:17.218 INFO:teuthology.orchestra.run.smithi022.stderr: 42800K .......... .......... .......... .......... .......... 25% 26.7M 7s 2024-03-29T04:19:17.221 INFO:teuthology.orchestra.run.smithi022.stderr: 42850K .......... .......... .......... .......... .......... 25% 25.7M 7s 2024-03-29T04:19:17.222 INFO:teuthology.orchestra.run.smithi022.stderr: 42900K .......... .......... .......... .......... .......... 25% 26.3M 7s 2024-03-29T04:19:17.227 INFO:teuthology.orchestra.run.smithi022.stderr: 42950K .......... .......... .......... .......... .......... 25% 22.3M 7s 2024-03-29T04:19:17.228 INFO:teuthology.orchestra.run.smithi022.stderr: 43000K .......... .......... .......... .......... .......... 25% 26.3M 7s 2024-03-29T04:19:17.228 INFO:teuthology.orchestra.run.smithi022.stderr: 43050K .......... .......... .......... .......... .......... 25% 25.8M 7s 2024-03-29T04:19:17.230 INFO:teuthology.orchestra.run.smithi022.stderr: 43100K .......... .......... .......... .......... .......... 25% 25.7M 7s 2024-03-29T04:19:17.232 INFO:teuthology.orchestra.run.smithi022.stderr: 43150K .......... .......... .......... .......... .......... 25% 21.7M 7s 2024-03-29T04:19:17.234 INFO:teuthology.orchestra.run.smithi022.stderr: 43200K .......... .......... .......... .......... .......... 25% 25.3M 7s 2024-03-29T04:19:17.236 INFO:teuthology.orchestra.run.smithi022.stderr: 43250K .......... .......... .......... .......... .......... 25% 22.6M 7s 2024-03-29T04:19:17.239 INFO:teuthology.orchestra.run.smithi022.stderr: 43300K .......... .......... .......... .......... .......... 25% 18.7M 7s 2024-03-29T04:19:17.243 INFO:teuthology.orchestra.run.smithi022.stderr: 43350K .......... .......... .......... .......... .......... 25% 14.8M 7s 2024-03-29T04:19:17.245 INFO:teuthology.orchestra.run.smithi022.stderr: 43400K .......... .......... .......... .......... .......... 25% 18.6M 7s 2024-03-29T04:19:17.247 INFO:teuthology.orchestra.run.smithi022.stderr: 43450K .......... .......... .......... .......... .......... 25% 18.4M 7s 2024-03-29T04:19:17.250 INFO:teuthology.orchestra.run.smithi022.stderr: 43500K .......... .......... .......... .......... .......... 25% 17.3M 7s 2024-03-29T04:19:17.255 INFO:teuthology.orchestra.run.smithi022.stderr: 43550K .......... .......... .......... .......... .......... 25% 11.1M 7s 2024-03-29T04:19:17.259 INFO:teuthology.orchestra.run.smithi022.stderr: 43600K .......... .......... .......... .......... .......... 25% 11.6M 7s 2024-03-29T04:19:17.264 INFO:teuthology.orchestra.run.smithi022.stderr: 43650K .......... .......... .......... .......... .......... 25% 10.7M 7s 2024-03-29T04:19:17.268 INFO:teuthology.orchestra.run.smithi022.stderr: 43700K .......... .......... .......... .......... .......... 25% 11.9M 7s 2024-03-29T04:19:17.272 INFO:teuthology.orchestra.run.smithi022.stderr: 43750K .......... .......... .......... .......... .......... 25% 11.7M 7s 2024-03-29T04:19:17.276 INFO:teuthology.orchestra.run.smithi022.stderr: 43800K .......... .......... .......... .......... .......... 26% 12.4M 7s 2024-03-29T04:19:17.279 INFO:teuthology.orchestra.run.smithi022.stderr: 43850K .......... .......... .......... .......... .......... 26% 15.1M 7s 2024-03-29T04:19:17.283 INFO:teuthology.orchestra.run.smithi022.stderr: 43900K .......... .......... .......... .......... .......... 26% 12.9M 7s 2024-03-29T04:19:17.288 INFO:teuthology.orchestra.run.smithi022.stderr: 43950K .......... .......... .......... .......... .......... 26% 10.6M 7s 2024-03-29T04:19:17.292 INFO:teuthology.orchestra.run.smithi022.stderr: 44000K .......... .......... .......... .......... .......... 26% 11.4M 7s 2024-03-29T04:19:17.296 INFO:teuthology.orchestra.run.smithi022.stderr: 44050K .......... .......... .......... .......... .......... 26% 12.4M 7s 2024-03-29T04:19:17.300 INFO:teuthology.orchestra.run.smithi022.stderr: 44100K .......... .......... .......... .......... .......... 26% 11.2M 7s 2024-03-29T04:19:17.305 INFO:teuthology.orchestra.run.smithi022.stderr: 44150K .......... .......... .......... .......... .......... 26% 11.3M 7s 2024-03-29T04:19:17.308 INFO:teuthology.orchestra.run.smithi022.stderr: 44200K .......... .......... .......... .......... .......... 26% 12.4M 7s 2024-03-29T04:19:17.312 INFO:teuthology.orchestra.run.smithi022.stderr: 44250K .......... .......... .......... .......... .......... 26% 13.0M 7s 2024-03-29T04:19:17.316 INFO:teuthology.orchestra.run.smithi022.stderr: 44300K .......... .......... .......... .......... .......... 26% 15.9M 7s 2024-03-29T04:19:17.319 INFO:teuthology.orchestra.run.smithi022.stderr: 44350K .......... .......... .......... .......... .......... 26% 13.3M 7s 2024-03-29T04:19:17.323 INFO:teuthology.orchestra.run.smithi022.stderr: 44400K .......... .......... .......... .......... .......... 26% 11.9M 7s 2024-03-29T04:19:17.327 INFO:teuthology.orchestra.run.smithi022.stderr: 44450K .......... .......... .......... .......... .......... 26% 11.4M 7s 2024-03-29T04:19:17.332 INFO:teuthology.orchestra.run.smithi022.stderr: 44500K .......... .......... .......... .......... .......... 26% 12.8M 7s 2024-03-29T04:19:17.335 INFO:teuthology.orchestra.run.smithi022.stderr: 44550K .......... .......... .......... .......... .......... 26% 11.0M 7s 2024-03-29T04:19:17.340 INFO:teuthology.orchestra.run.smithi022.stderr: 44600K .......... .......... .......... .......... .......... 26% 13.0M 7s 2024-03-29T04:19:17.349 INFO:teuthology.orchestra.run.smithi022.stderr: 44650K .......... .......... .......... .......... .......... 26% 11.0M 7s 2024-03-29T04:19:17.350 INFO:teuthology.orchestra.run.smithi022.stderr: 44700K .......... .......... .......... .......... .......... 26% 10.3M 7s 2024-03-29T04:19:17.354 INFO:teuthology.orchestra.run.smithi022.stderr: 44750K .......... .......... .......... .......... .......... 26% 9.09M 7s 2024-03-29T04:19:17.359 INFO:teuthology.orchestra.run.smithi022.stderr: 44800K .......... .......... .......... .......... .......... 26% 10.1M 7s 2024-03-29T04:19:17.364 INFO:teuthology.orchestra.run.smithi022.stderr: 44850K .......... .......... .......... .......... .......... 26% 9.83M 7s 2024-03-29T04:19:17.369 INFO:teuthology.orchestra.run.smithi022.stderr: 44900K .......... .......... .......... .......... .......... 26% 9.63M 7s 2024-03-29T04:19:17.375 INFO:teuthology.orchestra.run.smithi022.stderr: 44950K .......... .......... .......... .......... .......... 26% 8.35M 7s 2024-03-29T04:19:17.380 INFO:teuthology.orchestra.run.smithi022.stderr: 45000K .......... .......... .......... .......... .......... 26% 9.88M 7s 2024-03-29T04:19:17.384 INFO:teuthology.orchestra.run.smithi022.stderr: 45050K .......... .......... .......... .......... .......... 26% 11.0M 7s 2024-03-29T04:19:17.387 INFO:teuthology.orchestra.run.smithi022.stderr: 45100K .......... .......... .......... .......... .......... 26% 15.1M 7s 2024-03-29T04:19:17.391 INFO:teuthology.orchestra.run.smithi022.stderr: 45150K .......... .......... .......... .......... .......... 26% 13.5M 7s 2024-03-29T04:19:17.394 INFO:teuthology.orchestra.run.smithi022.stderr: 45200K .......... .......... .......... .......... .......... 26% 18.1M 7s 2024-03-29T04:19:17.397 INFO:teuthology.orchestra.run.smithi022.stderr: 45250K .......... .......... .......... .......... .......... 26% 14.1M 7s 2024-03-29T04:19:17.401 INFO:teuthology.orchestra.run.smithi022.stderr: 45300K .......... .......... .......... .......... .......... 26% 14.8M 7s 2024-03-29T04:19:17.406 INFO:teuthology.orchestra.run.smithi022.stderr: 45350K .......... .......... .......... .......... .......... 26% 9.07M 7s 2024-03-29T04:19:17.409 INFO:teuthology.orchestra.run.smithi022.stderr: 45400K .......... .......... .......... .......... .......... 26% 13.9M 7s 2024-03-29T04:19:17.411 INFO:teuthology.orchestra.run.smithi022.stderr: 45450K .......... .......... .......... .......... .......... 26% 18.1M 7s 2024-03-29T04:19:17.414 INFO:teuthology.orchestra.run.smithi022.stderr: 45500K .......... .......... .......... .......... .......... 27% 19.4M 7s 2024-03-29T04:19:17.417 INFO:teuthology.orchestra.run.smithi022.stderr: 45550K .......... .......... .......... .......... .......... 27% 16.3M 7s 2024-03-29T04:19:17.420 INFO:teuthology.orchestra.run.smithi022.stderr: 45600K .......... .......... .......... .......... .......... 27% 18.6M 7s 2024-03-29T04:19:17.422 INFO:teuthology.orchestra.run.smithi022.stderr: 45650K .......... .......... .......... .......... .......... 27% 20.5M 7s 2024-03-29T04:19:17.426 INFO:teuthology.orchestra.run.smithi022.stderr: 45700K .......... .......... .......... .......... .......... 27% 18.8M 7s 2024-03-29T04:19:17.428 INFO:teuthology.orchestra.run.smithi022.stderr: 45750K .......... .......... .......... .......... .......... 27% 16.4M 7s 2024-03-29T04:19:17.430 INFO:teuthology.orchestra.run.smithi022.stderr: 45800K .......... .......... .......... .......... .......... 27% 18.8M 7s 2024-03-29T04:19:17.433 INFO:teuthology.orchestra.run.smithi022.stderr: 45850K .......... .......... .......... .......... .......... 27% 18.3M 7s 2024-03-29T04:19:17.435 INFO:teuthology.orchestra.run.smithi022.stderr: 45900K .......... .......... .......... .......... .......... 27% 18.1M 7s 2024-03-29T04:19:17.439 INFO:teuthology.orchestra.run.smithi022.stderr: 45950K .......... .......... .......... .......... .......... 27% 15.9M 7s 2024-03-29T04:19:17.441 INFO:teuthology.orchestra.run.smithi022.stderr: 46000K .......... .......... .......... .......... .......... 27% 19.2M 7s 2024-03-29T04:19:17.444 INFO:teuthology.orchestra.run.smithi022.stderr: 46050K .......... .......... .......... .......... .......... 27% 17.9M 7s 2024-03-29T04:19:17.446 INFO:teuthology.orchestra.run.smithi022.stderr: 46100K .......... .......... .......... .......... .......... 27% 22.5M 7s 2024-03-29T04:19:17.448 INFO:teuthology.orchestra.run.smithi022.stderr: 46150K .......... .......... .......... .......... .......... 27% 19.0M 7s 2024-03-29T04:19:17.451 INFO:teuthology.orchestra.run.smithi022.stderr: 46200K .......... .......... .......... .......... .......... 27% 18.9M 7s 2024-03-29T04:19:17.454 INFO:teuthology.orchestra.run.smithi022.stderr: 46250K .......... .......... .......... .......... .......... 27% 19.0M 7s 2024-03-29T04:19:17.457 INFO:teuthology.orchestra.run.smithi022.stderr: 46300K .......... .......... .......... .......... .......... 27% 18.2M 7s 2024-03-29T04:19:17.460 INFO:teuthology.orchestra.run.smithi022.stderr: 46350K .......... .......... .......... .......... .......... 27% 15.5M 7s 2024-03-29T04:19:17.463 INFO:teuthology.orchestra.run.smithi022.stderr: 46400K .......... .......... .......... .......... .......... 27% 17.3M 7s 2024-03-29T04:19:17.466 INFO:teuthology.orchestra.run.smithi022.stderr: 46450K .......... .......... .......... .......... .......... 27% 14.3M 7s 2024-03-29T04:19:17.470 INFO:teuthology.orchestra.run.smithi022.stderr: 46500K .......... .......... .......... .......... .......... 27% 14.8M 7s 2024-03-29T04:19:17.473 INFO:teuthology.orchestra.run.smithi022.stderr: 46550K .......... .......... .......... .......... .......... 27% 13.1M 7s 2024-03-29T04:19:17.477 INFO:teuthology.orchestra.run.smithi022.stderr: 46600K .......... .......... .......... .......... .......... 27% 13.6M 7s 2024-03-29T04:19:17.481 INFO:teuthology.orchestra.run.smithi022.stderr: 46650K .......... .......... .......... .......... .......... 27% 12.7M 7s 2024-03-29T04:19:17.484 INFO:teuthology.orchestra.run.smithi022.stderr: 46700K .......... .......... .......... .......... .......... 27% 13.0M 7s 2024-03-29T04:19:17.489 INFO:teuthology.orchestra.run.smithi022.stderr: 46750K .......... .......... .......... .......... .......... 27% 11.1M 7s 2024-03-29T04:19:17.494 INFO:teuthology.orchestra.run.smithi022.stderr: 46800K .......... .......... .......... .......... .......... 27% 12.4M 7s 2024-03-29T04:19:17.497 INFO:teuthology.orchestra.run.smithi022.stderr: 46850K .......... .......... .......... .......... .......... 27% 12.3M 7s 2024-03-29T04:19:17.501 INFO:teuthology.orchestra.run.smithi022.stderr: 46900K .......... .......... .......... .......... .......... 27% 11.4M 7s 2024-03-29T04:19:17.507 INFO:teuthology.orchestra.run.smithi022.stderr: 46950K .......... .......... .......... .......... .......... 27% 9.16M 7s 2024-03-29T04:19:17.511 INFO:teuthology.orchestra.run.smithi022.stderr: 47000K .......... .......... .......... .......... .......... 27% 10.2M 7s 2024-03-29T04:19:17.515 INFO:teuthology.orchestra.run.smithi022.stderr: 47050K .......... .......... .......... .......... .......... 27% 12.0M 7s 2024-03-29T04:19:17.519 INFO:teuthology.orchestra.run.smithi022.stderr: 47100K .......... .......... .......... .......... .......... 27% 12.8M 7s 2024-03-29T04:19:17.524 INFO:teuthology.orchestra.run.smithi022.stderr: 47150K .......... .......... .......... .......... .......... 28% 10.5M 7s 2024-03-29T04:19:17.528 INFO:teuthology.orchestra.run.smithi022.stderr: 47200K .......... .......... .......... .......... .......... 28% 11.3M 7s 2024-03-29T04:19:17.532 INFO:teuthology.orchestra.run.smithi022.stderr: 47250K .......... .......... .......... .......... .......... 28% 11.6M 7s 2024-03-29T04:19:17.536 INFO:teuthology.orchestra.run.smithi022.stderr: 47300K .......... .......... .......... .......... .......... 28% 11.3M 7s 2024-03-29T04:19:17.542 INFO:teuthology.orchestra.run.smithi022.stderr: 47350K .......... .......... .......... .......... .......... 28% 9.71M 7s 2024-03-29T04:19:17.546 INFO:teuthology.orchestra.run.smithi022.stderr: 47400K .......... .......... .......... .......... .......... 28% 10.9M 7s 2024-03-29T04:19:17.550 INFO:teuthology.orchestra.run.smithi022.stderr: 47450K .......... .......... .......... .......... .......... 28% 11.2M 7s 2024-03-29T04:19:17.555 INFO:teuthology.orchestra.run.smithi022.stderr: 47500K .......... .......... .......... .......... .......... 28% 10.8M 7s 2024-03-29T04:19:17.560 INFO:teuthology.orchestra.run.smithi022.stderr: 47550K .......... .......... .......... .......... .......... 28% 9.86M 7s 2024-03-29T04:19:17.564 INFO:teuthology.orchestra.run.smithi022.stderr: 47600K .......... .......... .......... .......... .......... 28% 11.3M 7s 2024-03-29T04:19:17.568 INFO:teuthology.orchestra.run.smithi022.stderr: 47650K .......... .......... .......... .......... .......... 28% 11.3M 7s 2024-03-29T04:19:17.573 INFO:teuthology.orchestra.run.smithi022.stderr: 47700K .......... .......... .......... .......... .......... 28% 11.7M 7s 2024-03-29T04:19:17.578 INFO:teuthology.orchestra.run.smithi022.stderr: 47750K .......... .......... .......... .......... .......... 28% 9.77M 7s 2024-03-29T04:19:17.582 INFO:teuthology.orchestra.run.smithi022.stderr: 47800K .......... .......... .......... .......... .......... 28% 11.8M 7s 2024-03-29T04:19:17.586 INFO:teuthology.orchestra.run.smithi022.stderr: 47850K .......... .......... .......... .......... .......... 28% 11.1M 7s 2024-03-29T04:19:17.590 INFO:teuthology.orchestra.run.smithi022.stderr: 47900K .......... .......... .......... .......... .......... 28% 11.3M 7s 2024-03-29T04:19:17.595 INFO:teuthology.orchestra.run.smithi022.stderr: 47950K .......... .......... .......... .......... .......... 28% 9.93M 7s 2024-03-29T04:19:17.600 INFO:teuthology.orchestra.run.smithi022.stderr: 48000K .......... .......... .......... .......... .......... 28% 11.3M 7s 2024-03-29T04:19:17.604 INFO:teuthology.orchestra.run.smithi022.stderr: 48050K .......... .......... .......... .......... .......... 28% 11.8M 7s 2024-03-29T04:19:17.608 INFO:teuthology.orchestra.run.smithi022.stderr: 48100K .......... .......... .......... .......... .......... 28% 11.0M 7s 2024-03-29T04:19:17.613 INFO:teuthology.orchestra.run.smithi022.stderr: 48150K .......... .......... .......... .......... .......... 28% 9.70M 7s 2024-03-29T04:19:17.617 INFO:teuthology.orchestra.run.smithi022.stderr: 48200K .......... .......... .......... .......... .......... 28% 11.0M 7s 2024-03-29T04:19:17.621 INFO:teuthology.orchestra.run.smithi022.stderr: 48250K .......... .......... .......... .......... .......... 28% 13.2M 7s 2024-03-29T04:19:17.625 INFO:teuthology.orchestra.run.smithi022.stderr: 48300K .......... .......... .......... .......... .......... 28% 14.0M 7s 2024-03-29T04:19:17.629 INFO:teuthology.orchestra.run.smithi022.stderr: 48350K .......... .......... .......... .......... .......... 28% 11.3M 7s 2024-03-29T04:19:17.635 INFO:teuthology.orchestra.run.smithi022.stderr: 48400K .......... .......... .......... .......... .......... 28% 9.49M 7s 2024-03-29T04:19:17.640 INFO:teuthology.orchestra.run.smithi022.stderr: 48450K .......... .......... .......... .......... .......... 28% 9.63M 7s 2024-03-29T04:19:17.645 INFO:teuthology.orchestra.run.smithi022.stderr: 48500K .......... .......... .......... .......... .......... 28% 9.61M 7s 2024-03-29T04:19:17.651 INFO:teuthology.orchestra.run.smithi022.stderr: 48550K .......... .......... .......... .......... .......... 28% 8.12M 7s 2024-03-29T04:19:17.656 INFO:teuthology.orchestra.run.smithi022.stderr: 48600K .......... .......... .......... .......... .......... 28% 9.63M 7s 2024-03-29T04:19:17.661 INFO:teuthology.orchestra.run.smithi022.stderr: 48650K .......... .......... .......... .......... .......... 28% 9.49M 7s 2024-03-29T04:19:17.666 INFO:teuthology.orchestra.run.smithi022.stderr: 48700K .......... .......... .......... .......... .......... 28% 9.08M 7s 2024-03-29T04:19:17.672 INFO:teuthology.orchestra.run.smithi022.stderr: 48750K .......... .......... .......... .......... .......... 28% 7.94M 7s 2024-03-29T04:19:17.678 INFO:teuthology.orchestra.run.smithi022.stderr: 48800K .......... .......... .......... .......... .......... 28% 8.96M 7s 2024-03-29T04:19:17.683 INFO:teuthology.orchestra.run.smithi022.stderr: 48850K .......... .......... .......... .......... .......... 29% 8.79M 7s 2024-03-29T04:19:17.689 INFO:teuthology.orchestra.run.smithi022.stderr: 48900K .......... .......... .......... .......... .......... 29% 8.84M 7s 2024-03-29T04:19:17.696 INFO:teuthology.orchestra.run.smithi022.stderr: 48950K .......... .......... .......... .......... .......... 29% 7.71M 7s 2024-03-29T04:19:17.701 INFO:teuthology.orchestra.run.smithi022.stderr: 49000K .......... .......... .......... .......... .......... 29% 9.08M 7s 2024-03-29T04:19:17.707 INFO:teuthology.orchestra.run.smithi022.stderr: 49050K .......... .......... .......... .......... .......... 29% 8.34M 7s 2024-03-29T04:19:17.712 INFO:teuthology.orchestra.run.smithi022.stderr: 49100K .......... .......... .......... .......... .......... 29% 8.56M 7s 2024-03-29T04:19:17.717 INFO:teuthology.orchestra.run.smithi022.stderr: 49150K .......... .......... .......... .......... .......... 29% 10.1M 7s 2024-03-29T04:19:17.720 INFO:teuthology.orchestra.run.smithi022.stderr: 49200K .......... .......... .......... .......... .......... 29% 12.7M 7s 2024-03-29T04:19:17.723 INFO:teuthology.orchestra.run.smithi022.stderr: 49250K .......... .......... .......... .......... .......... 29% 20.1M 7s 2024-03-29T04:19:17.726 INFO:teuthology.orchestra.run.smithi022.stderr: 49300K .......... .......... .......... .......... .......... 29% 19.0M 7s 2024-03-29T04:19:17.729 INFO:teuthology.orchestra.run.smithi022.stderr: 49350K .......... .......... .......... .......... .......... 29% 14.0M 7s 2024-03-29T04:19:17.733 INFO:teuthology.orchestra.run.smithi022.stderr: 49400K .......... .......... .......... .......... .......... 29% 13.5M 7s 2024-03-29T04:19:17.736 INFO:teuthology.orchestra.run.smithi022.stderr: 49450K .......... .......... .......... .......... .......... 29% 13.3M 7s 2024-03-29T04:19:17.740 INFO:teuthology.orchestra.run.smithi022.stderr: 49500K .......... .......... .......... .......... .......... 29% 14.0M 7s 2024-03-29T04:19:17.744 INFO:teuthology.orchestra.run.smithi022.stderr: 49550K .......... .......... .......... .......... .......... 29% 12.1M 7s 2024-03-29T04:19:17.747 INFO:teuthology.orchestra.run.smithi022.stderr: 49600K .......... .......... .......... .......... .......... 29% 18.2M 7s 2024-03-29T04:19:17.749 INFO:teuthology.orchestra.run.smithi022.stderr: 49650K .......... .......... .......... .......... .......... 29% 16.3M 7s 2024-03-29T04:19:17.752 INFO:teuthology.orchestra.run.smithi022.stderr: 49700K .......... .......... .......... .......... .......... 29% 16.5M 7s 2024-03-29T04:19:17.755 INFO:teuthology.orchestra.run.smithi022.stderr: 49750K .......... .......... .......... .......... .......... 29% 19.9M 7s 2024-03-29T04:19:17.757 INFO:teuthology.orchestra.run.smithi022.stderr: 49800K .......... .......... .......... .......... .......... 29% 22.9M 7s 2024-03-29T04:19:17.759 INFO:teuthology.orchestra.run.smithi022.stderr: 49850K .......... .......... .......... .......... .......... 29% 23.5M 7s 2024-03-29T04:19:17.761 INFO:teuthology.orchestra.run.smithi022.stderr: 49900K .......... .......... .......... .......... .......... 29% 23.8M 7s 2024-03-29T04:19:17.763 INFO:teuthology.orchestra.run.smithi022.stderr: 49950K .......... .......... .......... .......... .......... 29% 20.0M 7s 2024-03-29T04:19:17.766 INFO:teuthology.orchestra.run.smithi022.stderr: 50000K .......... .......... .......... .......... .......... 29% 23.2M 7s 2024-03-29T04:19:17.768 INFO:teuthology.orchestra.run.smithi022.stderr: 50050K .......... .......... .......... .......... .......... 29% 23.6M 7s 2024-03-29T04:19:17.770 INFO:teuthology.orchestra.run.smithi022.stderr: 50100K .......... .......... .......... .......... .......... 29% 23.0M 7s 2024-03-29T04:19:17.772 INFO:teuthology.orchestra.run.smithi022.stderr: 50150K .......... .......... .......... .......... .......... 29% 19.8M 7s 2024-03-29T04:19:17.774 INFO:teuthology.orchestra.run.smithi022.stderr: 50200K .......... .......... .......... .......... .......... 29% 23.2M 7s 2024-03-29T04:19:17.777 INFO:teuthology.orchestra.run.smithi022.stderr: 50250K .......... .......... .......... .......... .......... 29% 21.2M 7s 2024-03-29T04:19:17.779 INFO:teuthology.orchestra.run.smithi022.stderr: 50300K .......... .......... .......... .......... .......... 29% 22.0M 7s 2024-03-29T04:19:17.781 INFO:teuthology.orchestra.run.smithi022.stderr: 50350K .......... .......... .......... .......... .......... 29% 19.7M 7s 2024-03-29T04:19:17.783 INFO:teuthology.orchestra.run.smithi022.stderr: 50400K .......... .......... .......... .......... .......... 29% 23.6M 7s 2024-03-29T04:19:17.785 INFO:teuthology.orchestra.run.smithi022.stderr: 50450K .......... .......... .......... .......... .......... 29% 23.9M 7s 2024-03-29T04:19:17.788 INFO:teuthology.orchestra.run.smithi022.stderr: 50500K .......... .......... .......... .......... .......... 29% 21.1M 7s 2024-03-29T04:19:17.792 INFO:teuthology.orchestra.run.smithi022.stderr: 50550K .......... .......... .......... .......... .......... 30% 11.6M 7s 2024-03-29T04:19:17.796 INFO:teuthology.orchestra.run.smithi022.stderr: 50600K .......... .......... .......... .......... .......... 30% 11.9M 7s 2024-03-29T04:19:17.798 INFO:teuthology.orchestra.run.smithi022.stderr: 50650K .......... .......... .......... .......... .......... 30% 20.4M 7s 2024-03-29T04:19:17.800 INFO:teuthology.orchestra.run.smithi022.stderr: 50700K .......... .......... .......... .......... .......... 30% 26.0M 7s 2024-03-29T04:19:17.803 INFO:teuthology.orchestra.run.smithi022.stderr: 50750K .......... .......... .......... .......... .......... 30% 21.8M 7s 2024-03-29T04:19:17.804 INFO:teuthology.orchestra.run.smithi022.stderr: 50800K .......... .......... .......... .......... .......... 30% 24.5M 7s 2024-03-29T04:19:17.806 INFO:teuthology.orchestra.run.smithi022.stderr: 50850K .......... .......... .......... .......... .......... 30% 26.3M 7s 2024-03-29T04:19:17.808 INFO:teuthology.orchestra.run.smithi022.stderr: 50900K .......... .......... .......... .......... .......... 30% 23.8M 7s 2024-03-29T04:19:17.811 INFO:teuthology.orchestra.run.smithi022.stderr: 50950K .......... .......... .......... .......... .......... 30% 20.0M 7s 2024-03-29T04:19:17.813 INFO:teuthology.orchestra.run.smithi022.stderr: 51000K .......... .......... .......... .......... .......... 30% 23.5M 7s 2024-03-29T04:19:17.815 INFO:teuthology.orchestra.run.smithi022.stderr: 51050K .......... .......... .......... .......... .......... 30% 23.8M 7s 2024-03-29T04:19:17.817 INFO:teuthology.orchestra.run.smithi022.stderr: 51100K .......... .......... .......... .......... .......... 30% 22.9M 7s 2024-03-29T04:19:17.820 INFO:teuthology.orchestra.run.smithi022.stderr: 51150K .......... .......... .......... .......... .......... 30% 20.3M 7s 2024-03-29T04:19:17.822 INFO:teuthology.orchestra.run.smithi022.stderr: 51200K .......... .......... .......... .......... .......... 30% 22.7M 7s 2024-03-29T04:19:17.824 INFO:teuthology.orchestra.run.smithi022.stderr: 51250K .......... .......... .......... .......... .......... 30% 22.3M 7s 2024-03-29T04:19:17.826 INFO:teuthology.orchestra.run.smithi022.stderr: 51300K .......... .......... .......... .......... .......... 30% 23.4M 7s 2024-03-29T04:19:17.828 INFO:teuthology.orchestra.run.smithi022.stderr: 51350K .......... .......... .......... .......... .......... 30% 19.3M 7s 2024-03-29T04:19:17.831 INFO:teuthology.orchestra.run.smithi022.stderr: 51400K .......... .......... .......... .......... .......... 30% 23.4M 7s 2024-03-29T04:19:17.833 INFO:teuthology.orchestra.run.smithi022.stderr: 51450K .......... .......... .......... .......... .......... 30% 23.0M 7s 2024-03-29T04:19:17.835 INFO:teuthology.orchestra.run.smithi022.stderr: 51500K .......... .......... .......... .......... .......... 30% 23.1M 7s 2024-03-29T04:19:17.837 INFO:teuthology.orchestra.run.smithi022.stderr: 51550K .......... .......... .......... .......... .......... 30% 19.6M 7s 2024-03-29T04:19:17.840 INFO:teuthology.orchestra.run.smithi022.stderr: 51600K .......... .......... .......... .......... .......... 30% 22.5M 7s 2024-03-29T04:19:17.842 INFO:teuthology.orchestra.run.smithi022.stderr: 51650K .......... .......... .......... .......... .......... 30% 22.8M 7s 2024-03-29T04:19:17.844 INFO:teuthology.orchestra.run.smithi022.stderr: 51700K .......... .......... .......... .......... .......... 30% 22.4M 7s 2024-03-29T04:19:17.846 INFO:teuthology.orchestra.run.smithi022.stderr: 51750K .......... .......... .......... .......... .......... 30% 19.9M 7s 2024-03-29T04:19:17.848 INFO:teuthology.orchestra.run.smithi022.stderr: 51800K .......... .......... .......... .......... .......... 30% 22.3M 7s 2024-03-29T04:19:17.851 INFO:teuthology.orchestra.run.smithi022.stderr: 51850K .......... .......... .......... .......... .......... 30% 22.0M 7s 2024-03-29T04:19:17.853 INFO:teuthology.orchestra.run.smithi022.stderr: 51900K .......... .......... .......... .......... .......... 30% 22.5M 7s 2024-03-29T04:19:17.855 INFO:teuthology.orchestra.run.smithi022.stderr: 51950K .......... .......... .......... .......... .......... 30% 19.0M 7s 2024-03-29T04:19:17.858 INFO:teuthology.orchestra.run.smithi022.stderr: 52000K .......... .......... .......... .......... .......... 30% 22.4M 7s 2024-03-29T04:19:17.860 INFO:teuthology.orchestra.run.smithi022.stderr: 52050K .......... .......... .......... .......... .......... 30% 21.9M 7s 2024-03-29T04:19:17.862 INFO:teuthology.orchestra.run.smithi022.stderr: 52100K .......... .......... .......... .......... .......... 30% 22.1M 7s 2024-03-29T04:19:17.865 INFO:teuthology.orchestra.run.smithi022.stderr: 52150K .......... .......... .......... .......... .......... 30% 18.7M 7s 2024-03-29T04:19:17.867 INFO:teuthology.orchestra.run.smithi022.stderr: 52200K .......... .......... .......... .......... .......... 30% 21.8M 7s 2024-03-29T04:19:17.869 INFO:teuthology.orchestra.run.smithi022.stderr: 52250K .......... .......... .......... .......... .......... 31% 23.1M 7s 2024-03-29T04:19:17.871 INFO:teuthology.orchestra.run.smithi022.stderr: 52300K .......... .......... .......... .......... .......... 31% 24.5M 7s 2024-03-29T04:19:17.874 INFO:teuthology.orchestra.run.smithi022.stderr: 52350K .......... .......... .......... .......... .......... 31% 18.9M 7s 2024-03-29T04:19:17.876 INFO:teuthology.orchestra.run.smithi022.stderr: 52400K .......... .......... .......... .......... .......... 31% 23.0M 7s 2024-03-29T04:19:17.878 INFO:teuthology.orchestra.run.smithi022.stderr: 52450K .......... .......... .......... .......... .......... 31% 23.8M 7s 2024-03-29T04:19:17.880 INFO:teuthology.orchestra.run.smithi022.stderr: 52500K .......... .......... .......... .......... .......... 31% 23.9M 7s 2024-03-29T04:19:17.882 INFO:teuthology.orchestra.run.smithi022.stderr: 52550K .......... .......... .......... .......... .......... 31% 20.9M 7s 2024-03-29T04:19:17.884 INFO:teuthology.orchestra.run.smithi022.stderr: 52600K .......... .......... .......... .......... .......... 31% 23.4M 7s 2024-03-29T04:19:17.886 INFO:teuthology.orchestra.run.smithi022.stderr: 52650K .......... .......... .......... .......... .......... 31% 23.6M 7s 2024-03-29T04:19:17.888 INFO:teuthology.orchestra.run.smithi022.stderr: 52700K .......... .......... .......... .......... .......... 31% 23.7M 7s 2024-03-29T04:19:17.891 INFO:teuthology.orchestra.run.smithi022.stderr: 52750K .......... .......... .......... .......... .......... 31% 20.4M 7s 2024-03-29T04:19:17.893 INFO:teuthology.orchestra.run.smithi022.stderr: 52800K .......... .......... .......... .......... .......... 31% 23.3M 7s 2024-03-29T04:19:17.895 INFO:teuthology.orchestra.run.smithi022.stderr: 52850K .......... .......... .......... .......... .......... 31% 23.5M 7s 2024-03-29T04:19:17.897 INFO:teuthology.orchestra.run.smithi022.stderr: 52900K .......... .......... .......... .......... .......... 31% 21.1M 7s 2024-03-29T04:19:17.900 INFO:teuthology.orchestra.run.smithi022.stderr: 52950K .......... .......... .......... .......... .......... 31% 18.0M 7s 2024-03-29T04:19:17.902 INFO:teuthology.orchestra.run.smithi022.stderr: 53000K .......... .......... .......... .......... .......... 31% 21.1M 7s 2024-03-29T04:19:17.905 INFO:teuthology.orchestra.run.smithi022.stderr: 53050K .......... .......... .......... .......... .......... 31% 20.6M 7s 2024-03-29T04:19:17.907 INFO:teuthology.orchestra.run.smithi022.stderr: 53100K .......... .......... .......... .......... .......... 31% 19.1M 7s 2024-03-29T04:19:17.910 INFO:teuthology.orchestra.run.smithi022.stderr: 53150K .......... .......... .......... .......... .......... 31% 18.2M 7s 2024-03-29T04:19:17.913 INFO:teuthology.orchestra.run.smithi022.stderr: 53200K .......... .......... .......... .......... .......... 31% 16.7M 7s 2024-03-29T04:19:17.915 INFO:teuthology.orchestra.run.smithi022.stderr: 53250K .......... .......... .......... .......... .......... 31% 21.3M 7s 2024-03-29T04:19:17.918 INFO:teuthology.orchestra.run.smithi022.stderr: 53300K .......... .......... .......... .......... .......... 31% 19.1M 7s 2024-03-29T04:19:17.920 INFO:teuthology.orchestra.run.smithi022.stderr: 53350K .......... .......... .......... .......... .......... 31% 17.8M 7s 2024-03-29T04:19:17.923 INFO:teuthology.orchestra.run.smithi022.stderr: 53400K .......... .......... .......... .......... .......... 31% 21.4M 7s 2024-03-29T04:19:17.925 INFO:teuthology.orchestra.run.smithi022.stderr: 53450K .......... .......... .......... .......... .......... 31% 20.4M 7s 2024-03-29T04:19:17.927 INFO:teuthology.orchestra.run.smithi022.stderr: 53500K .......... .......... .......... .......... .......... 31% 20.9M 7s 2024-03-29T04:19:17.930 INFO:teuthology.orchestra.run.smithi022.stderr: 53550K .......... .......... .......... .......... .......... 31% 18.1M 7s 2024-03-29T04:19:17.933 INFO:teuthology.orchestra.run.smithi022.stderr: 53600K .......... .......... .......... .......... .......... 31% 20.1M 7s 2024-03-29T04:19:17.935 INFO:teuthology.orchestra.run.smithi022.stderr: 53650K .......... .......... .......... .......... .......... 31% 20.6M 7s 2024-03-29T04:19:17.937 INFO:teuthology.orchestra.run.smithi022.stderr: 53700K .......... .......... .......... .......... .......... 31% 19.9M 7s 2024-03-29T04:19:17.940 INFO:teuthology.orchestra.run.smithi022.stderr: 53750K .......... .......... .......... .......... .......... 31% 16.9M 7s 2024-03-29T04:19:17.943 INFO:teuthology.orchestra.run.smithi022.stderr: 53800K .......... .......... .......... .......... .......... 31% 20.0M 7s 2024-03-29T04:19:17.946 INFO:teuthology.orchestra.run.smithi022.stderr: 53850K .......... .......... .......... .......... .......... 31% 17.5M 7s 2024-03-29T04:19:17.948 INFO:teuthology.orchestra.run.smithi022.stderr: 53900K .......... .......... .......... .......... .......... 32% 19.3M 7s 2024-03-29T04:19:17.951 INFO:teuthology.orchestra.run.smithi022.stderr: 53950K .......... .......... .......... .......... .......... 32% 17.4M 7s 2024-03-29T04:19:17.953 INFO:teuthology.orchestra.run.smithi022.stderr: 54000K .......... .......... .......... .......... .......... 32% 23.1M 7s 2024-03-29T04:19:17.955 INFO:teuthology.orchestra.run.smithi022.stderr: 54050K .......... .......... .......... .......... .......... 32% 23.6M 7s 2024-03-29T04:19:17.957 INFO:teuthology.orchestra.run.smithi022.stderr: 54100K .......... .......... .......... .......... .......... 32% 23.6M 7s 2024-03-29T04:19:17.959 INFO:teuthology.orchestra.run.smithi022.stderr: 54150K .......... .......... .......... .......... .......... 32% 19.9M 7s 2024-03-29T04:19:17.961 INFO:teuthology.orchestra.run.smithi022.stderr: 54200K .......... .......... .......... .......... .......... 32% 25.2M 6s 2024-03-29T04:19:17.963 INFO:teuthology.orchestra.run.smithi022.stderr: 54250K .......... .......... .......... .......... .......... 32% 25.7M 6s 2024-03-29T04:19:17.966 INFO:teuthology.orchestra.run.smithi022.stderr: 54300K .......... .......... .......... .......... .......... 32% 23.1M 6s 2024-03-29T04:19:17.968 INFO:teuthology.orchestra.run.smithi022.stderr: 54350K .......... .......... .......... .......... .......... 32% 20.2M 6s 2024-03-29T04:19:17.970 INFO:teuthology.orchestra.run.smithi022.stderr: 54400K .......... .......... .......... .......... .......... 32% 22.5M 6s 2024-03-29T04:19:17.972 INFO:teuthology.orchestra.run.smithi022.stderr: 54450K .......... .......... .......... .......... .......... 32% 21.6M 6s 2024-03-29T04:19:17.975 INFO:teuthology.orchestra.run.smithi022.stderr: 54500K .......... .......... .......... .......... .......... 32% 22.9M 6s 2024-03-29T04:19:17.977 INFO:teuthology.orchestra.run.smithi022.stderr: 54550K .......... .......... .......... .......... .......... 32% 18.3M 6s 2024-03-29T04:19:17.979 INFO:teuthology.orchestra.run.smithi022.stderr: 54600K .......... .......... .......... .......... .......... 32% 21.6M 6s 2024-03-29T04:19:17.982 INFO:teuthology.orchestra.run.smithi022.stderr: 54650K .......... .......... .......... .......... .......... 32% 22.5M 6s 2024-03-29T04:19:17.984 INFO:teuthology.orchestra.run.smithi022.stderr: 54700K .......... .......... .......... .......... .......... 32% 22.0M 6s 2024-03-29T04:19:17.986 INFO:teuthology.orchestra.run.smithi022.stderr: 54750K .......... .......... .......... .......... .......... 32% 18.0M 6s 2024-03-29T04:19:17.989 INFO:teuthology.orchestra.run.smithi022.stderr: 54800K .......... .......... .......... .......... .......... 32% 20.2M 6s 2024-03-29T04:19:17.991 INFO:teuthology.orchestra.run.smithi022.stderr: 54850K .......... .......... .......... .......... .......... 32% 26.2M 6s 2024-03-29T04:19:17.993 INFO:teuthology.orchestra.run.smithi022.stderr: 54900K .......... .......... .......... .......... .......... 32% 26.3M 6s 2024-03-29T04:19:17.995 INFO:teuthology.orchestra.run.smithi022.stderr: 54950K .......... .......... .......... .......... .......... 32% 22.4M 6s 2024-03-29T04:19:17.997 INFO:teuthology.orchestra.run.smithi022.stderr: 55000K .......... .......... .......... .......... .......... 32% 26.2M 6s 2024-03-29T04:19:17.999 INFO:teuthology.orchestra.run.smithi022.stderr: 55050K .......... .......... .......... .......... .......... 32% 21.1M 6s 2024-03-29T04:19:18.001 INFO:teuthology.orchestra.run.smithi022.stderr: 55100K .......... .......... .......... .......... .......... 32% 24.5M 6s 2024-03-29T04:19:18.004 INFO:teuthology.orchestra.run.smithi022.stderr: 55150K .......... .......... .......... .......... .......... 32% 16.8M 6s 2024-03-29T04:19:18.006 INFO:teuthology.orchestra.run.smithi022.stderr: 55200K .......... .......... .......... .......... .......... 32% 23.7M 6s 2024-03-29T04:19:18.008 INFO:teuthology.orchestra.run.smithi022.stderr: 55250K .......... .......... .......... .......... .......... 32% 24.1M 6s 2024-03-29T04:19:18.010 INFO:teuthology.orchestra.run.smithi022.stderr: 55300K .......... .......... .......... .......... .......... 32% 24.8M 6s 2024-03-29T04:19:18.013 INFO:teuthology.orchestra.run.smithi022.stderr: 55350K .......... .......... .......... .......... .......... 32% 15.8M 6s 2024-03-29T04:19:18.016 INFO:teuthology.orchestra.run.smithi022.stderr: 55400K .......... .......... .......... .......... .......... 32% 18.6M 6s 2024-03-29T04:19:18.018 INFO:teuthology.orchestra.run.smithi022.stderr: 55450K .......... .......... .......... .......... .......... 32% 18.9M 6s 2024-03-29T04:19:18.021 INFO:teuthology.orchestra.run.smithi022.stderr: 55500K .......... .......... .......... .......... .......... 32% 17.8M 6s 2024-03-29T04:19:18.025 INFO:teuthology.orchestra.run.smithi022.stderr: 55550K .......... .......... .......... .......... .......... 32% 15.1M 6s 2024-03-29T04:19:18.027 INFO:teuthology.orchestra.run.smithi022.stderr: 55600K .......... .......... .......... .......... .......... 33% 17.7M 6s 2024-03-29T04:19:18.030 INFO:teuthology.orchestra.run.smithi022.stderr: 55650K .......... .......... .......... .......... .......... 33% 17.8M 6s 2024-03-29T04:19:18.033 INFO:teuthology.orchestra.run.smithi022.stderr: 55700K .......... .......... .......... .......... .......... 33% 17.0M 6s 2024-03-29T04:19:18.036 INFO:teuthology.orchestra.run.smithi022.stderr: 55750K .......... .......... .......... .......... .......... 33% 13.8M 6s 2024-03-29T04:19:18.043 INFO:teuthology.orchestra.run.smithi022.stderr: 55800K .......... .......... .......... .......... .......... 33% 17.7M 6s 2024-03-29T04:19:18.043 INFO:teuthology.orchestra.run.smithi022.stderr: 55850K .......... .......... .......... .......... .......... 33% 16.9M 6s 2024-03-29T04:19:18.045 INFO:teuthology.orchestra.run.smithi022.stderr: 55900K .......... .......... .......... .......... .......... 33% 16.4M 6s 2024-03-29T04:19:18.048 INFO:teuthology.orchestra.run.smithi022.stderr: 55950K .......... .......... .......... .......... .......... 33% 14.6M 6s 2024-03-29T04:19:18.051 INFO:teuthology.orchestra.run.smithi022.stderr: 56000K .......... .......... .......... .......... .......... 33% 16.3M 6s 2024-03-29T04:19:18.054 INFO:teuthology.orchestra.run.smithi022.stderr: 56050K .......... .......... .......... .......... .......... 33% 17.6M 6s 2024-03-29T04:19:18.057 INFO:teuthology.orchestra.run.smithi022.stderr: 56100K .......... .......... .......... .......... .......... 33% 17.3M 6s 2024-03-29T04:19:18.060 INFO:teuthology.orchestra.run.smithi022.stderr: 56150K .......... .......... .......... .......... .......... 33% 14.8M 6s 2024-03-29T04:19:18.063 INFO:teuthology.orchestra.run.smithi022.stderr: 56200K .......... .......... .......... .......... .......... 33% 17.4M 6s 2024-03-29T04:19:18.066 INFO:teuthology.orchestra.run.smithi022.stderr: 56250K .......... .......... .......... .......... .......... 33% 17.1M 6s 2024-03-29T04:19:18.069 INFO:teuthology.orchestra.run.smithi022.stderr: 56300K .......... .......... .......... .......... .......... 33% 16.5M 6s 2024-03-29T04:19:18.072 INFO:teuthology.orchestra.run.smithi022.stderr: 56350K .......... .......... .......... .......... .......... 33% 13.4M 6s 2024-03-29T04:19:18.076 INFO:teuthology.orchestra.run.smithi022.stderr: 56400K .......... .......... .......... .......... .......... 33% 15.3M 6s 2024-03-29T04:19:18.079 INFO:teuthology.orchestra.run.smithi022.stderr: 56450K .......... .......... .......... .......... .......... 33% 15.3M 6s 2024-03-29T04:19:18.082 INFO:teuthology.orchestra.run.smithi022.stderr: 56500K .......... .......... .......... .......... .......... 33% 15.3M 6s 2024-03-29T04:19:18.086 INFO:teuthology.orchestra.run.smithi022.stderr: 56550K .......... .......... .......... .......... .......... 33% 12.9M 6s 2024-03-29T04:19:18.089 INFO:teuthology.orchestra.run.smithi022.stderr: 56600K .......... .......... .......... .......... .......... 33% 15.0M 6s 2024-03-29T04:19:18.092 INFO:teuthology.orchestra.run.smithi022.stderr: 56650K .......... .......... .......... .......... .......... 33% 14.9M 6s 2024-03-29T04:19:18.096 INFO:teuthology.orchestra.run.smithi022.stderr: 56700K .......... .......... .......... .......... .......... 33% 15.0M 6s 2024-03-29T04:19:18.099 INFO:teuthology.orchestra.run.smithi022.stderr: 56750K .......... .......... .......... .......... .......... 33% 13.0M 6s 2024-03-29T04:19:18.103 INFO:teuthology.orchestra.run.smithi022.stderr: 56800K .......... .......... .......... .......... .......... 33% 15.2M 6s 2024-03-29T04:19:18.106 INFO:teuthology.orchestra.run.smithi022.stderr: 56850K .......... .......... .......... .......... .......... 33% 15.0M 6s 2024-03-29T04:19:18.109 INFO:teuthology.orchestra.run.smithi022.stderr: 56900K .......... .......... .......... .......... .......... 33% 14.9M 6s 2024-03-29T04:19:18.113 INFO:teuthology.orchestra.run.smithi022.stderr: 56950K .......... .......... .......... .......... .......... 33% 12.9M 6s 2024-03-29T04:19:18.116 INFO:teuthology.orchestra.run.smithi022.stderr: 57000K .......... .......... .......... .......... .......... 33% 14.6M 6s 2024-03-29T04:19:18.120 INFO:teuthology.orchestra.run.smithi022.stderr: 57050K .......... .......... .......... .......... .......... 33% 14.5M 6s 2024-03-29T04:19:18.123 INFO:teuthology.orchestra.run.smithi022.stderr: 57100K .......... .......... .......... .......... .......... 33% 14.5M 6s 2024-03-29T04:19:18.127 INFO:teuthology.orchestra.run.smithi022.stderr: 57150K .......... .......... .......... .......... .......... 33% 12.5M 6s 2024-03-29T04:19:18.130 INFO:teuthology.orchestra.run.smithi022.stderr: 57200K .......... .......... .......... .......... .......... 33% 14.6M 6s 2024-03-29T04:19:18.134 INFO:teuthology.orchestra.run.smithi022.stderr: 57250K .......... .......... .......... .......... .......... 33% 13.8M 6s 2024-03-29T04:19:18.137 INFO:teuthology.orchestra.run.smithi022.stderr: 57300K .......... .......... .......... .......... .......... 34% 14.3M 6s 2024-03-29T04:19:18.140 INFO:teuthology.orchestra.run.smithi022.stderr: 57350K .......... .......... .......... .......... .......... 34% 14.1M 6s 2024-03-29T04:19:18.143 INFO:teuthology.orchestra.run.smithi022.stderr: 57400K .......... .......... .......... .......... .......... 34% 20.9M 6s 2024-03-29T04:19:18.145 INFO:teuthology.orchestra.run.smithi022.stderr: 57450K .......... .......... .......... .......... .......... 34% 21.7M 6s 2024-03-29T04:19:18.147 INFO:teuthology.orchestra.run.smithi022.stderr: 57500K .......... .......... .......... .......... .......... 34% 20.5M 6s 2024-03-29T04:19:18.150 INFO:teuthology.orchestra.run.smithi022.stderr: 57550K .......... .......... .......... .......... .......... 34% 18.0M 6s 2024-03-29T04:19:18.152 INFO:teuthology.orchestra.run.smithi022.stderr: 57600K .......... .......... .......... .......... .......... 34% 20.3M 6s 2024-03-29T04:19:18.155 INFO:teuthology.orchestra.run.smithi022.stderr: 57650K .......... .......... .......... .......... .......... 34% 20.7M 6s 2024-03-29T04:19:18.157 INFO:teuthology.orchestra.run.smithi022.stderr: 57700K .......... .......... .......... .......... .......... 34% 20.5M 6s 2024-03-29T04:19:18.160 INFO:teuthology.orchestra.run.smithi022.stderr: 57750K .......... .......... .......... .......... .......... 34% 17.6M 6s 2024-03-29T04:19:18.162 INFO:teuthology.orchestra.run.smithi022.stderr: 57800K .......... .......... .......... .......... .......... 34% 20.6M 6s 2024-03-29T04:19:18.165 INFO:teuthology.orchestra.run.smithi022.stderr: 57850K .......... .......... .......... .......... .......... 34% 20.5M 6s 2024-03-29T04:19:18.167 INFO:teuthology.orchestra.run.smithi022.stderr: 57900K .......... .......... .......... .......... .......... 34% 20.1M 6s 2024-03-29T04:19:18.170 INFO:teuthology.orchestra.run.smithi022.stderr: 57950K .......... .......... .......... .......... .......... 34% 17.3M 6s 2024-03-29T04:19:18.172 INFO:teuthology.orchestra.run.smithi022.stderr: 58000K .......... .......... .......... .......... .......... 34% 22.7M 6s 2024-03-29T04:19:18.174 INFO:teuthology.orchestra.run.smithi022.stderr: 58050K .......... .......... .......... .......... .......... 34% 24.5M 6s 2024-03-29T04:19:18.176 INFO:teuthology.orchestra.run.smithi022.stderr: 58100K .......... .......... .......... .......... .......... 34% 24.7M 6s 2024-03-29T04:19:18.179 INFO:teuthology.orchestra.run.smithi022.stderr: 58150K .......... .......... .......... .......... .......... 34% 20.3M 6s 2024-03-29T04:19:18.181 INFO:teuthology.orchestra.run.smithi022.stderr: 58200K .......... .......... .......... .......... .......... 34% 24.0M 6s 2024-03-29T04:19:18.183 INFO:teuthology.orchestra.run.smithi022.stderr: 58250K .......... .......... .......... .......... .......... 34% 23.9M 6s 2024-03-29T04:19:18.185 INFO:teuthology.orchestra.run.smithi022.stderr: 58300K .......... .......... .......... .......... .......... 34% 23.8M 6s 2024-03-29T04:19:18.187 INFO:teuthology.orchestra.run.smithi022.stderr: 58350K .......... .......... .......... .......... .......... 34% 20.5M 6s 2024-03-29T04:19:18.189 INFO:teuthology.orchestra.run.smithi022.stderr: 58400K .......... .......... .......... .......... .......... 34% 24.0M 6s 2024-03-29T04:19:18.191 INFO:teuthology.orchestra.run.smithi022.stderr: 58450K .......... .......... .......... .......... .......... 34% 24.4M 6s 2024-03-29T04:19:18.193 INFO:teuthology.orchestra.run.smithi022.stderr: 58500K .......... .......... .......... .......... .......... 34% 22.6M 6s 2024-03-29T04:19:18.196 INFO:teuthology.orchestra.run.smithi022.stderr: 58550K .......... .......... .......... .......... .......... 34% 19.9M 6s 2024-03-29T04:19:18.198 INFO:teuthology.orchestra.run.smithi022.stderr: 58600K .......... .......... .......... .......... .......... 34% 23.4M 6s 2024-03-29T04:19:18.200 INFO:teuthology.orchestra.run.smithi022.stderr: 58650K .......... .......... .......... .......... .......... 34% 23.3M 6s 2024-03-29T04:19:18.202 INFO:teuthology.orchestra.run.smithi022.stderr: 58700K .......... .......... .......... .......... .......... 34% 22.7M 6s 2024-03-29T04:19:18.204 INFO:teuthology.orchestra.run.smithi022.stderr: 58750K .......... .......... .......... .......... .......... 34% 19.5M 6s 2024-03-29T04:19:18.206 INFO:teuthology.orchestra.run.smithi022.stderr: 58800K .......... .......... .......... .......... .......... 34% 23.3M 6s 2024-03-29T04:19:18.209 INFO:teuthology.orchestra.run.smithi022.stderr: 58850K .......... .......... .......... .......... .......... 34% 23.1M 6s 2024-03-29T04:19:18.211 INFO:teuthology.orchestra.run.smithi022.stderr: 58900K .......... .......... .......... .......... .......... 34% 23.0M 6s 2024-03-29T04:19:18.213 INFO:teuthology.orchestra.run.smithi022.stderr: 58950K .......... .......... .......... .......... .......... 35% 19.4M 6s 2024-03-29T04:19:18.216 INFO:teuthology.orchestra.run.smithi022.stderr: 59000K .......... .......... .......... .......... .......... 35% 21.9M 6s 2024-03-29T04:19:18.218 INFO:teuthology.orchestra.run.smithi022.stderr: 59050K .......... .......... .......... .......... .......... 35% 22.2M 6s 2024-03-29T04:19:18.220 INFO:teuthology.orchestra.run.smithi022.stderr: 59100K .......... .......... .......... .......... .......... 35% 21.0M 6s 2024-03-29T04:19:18.223 INFO:teuthology.orchestra.run.smithi022.stderr: 59150K .......... .......... .......... .......... .......... 35% 18.3M 6s 2024-03-29T04:19:18.225 INFO:teuthology.orchestra.run.smithi022.stderr: 59200K .......... .......... .......... .......... .......... 35% 19.9M 6s 2024-03-29T04:19:18.228 INFO:teuthology.orchestra.run.smithi022.stderr: 59250K .......... .......... .......... .......... .......... 35% 20.1M 6s 2024-03-29T04:19:18.230 INFO:teuthology.orchestra.run.smithi022.stderr: 59300K .......... .......... .......... .......... .......... 35% 20.1M 6s 2024-03-29T04:19:18.233 INFO:teuthology.orchestra.run.smithi022.stderr: 59350K .......... .......... .......... .......... .......... 35% 16.4M 6s 2024-03-29T04:19:18.236 INFO:teuthology.orchestra.run.smithi022.stderr: 59400K .......... .......... .......... .......... .......... 35% 19.8M 6s 2024-03-29T04:19:18.238 INFO:teuthology.orchestra.run.smithi022.stderr: 59450K .......... .......... .......... .......... .......... 35% 19.9M 6s 2024-03-29T04:19:18.240 INFO:teuthology.orchestra.run.smithi022.stderr: 59500K .......... .......... .......... .......... .......... 35% 19.8M 6s 2024-03-29T04:19:18.243 INFO:teuthology.orchestra.run.smithi022.stderr: 59550K .......... .......... .......... .......... .......... 35% 17.0M 6s 2024-03-29T04:19:18.246 INFO:teuthology.orchestra.run.smithi022.stderr: 59600K .......... .......... .......... .......... .......... 35% 19.8M 6s 2024-03-29T04:19:18.248 INFO:teuthology.orchestra.run.smithi022.stderr: 59650K .......... .......... .......... .......... .......... 35% 20.0M 6s 2024-03-29T04:19:18.251 INFO:teuthology.orchestra.run.smithi022.stderr: 59700K .......... .......... .......... .......... .......... 35% 19.7M 6s 2024-03-29T04:19:18.254 INFO:teuthology.orchestra.run.smithi022.stderr: 59750K .......... .......... .......... .......... .......... 35% 15.8M 6s 2024-03-29T04:19:18.257 INFO:teuthology.orchestra.run.smithi022.stderr: 59800K .......... .......... .......... .......... .......... 35% 16.1M 6s 2024-03-29T04:19:18.260 INFO:teuthology.orchestra.run.smithi022.stderr: 59850K .......... .......... .......... .......... .......... 35% 16.1M 6s 2024-03-29T04:19:18.263 INFO:teuthology.orchestra.run.smithi022.stderr: 59900K .......... .......... .......... .......... .......... 35% 16.3M 6s 2024-03-29T04:19:18.267 INFO:teuthology.orchestra.run.smithi022.stderr: 59950K .......... .......... .......... .......... .......... 35% 13.9M 6s 2024-03-29T04:19:18.269 INFO:teuthology.orchestra.run.smithi022.stderr: 60000K .......... .......... .......... .......... .......... 35% 16.6M 6s 2024-03-29T04:19:18.272 INFO:teuthology.orchestra.run.smithi022.stderr: 60050K .......... .......... .......... .......... .......... 35% 15.8M 6s 2024-03-29T04:19:18.275 INFO:teuthology.orchestra.run.smithi022.stderr: 60100K .......... .......... .......... .......... .......... 35% 19.1M 6s 2024-03-29T04:19:18.278 INFO:teuthology.orchestra.run.smithi022.stderr: 60150K .......... .......... .......... .......... .......... 35% 16.0M 6s 2024-03-29T04:19:18.281 INFO:teuthology.orchestra.run.smithi022.stderr: 60200K .......... .......... .......... .......... .......... 35% 18.2M 6s 2024-03-29T04:19:18.283 INFO:teuthology.orchestra.run.smithi022.stderr: 60250K .......... .......... .......... .......... .......... 35% 18.7M 6s 2024-03-29T04:19:18.286 INFO:teuthology.orchestra.run.smithi022.stderr: 60300K .......... .......... .......... .......... .......... 35% 18.5M 6s 2024-03-29T04:19:18.289 INFO:teuthology.orchestra.run.smithi022.stderr: 60350K .......... .......... .......... .......... .......... 35% 16.2M 6s 2024-03-29T04:19:18.292 INFO:teuthology.orchestra.run.smithi022.stderr: 60400K .......... .......... .......... .......... .......... 35% 18.6M 6s 2024-03-29T04:19:18.294 INFO:teuthology.orchestra.run.smithi022.stderr: 60450K .......... .......... .......... .......... .......... 35% 18.8M 6s 2024-03-29T04:19:18.297 INFO:teuthology.orchestra.run.smithi022.stderr: 60500K .......... .......... .......... .......... .......... 35% 17.4M 6s 2024-03-29T04:19:18.300 INFO:teuthology.orchestra.run.smithi022.stderr: 60550K .......... .......... .......... .......... .......... 35% 16.1M 6s 2024-03-29T04:19:18.303 INFO:teuthology.orchestra.run.smithi022.stderr: 60600K .......... .......... .......... .......... .......... 35% 17.8M 6s 2024-03-29T04:19:18.306 INFO:teuthology.orchestra.run.smithi022.stderr: 60650K .......... .......... .......... .......... .......... 36% 16.2M 6s 2024-03-29T04:19:18.308 INFO:teuthology.orchestra.run.smithi022.stderr: 60700K .......... .......... .......... .......... .......... 36% 19.4M 6s 2024-03-29T04:19:18.311 INFO:teuthology.orchestra.run.smithi022.stderr: 60750K .......... .......... .......... .......... .......... 36% 15.8M 6s 2024-03-29T04:19:18.314 INFO:teuthology.orchestra.run.smithi022.stderr: 60800K .......... .......... .......... .......... .......... 36% 19.5M 6s 2024-03-29T04:19:18.317 INFO:teuthology.orchestra.run.smithi022.stderr: 60850K .......... .......... .......... .......... .......... 36% 15.7M 6s 2024-03-29T04:19:18.321 INFO:teuthology.orchestra.run.smithi022.stderr: 60900K .......... .......... .......... .......... .......... 36% 12.0M 6s 2024-03-29T04:19:18.326 INFO:teuthology.orchestra.run.smithi022.stderr: 60950K .......... .......... .......... .......... .......... 36% 10.7M 6s 2024-03-29T04:19:18.330 INFO:teuthology.orchestra.run.smithi022.stderr: 61000K .......... .......... .......... .......... .......... 36% 11.8M 6s 2024-03-29T04:19:18.334 INFO:teuthology.orchestra.run.smithi022.stderr: 61050K .......... .......... .......... .......... .......... 36% 12.1M 6s 2024-03-29T04:19:18.338 INFO:teuthology.orchestra.run.smithi022.stderr: 61100K .......... .......... .......... .......... .......... 36% 11.8M 6s 2024-03-29T04:19:18.343 INFO:teuthology.orchestra.run.smithi022.stderr: 61150K .......... .......... .......... .......... .......... 36% 10.1M 6s 2024-03-29T04:19:18.347 INFO:teuthology.orchestra.run.smithi022.stderr: 61200K .......... .......... .......... .......... .......... 36% 12.2M 6s 2024-03-29T04:19:18.352 INFO:teuthology.orchestra.run.smithi022.stderr: 61250K .......... .......... .......... .......... .......... 36% 12.2M 6s 2024-03-29T04:19:18.356 INFO:teuthology.orchestra.run.smithi022.stderr: 61300K .......... .......... .......... .......... .......... 36% 9.46M 6s 2024-03-29T04:19:18.363 INFO:teuthology.orchestra.run.smithi022.stderr: 61350K .......... .......... .......... .......... .......... 36% 8.03M 6s 2024-03-29T04:19:18.367 INFO:teuthology.orchestra.run.smithi022.stderr: 61400K .......... .......... .......... .......... .......... 36% 10.1M 6s 2024-03-29T04:19:18.371 INFO:teuthology.orchestra.run.smithi022.stderr: 61450K .......... .......... .......... .......... .......... 36% 11.3M 6s 2024-03-29T04:19:18.375 INFO:teuthology.orchestra.run.smithi022.stderr: 61500K .......... .......... .......... .......... .......... 36% 14.9M 6s 2024-03-29T04:19:18.378 INFO:teuthology.orchestra.run.smithi022.stderr: 61550K .......... .......... .......... .......... .......... 36% 14.2M 6s 2024-03-29T04:19:18.382 INFO:teuthology.orchestra.run.smithi022.stderr: 61600K .......... .......... .......... .......... .......... 36% 14.2M 6s 2024-03-29T04:19:18.384 INFO:teuthology.orchestra.run.smithi022.stderr: 61650K .......... .......... .......... .......... .......... 36% 16.2M 6s 2024-03-29T04:19:18.387 INFO:teuthology.orchestra.run.smithi022.stderr: 61700K .......... .......... .......... .......... .......... 36% 19.3M 6s 2024-03-29T04:19:18.389 INFO:teuthology.orchestra.run.smithi022.stderr: 61750K .......... .......... .......... .......... .......... 36% 19.4M 6s 2024-03-29T04:19:18.392 INFO:teuthology.orchestra.run.smithi022.stderr: 61800K .......... .......... .......... .......... .......... 36% 17.9M 6s 2024-03-29T04:19:18.394 INFO:teuthology.orchestra.run.smithi022.stderr: 61850K .......... .......... .......... .......... .......... 36% 23.2M 6s 2024-03-29T04:19:18.396 INFO:teuthology.orchestra.run.smithi022.stderr: 61900K .......... .......... .......... .......... .......... 36% 22.9M 6s 2024-03-29T04:19:18.399 INFO:teuthology.orchestra.run.smithi022.stderr: 61950K .......... .......... .......... .......... .......... 36% 19.2M 6s 2024-03-29T04:19:18.401 INFO:teuthology.orchestra.run.smithi022.stderr: 62000K .......... .......... .......... .......... .......... 36% 22.4M 6s 2024-03-29T04:19:18.403 INFO:teuthology.orchestra.run.smithi022.stderr: 62050K .......... .......... .......... .......... .......... 36% 21.5M 6s 2024-03-29T04:19:18.406 INFO:teuthology.orchestra.run.smithi022.stderr: 62100K .......... .......... .......... .......... .......... 36% 19.8M 6s 2024-03-29T04:19:18.409 INFO:teuthology.orchestra.run.smithi022.stderr: 62150K .......... .......... .......... .......... .......... 36% 17.6M 6s 2024-03-29T04:19:18.411 INFO:teuthology.orchestra.run.smithi022.stderr: 62200K .......... .......... .......... .......... .......... 36% 21.0M 6s 2024-03-29T04:19:18.413 INFO:teuthology.orchestra.run.smithi022.stderr: 62250K .......... .......... .......... .......... .......... 36% 20.7M 6s 2024-03-29T04:19:18.416 INFO:teuthology.orchestra.run.smithi022.stderr: 62300K .......... .......... .......... .......... .......... 36% 20.7M 6s 2024-03-29T04:19:18.418 INFO:teuthology.orchestra.run.smithi022.stderr: 62350K .......... .......... .......... .......... .......... 37% 17.5M 6s 2024-03-29T04:19:18.421 INFO:teuthology.orchestra.run.smithi022.stderr: 62400K .......... .......... .......... .......... .......... 37% 22.9M 6s 2024-03-29T04:19:18.423 INFO:teuthology.orchestra.run.smithi022.stderr: 62450K .......... .......... .......... .......... .......... 37% 18.4M 6s 2024-03-29T04:19:18.426 INFO:teuthology.orchestra.run.smithi022.stderr: 62500K .......... .......... .......... .......... .......... 37% 18.1M 6s 2024-03-29T04:19:18.429 INFO:teuthology.orchestra.run.smithi022.stderr: 62550K .......... .......... .......... .......... .......... 37% 16.0M 6s 2024-03-29T04:19:18.432 INFO:teuthology.orchestra.run.smithi022.stderr: 62600K .......... .......... .......... .......... .......... 37% 17.8M 6s 2024-03-29T04:19:18.435 INFO:teuthology.orchestra.run.smithi022.stderr: 62650K .......... .......... .......... .......... .......... 37% 17.4M 6s 2024-03-29T04:19:18.437 INFO:teuthology.orchestra.run.smithi022.stderr: 62700K .......... .......... .......... .......... .......... 37% 17.2M 6s 2024-03-29T04:19:18.441 INFO:teuthology.orchestra.run.smithi022.stderr: 62750K .......... .......... .......... .......... .......... 37% 13.5M 6s 2024-03-29T04:19:18.445 INFO:teuthology.orchestra.run.smithi022.stderr: 62800K .......... .......... .......... .......... .......... 37% 14.0M 6s 2024-03-29T04:19:18.451 INFO:teuthology.orchestra.run.smithi022.stderr: 62850K .......... .......... .......... .......... .......... 37% 13.7M 6s 2024-03-29T04:19:18.451 INFO:teuthology.orchestra.run.smithi022.stderr: 62900K .......... .......... .......... .......... .......... 37% 17.9M 6s 2024-03-29T04:19:18.454 INFO:teuthology.orchestra.run.smithi022.stderr: 62950K .......... .......... .......... .......... .......... 37% 16.3M 6s 2024-03-29T04:19:18.457 INFO:teuthology.orchestra.run.smithi022.stderr: 63000K .......... .......... .......... .......... .......... 37% 15.6M 6s 2024-03-29T04:19:18.460 INFO:teuthology.orchestra.run.smithi022.stderr: 63050K .......... .......... .......... .......... .......... 37% 15.7M 6s 2024-03-29T04:19:18.463 INFO:teuthology.orchestra.run.smithi022.stderr: 63100K .......... .......... .......... .......... .......... 37% 17.0M 6s 2024-03-29T04:19:18.466 INFO:teuthology.orchestra.run.smithi022.stderr: 63150K .......... .......... .......... .......... .......... 37% 17.6M 6s 2024-03-29T04:19:18.468 INFO:teuthology.orchestra.run.smithi022.stderr: 63200K .......... .......... .......... .......... .......... 37% 20.2M 6s 2024-03-29T04:19:18.470 INFO:teuthology.orchestra.run.smithi022.stderr: 63250K .......... .......... .......... .......... .......... 37% 20.5M 6s 2024-03-29T04:19:18.474 INFO:teuthology.orchestra.run.smithi022.stderr: 63300K .......... .......... .......... .......... .......... 37% 16.9M 6s 2024-03-29T04:19:18.478 INFO:teuthology.orchestra.run.smithi022.stderr: 63350K .......... .......... .......... .......... .......... 37% 11.9M 6s 2024-03-29T04:19:18.481 INFO:teuthology.orchestra.run.smithi022.stderr: 63400K .......... .......... .......... .......... .......... 37% 13.6M 6s 2024-03-29T04:19:18.485 INFO:teuthology.orchestra.run.smithi022.stderr: 63450K .......... .......... .......... .......... .......... 37% 13.9M 6s 2024-03-29T04:19:18.488 INFO:teuthology.orchestra.run.smithi022.stderr: 63500K .......... .......... .......... .......... .......... 37% 13.8M 6s 2024-03-29T04:19:18.492 INFO:teuthology.orchestra.run.smithi022.stderr: 63550K .......... .......... .......... .......... .......... 37% 12.1M 6s 2024-03-29T04:19:18.496 INFO:teuthology.orchestra.run.smithi022.stderr: 63600K .......... .......... .......... .......... .......... 37% 14.3M 6s 2024-03-29T04:19:18.499 INFO:teuthology.orchestra.run.smithi022.stderr: 63650K .......... .......... .......... .......... .......... 37% 13.9M 6s 2024-03-29T04:19:18.503 INFO:teuthology.orchestra.run.smithi022.stderr: 63700K .......... .......... .......... .......... .......... 37% 14.2M 6s 2024-03-29T04:19:18.507 INFO:teuthology.orchestra.run.smithi022.stderr: 63750K .......... .......... .......... .......... .......... 37% 12.1M 6s 2024-03-29T04:19:18.511 INFO:teuthology.orchestra.run.smithi022.stderr: 63800K .......... .......... .......... .......... .......... 37% 13.0M 6s 2024-03-29T04:19:18.514 INFO:teuthology.orchestra.run.smithi022.stderr: 63850K .......... .......... .......... .......... .......... 37% 13.7M 6s 2024-03-29T04:19:18.518 INFO:teuthology.orchestra.run.smithi022.stderr: 63900K .......... .......... .......... .......... .......... 37% 13.9M 6s 2024-03-29T04:19:18.521 INFO:teuthology.orchestra.run.smithi022.stderr: 63950K .......... .......... .......... .......... .......... 37% 14.5M 6s 2024-03-29T04:19:18.524 INFO:teuthology.orchestra.run.smithi022.stderr: 64000K .......... .......... .......... .......... .......... 37% 18.0M 6s 2024-03-29T04:19:18.526 INFO:teuthology.orchestra.run.smithi022.stderr: 64050K .......... .......... .......... .......... .......... 38% 17.5M 6s 2024-03-29T04:19:18.529 INFO:teuthology.orchestra.run.smithi022.stderr: 64100K .......... .......... .......... .......... .......... 38% 17.5M 6s 2024-03-29T04:19:18.532 INFO:teuthology.orchestra.run.smithi022.stderr: 64150K .......... .......... .......... .......... .......... 38% 15.6M 6s 2024-03-29T04:19:18.535 INFO:teuthology.orchestra.run.smithi022.stderr: 64200K .......... .......... .......... .......... .......... 38% 18.2M 6s 2024-03-29T04:19:18.537 INFO:teuthology.orchestra.run.smithi022.stderr: 64250K .......... .......... .......... .......... .......... 38% 18.3M 6s 2024-03-29T04:19:18.541 INFO:teuthology.orchestra.run.smithi022.stderr: 64300K .......... .......... .......... .......... .......... 38% 16.6M 6s 2024-03-29T04:19:18.546 INFO:teuthology.orchestra.run.smithi022.stderr: 64350K .......... .......... .......... .......... .......... 38% 10.1M 6s 2024-03-29T04:19:18.550 INFO:teuthology.orchestra.run.smithi022.stderr: 64400K .......... .......... .......... .......... .......... 38% 12.7M 6s 2024-03-29T04:19:18.554 INFO:teuthology.orchestra.run.smithi022.stderr: 64450K .......... .......... .......... .......... .......... 38% 11.9M 6s 2024-03-29T04:19:18.557 INFO:teuthology.orchestra.run.smithi022.stderr: 64500K .......... .......... .......... .......... .......... 38% 12.5M 6s 2024-03-29T04:19:18.562 INFO:teuthology.orchestra.run.smithi022.stderr: 64550K .......... .......... .......... .......... .......... 38% 10.8M 6s 2024-03-29T04:19:18.566 INFO:teuthology.orchestra.run.smithi022.stderr: 64600K .......... .......... .......... .......... .......... 38% 11.8M 6s 2024-03-29T04:19:18.570 INFO:teuthology.orchestra.run.smithi022.stderr: 64650K .......... .......... .......... .......... .......... 38% 11.8M 6s 2024-03-29T04:19:18.575 INFO:teuthology.orchestra.run.smithi022.stderr: 64700K .......... .......... .......... .......... .......... 38% 10.3M 6s 2024-03-29T04:19:18.581 INFO:teuthology.orchestra.run.smithi022.stderr: 64750K .......... .......... .......... .......... .......... 38% 8.13M 6s 2024-03-29T04:19:18.586 INFO:teuthology.orchestra.run.smithi022.stderr: 64800K .......... .......... .......... .......... .......... 38% 9.69M 6s 2024-03-29T04:19:18.591 INFO:teuthology.orchestra.run.smithi022.stderr: 64850K .......... .......... .......... .......... .......... 38% 9.58M 6s 2024-03-29T04:19:18.595 INFO:teuthology.orchestra.run.smithi022.stderr: 64900K .......... .......... .......... .......... .......... 38% 11.5M 6s 2024-03-29T04:19:18.600 INFO:teuthology.orchestra.run.smithi022.stderr: 64950K .......... .......... .......... .......... .......... 38% 10.5M 6s 2024-03-29T04:19:18.604 INFO:teuthology.orchestra.run.smithi022.stderr: 65000K .......... .......... .......... .......... .......... 38% 12.3M 6s 2024-03-29T04:19:18.608 INFO:teuthology.orchestra.run.smithi022.stderr: 65050K .......... .......... .......... .......... .......... 38% 12.8M 6s 2024-03-29T04:19:18.612 INFO:teuthology.orchestra.run.smithi022.stderr: 65100K .......... .......... .......... .......... .......... 38% 12.8M 6s 2024-03-29T04:19:18.616 INFO:teuthology.orchestra.run.smithi022.stderr: 65150K .......... .......... .......... .......... .......... 38% 10.7M 6s 2024-03-29T04:19:18.620 INFO:teuthology.orchestra.run.smithi022.stderr: 65200K .......... .......... .......... .......... .......... 38% 12.5M 6s 2024-03-29T04:19:18.624 INFO:teuthology.orchestra.run.smithi022.stderr: 65250K .......... .......... .......... .......... .......... 38% 11.8M 6s 2024-03-29T04:19:18.628 INFO:teuthology.orchestra.run.smithi022.stderr: 65300K .......... .......... .......... .......... .......... 38% 12.3M 6s 2024-03-29T04:19:18.633 INFO:teuthology.orchestra.run.smithi022.stderr: 65350K .......... .......... .......... .......... .......... 38% 10.8M 6s 2024-03-29T04:19:18.637 INFO:teuthology.orchestra.run.smithi022.stderr: 65400K .......... .......... .......... .......... .......... 38% 12.7M 6s 2024-03-29T04:19:18.640 INFO:teuthology.orchestra.run.smithi022.stderr: 65450K .......... .......... .......... .......... .......... 38% 13.1M 6s 2024-03-29T04:19:18.645 INFO:teuthology.orchestra.run.smithi022.stderr: 65500K .......... .......... .......... .......... .......... 38% 12.4M 6s 2024-03-29T04:19:18.649 INFO:teuthology.orchestra.run.smithi022.stderr: 65550K .......... .......... .......... .......... .......... 38% 9.19M 6s 2024-03-29T04:19:18.652 INFO:teuthology.orchestra.run.smithi022.stderr: 65600K .......... .......... .......... .......... .......... 38% 16.8M 6s 2024-03-29T04:19:18.659 INFO:teuthology.orchestra.run.smithi022.stderr: 65650K .......... .......... .......... .......... .......... 38% 14.3M 6s 2024-03-29T04:19:18.659 INFO:teuthology.orchestra.run.smithi022.stderr: 65700K .......... .......... .......... .......... .......... 39% 19.2M 6s 2024-03-29T04:19:18.661 INFO:teuthology.orchestra.run.smithi022.stderr: 65750K .......... .......... .......... .......... .......... 39% 17.4M 6s 2024-03-29T04:19:18.663 INFO:teuthology.orchestra.run.smithi022.stderr: 65800K .......... .......... .......... .......... .......... 39% 22.1M 6s 2024-03-29T04:19:18.665 INFO:teuthology.orchestra.run.smithi022.stderr: 65850K .......... .......... .......... .......... .......... 39% 22.6M 6s 2024-03-29T04:19:18.667 INFO:teuthology.orchestra.run.smithi022.stderr: 65900K .......... .......... .......... .......... .......... 39% 20.4M 6s 2024-03-29T04:19:18.670 INFO:teuthology.orchestra.run.smithi022.stderr: 65950K .......... .......... .......... .......... .......... 39% 19.4M 6s 2024-03-29T04:19:18.672 INFO:teuthology.orchestra.run.smithi022.stderr: 66000K .......... .......... .......... .......... .......... 39% 24.9M 6s 2024-03-29T04:19:18.674 INFO:teuthology.orchestra.run.smithi022.stderr: 66050K .......... .......... .......... .......... .......... 39% 27.8M 6s 2024-03-29T04:19:18.676 INFO:teuthology.orchestra.run.smithi022.stderr: 66100K .......... .......... .......... .......... .......... 39% 26.7M 6s 2024-03-29T04:19:18.678 INFO:teuthology.orchestra.run.smithi022.stderr: 66150K .......... .......... .......... .......... .......... 39% 22.7M 6s 2024-03-29T04:19:18.680 INFO:teuthology.orchestra.run.smithi022.stderr: 66200K .......... .......... .......... .......... .......... 39% 27.3M 6s 2024-03-29T04:19:18.682 INFO:teuthology.orchestra.run.smithi022.stderr: 66250K .......... .......... .......... .......... .......... 39% 26.9M 6s 2024-03-29T04:19:18.683 INFO:teuthology.orchestra.run.smithi022.stderr: 66300K .......... .......... .......... .......... .......... 39% 26.6M 6s 2024-03-29T04:19:18.685 INFO:teuthology.orchestra.run.smithi022.stderr: 66350K .......... .......... .......... .......... .......... 39% 22.3M 6s 2024-03-29T04:19:18.687 INFO:teuthology.orchestra.run.smithi022.stderr: 66400K .......... .......... .......... .......... .......... 39% 25.1M 6s 2024-03-29T04:19:18.689 INFO:teuthology.orchestra.run.smithi022.stderr: 66450K .......... .......... .......... .......... .......... 39% 25.4M 6s 2024-03-29T04:19:18.691 INFO:teuthology.orchestra.run.smithi022.stderr: 66500K .......... .......... .......... .......... .......... 39% 26.2M 6s 2024-03-29T04:19:18.694 INFO:teuthology.orchestra.run.smithi022.stderr: 66550K .......... .......... .......... .......... .......... 39% 20.6M 6s 2024-03-29T04:19:18.695 INFO:teuthology.orchestra.run.smithi022.stderr: 66600K .......... .......... .......... .......... .......... 39% 24.2M 6s 2024-03-29T04:19:18.697 INFO:teuthology.orchestra.run.smithi022.stderr: 66650K .......... .......... .......... .......... .......... 39% 24.2M 6s 2024-03-29T04:19:18.700 INFO:teuthology.orchestra.run.smithi022.stderr: 66700K .......... .......... .......... .......... .......... 39% 23.8M 6s 2024-03-29T04:19:18.702 INFO:teuthology.orchestra.run.smithi022.stderr: 66750K .......... .......... .......... .......... .......... 39% 18.3M 6s 2024-03-29T04:19:18.705 INFO:teuthology.orchestra.run.smithi022.stderr: 66800K .......... .......... .......... .......... .......... 39% 22.0M 6s 2024-03-29T04:19:18.707 INFO:teuthology.orchestra.run.smithi022.stderr: 66850K .......... .......... .......... .......... .......... 39% 23.4M 6s 2024-03-29T04:19:18.709 INFO:teuthology.orchestra.run.smithi022.stderr: 66900K .......... .......... .......... .......... .......... 39% 22.0M 6s 2024-03-29T04:19:18.711 INFO:teuthology.orchestra.run.smithi022.stderr: 66950K .......... .......... .......... .......... .......... 39% 18.8M 6s 2024-03-29T04:19:18.714 INFO:teuthology.orchestra.run.smithi022.stderr: 67000K .......... .......... .......... .......... .......... 39% 20.0M 6s 2024-03-29T04:19:18.716 INFO:teuthology.orchestra.run.smithi022.stderr: 67050K .......... .......... .......... .......... .......... 39% 18.6M 6s 2024-03-29T04:19:18.719 INFO:teuthology.orchestra.run.smithi022.stderr: 67100K .......... .......... .......... .......... .......... 39% 17.8M 6s 2024-03-29T04:19:18.722 INFO:teuthology.orchestra.run.smithi022.stderr: 67150K .......... .......... .......... .......... .......... 39% 16.7M 6s 2024-03-29T04:19:18.724 INFO:teuthology.orchestra.run.smithi022.stderr: 67200K .......... .......... .......... .......... .......... 39% 22.7M 6s 2024-03-29T04:19:18.726 INFO:teuthology.orchestra.run.smithi022.stderr: 67250K .......... .......... .......... .......... .......... 39% 23.0M 6s 2024-03-29T04:19:18.729 INFO:teuthology.orchestra.run.smithi022.stderr: 67300K .......... .......... .......... .......... .......... 39% 22.0M 6s 2024-03-29T04:19:18.731 INFO:teuthology.orchestra.run.smithi022.stderr: 67350K .......... .......... .......... .......... .......... 39% 19.0M 6s 2024-03-29T04:19:18.734 INFO:teuthology.orchestra.run.smithi022.stderr: 67400K .......... .......... .......... .......... .......... 40% 21.4M 6s 2024-03-29T04:19:18.736 INFO:teuthology.orchestra.run.smithi022.stderr: 67450K .......... .......... .......... .......... .......... 40% 22.2M 6s 2024-03-29T04:19:18.738 INFO:teuthology.orchestra.run.smithi022.stderr: 67500K .......... .......... .......... .......... .......... 40% 22.0M 6s 2024-03-29T04:19:18.741 INFO:teuthology.orchestra.run.smithi022.stderr: 67550K .......... .......... .......... .......... .......... 40% 18.6M 6s 2024-03-29T04:19:18.743 INFO:teuthology.orchestra.run.smithi022.stderr: 67600K .......... .......... .......... .......... .......... 40% 21.7M 6s 2024-03-29T04:19:18.745 INFO:teuthology.orchestra.run.smithi022.stderr: 67650K .......... .......... .......... .......... .......... 40% 21.7M 6s 2024-03-29T04:19:18.747 INFO:teuthology.orchestra.run.smithi022.stderr: 67700K .......... .......... .......... .......... .......... 40% 21.1M 6s 2024-03-29T04:19:18.750 INFO:teuthology.orchestra.run.smithi022.stderr: 67750K .......... .......... .......... .......... .......... 40% 18.1M 6s 2024-03-29T04:19:18.753 INFO:teuthology.orchestra.run.smithi022.stderr: 67800K .......... .......... .......... .......... .......... 40% 18.1M 6s 2024-03-29T04:19:18.757 INFO:teuthology.orchestra.run.smithi022.stderr: 67850K .......... .......... .......... .......... .......... 40% 12.5M 6s 2024-03-29T04:19:18.761 INFO:teuthology.orchestra.run.smithi022.stderr: 67900K .......... .......... .......... .......... .......... 40% 12.7M 6s 2024-03-29T04:19:18.765 INFO:teuthology.orchestra.run.smithi022.stderr: 67950K .......... .......... .......... .......... .......... 40% 10.6M 6s 2024-03-29T04:19:18.769 INFO:teuthology.orchestra.run.smithi022.stderr: 68000K .......... .......... .......... .......... .......... 40% 15.3M 6s 2024-03-29T04:19:18.772 INFO:teuthology.orchestra.run.smithi022.stderr: 68050K .......... .......... .......... .......... .......... 40% 15.6M 6s 2024-03-29T04:19:18.775 INFO:teuthology.orchestra.run.smithi022.stderr: 68100K .......... .......... .......... .......... .......... 40% 15.3M 6s 2024-03-29T04:19:18.779 INFO:teuthology.orchestra.run.smithi022.stderr: 68150K .......... .......... .......... .......... .......... 40% 12.8M 6s 2024-03-29T04:19:18.782 INFO:teuthology.orchestra.run.smithi022.stderr: 68200K .......... .......... .......... .......... .......... 40% 15.2M 6s 2024-03-29T04:19:18.785 INFO:teuthology.orchestra.run.smithi022.stderr: 68250K .......... .......... .......... .......... .......... 40% 14.3M 6s 2024-03-29T04:19:18.789 INFO:teuthology.orchestra.run.smithi022.stderr: 68300K .......... .......... .......... .......... .......... 40% 14.7M 6s 2024-03-29T04:19:18.793 INFO:teuthology.orchestra.run.smithi022.stderr: 68350K .......... .......... .......... .......... .......... 40% 12.1M 6s 2024-03-29T04:19:18.796 INFO:teuthology.orchestra.run.smithi022.stderr: 68400K .......... .......... .......... .......... .......... 40% 14.1M 6s 2024-03-29T04:19:18.800 INFO:teuthology.orchestra.run.smithi022.stderr: 68450K .......... .......... .......... .......... .......... 40% 14.2M 6s 2024-03-29T04:19:18.803 INFO:teuthology.orchestra.run.smithi022.stderr: 68500K .......... .......... .......... .......... .......... 40% 14.3M 6s 2024-03-29T04:19:18.807 INFO:teuthology.orchestra.run.smithi022.stderr: 68550K .......... .......... .......... .......... .......... 40% 11.7M 6s 2024-03-29T04:19:18.811 INFO:teuthology.orchestra.run.smithi022.stderr: 68600K .......... .......... .......... .......... .......... 40% 14.3M 6s 2024-03-29T04:19:18.814 INFO:teuthology.orchestra.run.smithi022.stderr: 68650K .......... .......... .......... .......... .......... 40% 14.5M 6s 2024-03-29T04:19:18.817 INFO:teuthology.orchestra.run.smithi022.stderr: 68700K .......... .......... .......... .......... .......... 40% 15.7M 6s 2024-03-29T04:19:18.821 INFO:teuthology.orchestra.run.smithi022.stderr: 68750K .......... .......... .......... .......... .......... 40% 13.2M 6s 2024-03-29T04:19:18.824 INFO:teuthology.orchestra.run.smithi022.stderr: 68800K .......... .......... .......... .......... .......... 40% 16.2M 6s 2024-03-29T04:19:18.827 INFO:teuthology.orchestra.run.smithi022.stderr: 68850K .......... .......... .......... .......... .......... 40% 14.9M 6s 2024-03-29T04:19:18.830 INFO:teuthology.orchestra.run.smithi022.stderr: 68900K .......... .......... .......... .......... .......... 40% 15.4M 6s 2024-03-29T04:19:18.834 INFO:teuthology.orchestra.run.smithi022.stderr: 68950K .......... .......... .......... .......... .......... 40% 12.7M 6s 2024-03-29T04:19:18.837 INFO:teuthology.orchestra.run.smithi022.stderr: 69000K .......... .......... .......... .......... .......... 40% 14.7M 6s 2024-03-29T04:19:18.841 INFO:teuthology.orchestra.run.smithi022.stderr: 69050K .......... .......... .......... .......... .......... 40% 15.2M 6s 2024-03-29T04:19:18.844 INFO:teuthology.orchestra.run.smithi022.stderr: 69100K .......... .......... .......... .......... .......... 41% 15.9M 6s 2024-03-29T04:19:18.849 INFO:teuthology.orchestra.run.smithi022.stderr: 69150K .......... .......... .......... .......... .......... 41% 9.22M 6s 2024-03-29T04:19:18.853 INFO:teuthology.orchestra.run.smithi022.stderr: 69200K .......... .......... .......... .......... .......... 41% 11.4M 6s 2024-03-29T04:19:18.858 INFO:teuthology.orchestra.run.smithi022.stderr: 69250K .......... .......... .......... .......... .......... 41% 11.0M 6s 2024-03-29T04:19:18.862 INFO:teuthology.orchestra.run.smithi022.stderr: 69300K .......... .......... .......... .......... .......... 41% 12.3M 6s 2024-03-29T04:19:18.866 INFO:teuthology.orchestra.run.smithi022.stderr: 69350K .......... .......... .......... .......... .......... 41% 11.6M 6s 2024-03-29T04:19:18.869 INFO:teuthology.orchestra.run.smithi022.stderr: 69400K .......... .......... .......... .......... .......... 41% 15.1M 6s 2024-03-29T04:19:18.872 INFO:teuthology.orchestra.run.smithi022.stderr: 69450K .......... .......... .......... .......... .......... 41% 16.8M 6s 2024-03-29T04:19:18.875 INFO:teuthology.orchestra.run.smithi022.stderr: 69500K .......... .......... .......... .......... .......... 41% 13.9M 6s 2024-03-29T04:19:18.880 INFO:teuthology.orchestra.run.smithi022.stderr: 69550K .......... .......... .......... .......... .......... 41% 10.8M 6s 2024-03-29T04:19:18.884 INFO:teuthology.orchestra.run.smithi022.stderr: 69600K .......... .......... .......... .......... .......... 41% 11.9M 6s 2024-03-29T04:19:18.888 INFO:teuthology.orchestra.run.smithi022.stderr: 69650K .......... .......... .......... .......... .......... 41% 13.5M 6s 2024-03-29T04:19:18.892 INFO:teuthology.orchestra.run.smithi022.stderr: 69700K .......... .......... .......... .......... .......... 41% 11.8M 6s 2024-03-29T04:19:18.896 INFO:teuthology.orchestra.run.smithi022.stderr: 69750K .......... .......... .......... .......... .......... 41% 11.4M 6s 2024-03-29T04:19:18.900 INFO:teuthology.orchestra.run.smithi022.stderr: 69800K .......... .......... .......... .......... .......... 41% 13.3M 6s 2024-03-29T04:19:18.903 INFO:teuthology.orchestra.run.smithi022.stderr: 69850K .......... .......... .......... .......... .......... 41% 14.0M 6s 2024-03-29T04:19:18.906 INFO:teuthology.orchestra.run.smithi022.stderr: 69900K .......... .......... .......... .......... .......... 41% 17.1M 6s 2024-03-29T04:19:18.909 INFO:teuthology.orchestra.run.smithi022.stderr: 69950K .......... .......... .......... .......... .......... 41% 18.5M 6s 2024-03-29T04:19:18.911 INFO:teuthology.orchestra.run.smithi022.stderr: 70000K .......... .......... .......... .......... .......... 41% 18.2M 6s 2024-03-29T04:19:18.913 INFO:teuthology.orchestra.run.smithi022.stderr: 70050K .......... .......... .......... .......... .......... 41% 23.7M 6s 2024-03-29T04:19:18.915 INFO:teuthology.orchestra.run.smithi022.stderr: 70100K .......... .......... .......... .......... .......... 41% 23.6M 6s 2024-03-29T04:19:18.918 INFO:teuthology.orchestra.run.smithi022.stderr: 70150K .......... .......... .......... .......... .......... 41% 20.7M 6s 2024-03-29T04:19:18.920 INFO:teuthology.orchestra.run.smithi022.stderr: 70200K .......... .......... .......... .......... .......... 41% 23.0M 6s 2024-03-29T04:19:18.922 INFO:teuthology.orchestra.run.smithi022.stderr: 70250K .......... .......... .......... .......... .......... 41% 22.7M 6s 2024-03-29T04:19:18.925 INFO:teuthology.orchestra.run.smithi022.stderr: 70300K .......... .......... .......... .......... .......... 41% 17.6M 6s 2024-03-29T04:19:18.928 INFO:teuthology.orchestra.run.smithi022.stderr: 70350K .......... .......... .......... .......... .......... 41% 14.5M 6s 2024-03-29T04:19:18.931 INFO:teuthology.orchestra.run.smithi022.stderr: 70400K .......... .......... .......... .......... .......... 41% 16.1M 6s 2024-03-29T04:19:18.934 INFO:teuthology.orchestra.run.smithi022.stderr: 70450K .......... .......... .......... .......... .......... 41% 16.1M 6s 2024-03-29T04:19:18.937 INFO:teuthology.orchestra.run.smithi022.stderr: 70500K .......... .......... .......... .......... .......... 41% 17.5M 6s 2024-03-29T04:19:18.940 INFO:teuthology.orchestra.run.smithi022.stderr: 70550K .......... .......... .......... .......... .......... 41% 14.9M 6s 2024-03-29T04:19:18.943 INFO:teuthology.orchestra.run.smithi022.stderr: 70600K .......... .......... .......... .......... .......... 41% 16.4M 6s 2024-03-29T04:19:18.946 INFO:teuthology.orchestra.run.smithi022.stderr: 70650K .......... .......... .......... .......... .......... 41% 17.0M 6s 2024-03-29T04:19:18.949 INFO:teuthology.orchestra.run.smithi022.stderr: 70700K .......... .......... .......... .......... .......... 41% 17.3M 6s 2024-03-29T04:19:18.953 INFO:teuthology.orchestra.run.smithi022.stderr: 70750K .......... .......... .......... .......... .......... 42% 14.0M 6s 2024-03-29T04:19:18.955 INFO:teuthology.orchestra.run.smithi022.stderr: 70800K .......... .......... .......... .......... .......... 42% 19.6M 6s 2024-03-29T04:19:18.957 INFO:teuthology.orchestra.run.smithi022.stderr: 70850K .......... .......... .......... .......... .......... 42% 19.9M 6s 2024-03-29T04:19:18.960 INFO:teuthology.orchestra.run.smithi022.stderr: 70900K .......... .......... .......... .......... .......... 42% 22.1M 6s 2024-03-29T04:19:18.962 INFO:teuthology.orchestra.run.smithi022.stderr: 70950K .......... .......... .......... .......... .......... 42% 18.8M 6s 2024-03-29T04:19:18.964 INFO:teuthology.orchestra.run.smithi022.stderr: 71000K .......... .......... .......... .......... .......... 42% 22.0M 6s 2024-03-29T04:19:18.967 INFO:teuthology.orchestra.run.smithi022.stderr: 71050K .......... .......... .......... .......... .......... 42% 17.5M 6s 2024-03-29T04:19:18.970 INFO:teuthology.orchestra.run.smithi022.stderr: 71100K .......... .......... .......... .......... .......... 42% 16.6M 6s 2024-03-29T04:19:18.974 INFO:teuthology.orchestra.run.smithi022.stderr: 71150K .......... .......... .......... .......... .......... 42% 14.2M 6s 2024-03-29T04:19:18.977 INFO:teuthology.orchestra.run.smithi022.stderr: 71200K .......... .......... .......... .......... .......... 42% 15.8M 6s 2024-03-29T04:19:18.980 INFO:teuthology.orchestra.run.smithi022.stderr: 71250K .......... .......... .......... .......... .......... 42% 16.1M 6s 2024-03-29T04:19:18.983 INFO:teuthology.orchestra.run.smithi022.stderr: 71300K .......... .......... .......... .......... .......... 42% 15.7M 6s 2024-03-29T04:19:18.987 INFO:teuthology.orchestra.run.smithi022.stderr: 71350K .......... .......... .......... .......... .......... 42% 13.4M 6s 2024-03-29T04:19:18.990 INFO:teuthology.orchestra.run.smithi022.stderr: 71400K .......... .......... .......... .......... .......... 42% 15.5M 6s 2024-03-29T04:19:18.993 INFO:teuthology.orchestra.run.smithi022.stderr: 71450K .......... .......... .......... .......... .......... 42% 15.7M 6s 2024-03-29T04:19:18.996 INFO:teuthology.orchestra.run.smithi022.stderr: 71500K .......... .......... .......... .......... .......... 42% 15.5M 6s 2024-03-29T04:19:19.000 INFO:teuthology.orchestra.run.smithi022.stderr: 71550K .......... .......... .......... .......... .......... 42% 13.3M 6s 2024-03-29T04:19:19.003 INFO:teuthology.orchestra.run.smithi022.stderr: 71600K .......... .......... .......... .......... .......... 42% 15.6M 6s 2024-03-29T04:19:19.006 INFO:teuthology.orchestra.run.smithi022.stderr: 71650K .......... .......... .......... .......... .......... 42% 14.3M 6s 2024-03-29T04:19:19.010 INFO:teuthology.orchestra.run.smithi022.stderr: 71700K .......... .......... .......... .......... .......... 42% 15.6M 6s 2024-03-29T04:19:19.013 INFO:teuthology.orchestra.run.smithi022.stderr: 71750K .......... .......... .......... .......... .......... 42% 13.5M 6s 2024-03-29T04:19:19.016 INFO:teuthology.orchestra.run.smithi022.stderr: 71800K .......... .......... .......... .......... .......... 42% 15.6M 6s 2024-03-29T04:19:19.019 INFO:teuthology.orchestra.run.smithi022.stderr: 71850K .......... .......... .......... .......... .......... 42% 15.5M 6s 2024-03-29T04:19:19.023 INFO:teuthology.orchestra.run.smithi022.stderr: 71900K .......... .......... .......... .......... .......... 42% 15.4M 6s 2024-03-29T04:19:19.026 INFO:teuthology.orchestra.run.smithi022.stderr: 71950K .......... .......... .......... .......... .......... 42% 12.9M 6s 2024-03-29T04:19:19.029 INFO:teuthology.orchestra.run.smithi022.stderr: 72000K .......... .......... .......... .......... .......... 42% 15.2M 6s 2024-03-29T04:19:19.033 INFO:teuthology.orchestra.run.smithi022.stderr: 72050K .......... .......... .......... .......... .......... 42% 15.3M 6s 2024-03-29T04:19:19.036 INFO:teuthology.orchestra.run.smithi022.stderr: 72100K .......... .......... .......... .......... .......... 42% 15.4M 6s 2024-03-29T04:19:19.040 INFO:teuthology.orchestra.run.smithi022.stderr: 72150K .......... .......... .......... .......... .......... 42% 13.0M 6s 2024-03-29T04:19:19.043 INFO:teuthology.orchestra.run.smithi022.stderr: 72200K .......... .......... .......... .......... .......... 42% 14.7M 6s 2024-03-29T04:19:19.046 INFO:teuthology.orchestra.run.smithi022.stderr: 72250K .......... .......... .......... .......... .......... 42% 15.1M 6s 2024-03-29T04:19:19.050 INFO:teuthology.orchestra.run.smithi022.stderr: 72300K .......... .......... .......... .......... .......... 42% 12.8M 6s 2024-03-29T04:19:19.054 INFO:teuthology.orchestra.run.smithi022.stderr: 72350K .......... .......... .......... .......... .......... 42% 12.2M 6s 2024-03-29T04:19:19.057 INFO:teuthology.orchestra.run.smithi022.stderr: 72400K .......... .......... .......... .......... .......... 42% 14.8M 6s 2024-03-29T04:19:19.061 INFO:teuthology.orchestra.run.smithi022.stderr: 72450K .......... .......... .......... .......... .......... 43% 14.4M 6s 2024-03-29T04:19:19.067 INFO:teuthology.orchestra.run.smithi022.stderr: 72500K .......... .......... .......... .......... .......... 43% 14.6M 6s 2024-03-29T04:19:19.068 INFO:teuthology.orchestra.run.smithi022.stderr: 72550K .......... .......... .......... .......... .......... 43% 12.5M 6s 2024-03-29T04:19:19.071 INFO:teuthology.orchestra.run.smithi022.stderr: 72600K .......... .......... .......... .......... .......... 43% 14.7M 6s 2024-03-29T04:19:19.075 INFO:teuthology.orchestra.run.smithi022.stderr: 72650K .......... .......... .......... .......... .......... 43% 14.6M 6s 2024-03-29T04:19:19.078 INFO:teuthology.orchestra.run.smithi022.stderr: 72700K .......... .......... .......... .......... .......... 43% 14.8M 6s 2024-03-29T04:19:19.082 INFO:teuthology.orchestra.run.smithi022.stderr: 72750K .......... .......... .......... .......... .......... 43% 12.2M 6s 2024-03-29T04:19:19.085 INFO:teuthology.orchestra.run.smithi022.stderr: 72800K .......... .......... .......... .......... .......... 43% 14.4M 6s 2024-03-29T04:19:19.089 INFO:teuthology.orchestra.run.smithi022.stderr: 72850K .......... .......... .......... .......... .......... 43% 13.5M 6s 2024-03-29T04:19:19.092 INFO:teuthology.orchestra.run.smithi022.stderr: 72900K .......... .......... .......... .......... .......... 43% 14.3M 6s 2024-03-29T04:19:19.096 INFO:teuthology.orchestra.run.smithi022.stderr: 72950K .......... .......... .......... .......... .......... 43% 12.2M 6s 2024-03-29T04:19:19.099 INFO:teuthology.orchestra.run.smithi022.stderr: 73000K .......... .......... .......... .......... .......... 43% 17.7M 6s 2024-03-29T04:19:19.102 INFO:teuthology.orchestra.run.smithi022.stderr: 73050K .......... .......... .......... .......... .......... 43% 17.9M 6s 2024-03-29T04:19:19.105 INFO:teuthology.orchestra.run.smithi022.stderr: 73100K .......... .......... .......... .......... .......... 43% 17.2M 6s 2024-03-29T04:19:19.108 INFO:teuthology.orchestra.run.smithi022.stderr: 73150K .......... .......... .......... .......... .......... 43% 15.2M 6s 2024-03-29T04:19:19.111 INFO:teuthology.orchestra.run.smithi022.stderr: 73200K .......... .......... .......... .......... .......... 43% 17.5M 6s 2024-03-29T04:19:19.113 INFO:teuthology.orchestra.run.smithi022.stderr: 73250K .......... .......... .......... .......... .......... 43% 17.5M 6s 2024-03-29T04:19:19.116 INFO:teuthology.orchestra.run.smithi022.stderr: 73300K .......... .......... .......... .......... .......... 43% 17.8M 6s 2024-03-29T04:19:19.120 INFO:teuthology.orchestra.run.smithi022.stderr: 73350K .......... .......... .......... .......... .......... 43% 14.0M 5s 2024-03-29T04:19:19.125 INFO:teuthology.orchestra.run.smithi022.stderr: 73400K .......... .......... .......... .......... .......... 43% 10.5M 5s 2024-03-29T04:19:19.129 INFO:teuthology.orchestra.run.smithi022.stderr: 73450K .......... .......... .......... .......... .......... 43% 10.7M 5s 2024-03-29T04:19:19.134 INFO:teuthology.orchestra.run.smithi022.stderr: 73500K .......... .......... .......... .......... .......... 43% 10.3M 5s 2024-03-29T04:19:19.139 INFO:teuthology.orchestra.run.smithi022.stderr: 73550K .......... .......... .......... .......... .......... 43% 8.84M 5s 2024-03-29T04:19:19.144 INFO:teuthology.orchestra.run.smithi022.stderr: 73600K .......... .......... .......... .......... .......... 43% 10.1M 5s 2024-03-29T04:19:19.149 INFO:teuthology.orchestra.run.smithi022.stderr: 73650K .......... .......... .......... .......... .......... 43% 10.5M 5s 2024-03-29T04:19:19.153 INFO:teuthology.orchestra.run.smithi022.stderr: 73700K .......... .......... .......... .......... .......... 43% 11.1M 5s 2024-03-29T04:19:19.156 INFO:teuthology.orchestra.run.smithi022.stderr: 73750K .......... .......... .......... .......... .......... 43% 15.2M 5s 2024-03-29T04:19:19.160 INFO:teuthology.orchestra.run.smithi022.stderr: 73800K .......... .......... .......... .......... .......... 43% 12.8M 5s 2024-03-29T04:19:19.162 INFO:teuthology.orchestra.run.smithi022.stderr: 73850K .......... .......... .......... .......... .......... 43% 21.1M 5s 2024-03-29T04:19:19.166 INFO:teuthology.orchestra.run.smithi022.stderr: 73900K .......... .......... .......... .......... .......... 43% 14.5M 5s 2024-03-29T04:19:19.170 INFO:teuthology.orchestra.run.smithi022.stderr: 73950K .......... .......... .......... .......... .......... 43% 12.9M 5s 2024-03-29T04:19:19.173 INFO:teuthology.orchestra.run.smithi022.stderr: 74000K .......... .......... .......... .......... .......... 43% 14.5M 5s 2024-03-29T04:19:19.176 INFO:teuthology.orchestra.run.smithi022.stderr: 74050K .......... .......... .......... .......... .......... 43% 16.2M 5s 2024-03-29T04:19:19.179 INFO:teuthology.orchestra.run.smithi022.stderr: 74100K .......... .......... .......... .......... .......... 43% 15.9M 5s 2024-03-29T04:19:19.183 INFO:teuthology.orchestra.run.smithi022.stderr: 74150K .......... .......... .......... .......... .......... 44% 13.3M 5s 2024-03-29T04:19:19.186 INFO:teuthology.orchestra.run.smithi022.stderr: 74200K .......... .......... .......... .......... .......... 44% 16.1M 5s 2024-03-29T04:19:19.188 INFO:teuthology.orchestra.run.smithi022.stderr: 74250K .......... .......... .......... .......... .......... 44% 17.5M 5s 2024-03-29T04:19:19.191 INFO:teuthology.orchestra.run.smithi022.stderr: 74300K .......... .......... .......... .......... .......... 44% 17.5M 5s 2024-03-29T04:19:19.194 INFO:teuthology.orchestra.run.smithi022.stderr: 74350K .......... .......... .......... .......... .......... 44% 15.4M 5s 2024-03-29T04:19:19.197 INFO:teuthology.orchestra.run.smithi022.stderr: 74400K .......... .......... .......... .......... .......... 44% 17.9M 5s 2024-03-29T04:19:19.200 INFO:teuthology.orchestra.run.smithi022.stderr: 74450K .......... .......... .......... .......... .......... 44% 17.3M 5s 2024-03-29T04:19:19.203 INFO:teuthology.orchestra.run.smithi022.stderr: 74500K .......... .......... .......... .......... .......... 44% 17.1M 5s 2024-03-29T04:19:19.206 INFO:teuthology.orchestra.run.smithi022.stderr: 74550K .......... .......... .......... .......... .......... 44% 14.0M 5s 2024-03-29T04:19:19.209 INFO:teuthology.orchestra.run.smithi022.stderr: 74600K .......... .......... .......... .......... .......... 44% 18.3M 5s 2024-03-29T04:19:19.212 INFO:teuthology.orchestra.run.smithi022.stderr: 74650K .......... .......... .......... .......... .......... 44% 18.6M 5s 2024-03-29T04:19:19.215 INFO:teuthology.orchestra.run.smithi022.stderr: 74700K .......... .......... .......... .......... .......... 44% 18.6M 5s 2024-03-29T04:19:19.217 INFO:teuthology.orchestra.run.smithi022.stderr: 74750K .......... .......... .......... .......... .......... 44% 16.3M 5s 2024-03-29T04:19:19.220 INFO:teuthology.orchestra.run.smithi022.stderr: 74800K .......... .......... .......... .......... .......... 44% 19.2M 5s 2024-03-29T04:19:19.222 INFO:teuthology.orchestra.run.smithi022.stderr: 74850K .......... .......... .......... .......... .......... 44% 19.4M 5s 2024-03-29T04:19:19.225 INFO:teuthology.orchestra.run.smithi022.stderr: 74900K .......... .......... .......... .......... .......... 44% 18.1M 5s 2024-03-29T04:19:19.228 INFO:teuthology.orchestra.run.smithi022.stderr: 74950K .......... .......... .......... .......... .......... 44% 14.7M 5s 2024-03-29T04:19:19.231 INFO:teuthology.orchestra.run.smithi022.stderr: 75000K .......... .......... .......... .......... .......... 44% 16.9M 5s 2024-03-29T04:19:19.235 INFO:teuthology.orchestra.run.smithi022.stderr: 75050K .......... .......... .......... .......... .......... 44% 14.2M 5s 2024-03-29T04:19:19.238 INFO:teuthology.orchestra.run.smithi022.stderr: 75100K .......... .......... .......... .......... .......... 44% 16.7M 5s 2024-03-29T04:19:19.241 INFO:teuthology.orchestra.run.smithi022.stderr: 75150K .......... .......... .......... .......... .......... 44% 13.8M 5s 2024-03-29T04:19:19.244 INFO:teuthology.orchestra.run.smithi022.stderr: 75200K .......... .......... .......... .......... .......... 44% 15.6M 5s 2024-03-29T04:19:19.247 INFO:teuthology.orchestra.run.smithi022.stderr: 75250K .......... .......... .......... .......... .......... 44% 15.6M 5s 2024-03-29T04:19:19.251 INFO:teuthology.orchestra.run.smithi022.stderr: 75300K .......... .......... .......... .......... .......... 44% 15.9M 5s 2024-03-29T04:19:19.254 INFO:teuthology.orchestra.run.smithi022.stderr: 75350K .......... .......... .......... .......... .......... 44% 13.2M 5s 2024-03-29T04:19:19.257 INFO:teuthology.orchestra.run.smithi022.stderr: 75400K .......... .......... .......... .......... .......... 44% 18.8M 5s 2024-03-29T04:19:19.260 INFO:teuthology.orchestra.run.smithi022.stderr: 75450K .......... .......... .......... .......... .......... 44% 16.8M 5s 2024-03-29T04:19:19.263 INFO:teuthology.orchestra.run.smithi022.stderr: 75500K .......... .......... .......... .......... .......... 44% 15.5M 5s 2024-03-29T04:19:19.266 INFO:teuthology.orchestra.run.smithi022.stderr: 75550K .......... .......... .......... .......... .......... 44% 14.4M 5s 2024-03-29T04:19:19.271 INFO:teuthology.orchestra.run.smithi022.stderr: 75600K .......... .......... .......... .......... .......... 44% 16.9M 5s 2024-03-29T04:19:19.272 INFO:teuthology.orchestra.run.smithi022.stderr: 75650K .......... .......... .......... .......... .......... 44% 15.2M 5s 2024-03-29T04:19:19.275 INFO:teuthology.orchestra.run.smithi022.stderr: 75700K .......... .......... .......... .......... .......... 44% 16.1M 5s 2024-03-29T04:19:19.279 INFO:teuthology.orchestra.run.smithi022.stderr: 75750K .......... .......... .......... .......... .......... 44% 13.9M 5s 2024-03-29T04:19:19.282 INFO:teuthology.orchestra.run.smithi022.stderr: 75800K .......... .......... .......... .......... .......... 44% 14.7M 5s 2024-03-29T04:19:19.286 INFO:teuthology.orchestra.run.smithi022.stderr: 75850K .......... .......... .......... .......... .......... 45% 14.5M 5s 2024-03-29T04:19:19.289 INFO:teuthology.orchestra.run.smithi022.stderr: 75900K .......... .......... .......... .......... .......... 45% 14.9M 5s 2024-03-29T04:19:19.292 INFO:teuthology.orchestra.run.smithi022.stderr: 75950K .......... .......... .......... .......... .......... 45% 13.5M 5s 2024-03-29T04:19:19.295 INFO:teuthology.orchestra.run.smithi022.stderr: 76000K .......... .......... .......... .......... .......... 45% 16.1M 5s 2024-03-29T04:19:19.298 INFO:teuthology.orchestra.run.smithi022.stderr: 76050K .......... .......... .......... .......... .......... 45% 16.4M 5s 2024-03-29T04:19:19.301 INFO:teuthology.orchestra.run.smithi022.stderr: 76100K .......... .......... .......... .......... .......... 45% 16.3M 5s 2024-03-29T04:19:19.305 INFO:teuthology.orchestra.run.smithi022.stderr: 76150K .......... .......... .......... .......... .......... 45% 13.7M 5s 2024-03-29T04:19:19.308 INFO:teuthology.orchestra.run.smithi022.stderr: 76200K .......... .......... .......... .......... .......... 45% 14.2M 5s 2024-03-29T04:19:19.312 INFO:teuthology.orchestra.run.smithi022.stderr: 76250K .......... .......... .......... .......... .......... 45% 15.7M 5s 2024-03-29T04:19:19.315 INFO:teuthology.orchestra.run.smithi022.stderr: 76300K .......... .......... .......... .......... .......... 45% 14.4M 5s 2024-03-29T04:19:19.319 INFO:teuthology.orchestra.run.smithi022.stderr: 76350K .......... .......... .......... .......... .......... 45% 12.3M 5s 2024-03-29T04:19:19.322 INFO:teuthology.orchestra.run.smithi022.stderr: 76400K .......... .......... .......... .......... .......... 45% 14.6M 5s 2024-03-29T04:19:19.325 INFO:teuthology.orchestra.run.smithi022.stderr: 76450K .......... .......... .......... .......... .......... 45% 14.8M 5s 2024-03-29T04:19:19.329 INFO:teuthology.orchestra.run.smithi022.stderr: 76500K .......... .......... .......... .......... .......... 45% 15.1M 5s 2024-03-29T04:19:19.333 INFO:teuthology.orchestra.run.smithi022.stderr: 76550K .......... .......... .......... .......... .......... 45% 12.7M 5s 2024-03-29T04:19:19.336 INFO:teuthology.orchestra.run.smithi022.stderr: 76600K .......... .......... .......... .......... .......... 45% 14.3M 5s 2024-03-29T04:19:19.339 INFO:teuthology.orchestra.run.smithi022.stderr: 76650K .......... .......... .......... .......... .......... 45% 14.9M 5s 2024-03-29T04:19:19.342 INFO:teuthology.orchestra.run.smithi022.stderr: 76700K .......... .......... .......... .......... .......... 45% 15.6M 5s 2024-03-29T04:19:19.347 INFO:teuthology.orchestra.run.smithi022.stderr: 76750K .......... .......... .......... .......... .......... 45% 12.6M 5s 2024-03-29T04:19:19.350 INFO:teuthology.orchestra.run.smithi022.stderr: 76800K .......... .......... .......... .......... .......... 45% 13.2M 5s 2024-03-29T04:19:19.354 INFO:teuthology.orchestra.run.smithi022.stderr: 76850K .......... .......... .......... .......... .......... 45% 12.3M 5s 2024-03-29T04:19:19.358 INFO:teuthology.orchestra.run.smithi022.stderr: 76900K .......... .......... .......... .......... .......... 45% 12.5M 5s 2024-03-29T04:19:19.362 INFO:teuthology.orchestra.run.smithi022.stderr: 76950K .......... .......... .......... .......... .......... 45% 11.3M 5s 2024-03-29T04:19:19.366 INFO:teuthology.orchestra.run.smithi022.stderr: 77000K .......... .......... .......... .......... .......... 45% 13.6M 5s 2024-03-29T04:19:19.370 INFO:teuthology.orchestra.run.smithi022.stderr: 77050K .......... .......... .......... .......... .......... 45% 12.4M 5s 2024-03-29T04:19:19.374 INFO:teuthology.orchestra.run.smithi022.stderr: 77100K .......... .......... .......... .......... .......... 45% 11.9M 5s 2024-03-29T04:19:19.379 INFO:teuthology.orchestra.run.smithi022.stderr: 77150K .......... .......... .......... .......... .......... 45% 10.8M 5s 2024-03-29T04:19:19.382 INFO:teuthology.orchestra.run.smithi022.stderr: 77200K .......... .......... .......... .......... .......... 45% 13.1M 5s 2024-03-29T04:19:19.386 INFO:teuthology.orchestra.run.smithi022.stderr: 77250K .......... .......... .......... .......... .......... 45% 13.3M 5s 2024-03-29T04:19:19.390 INFO:teuthology.orchestra.run.smithi022.stderr: 77300K .......... .......... .......... .......... .......... 45% 13.0M 5s 2024-03-29T04:19:19.394 INFO:teuthology.orchestra.run.smithi022.stderr: 77350K .......... .......... .......... .......... .......... 45% 11.4M 5s 2024-03-29T04:19:19.398 INFO:teuthology.orchestra.run.smithi022.stderr: 77400K .......... .......... .......... .......... .......... 45% 13.2M 5s 2024-03-29T04:19:19.401 INFO:teuthology.orchestra.run.smithi022.stderr: 77450K .......... .......... .......... .......... .......... 45% 13.7M 5s 2024-03-29T04:19:19.404 INFO:teuthology.orchestra.run.smithi022.stderr: 77500K .......... .......... .......... .......... .......... 46% 17.0M 5s 2024-03-29T04:19:19.408 INFO:teuthology.orchestra.run.smithi022.stderr: 77550K .......... .......... .......... .......... .......... 46% 11.8M 5s 2024-03-29T04:19:19.412 INFO:teuthology.orchestra.run.smithi022.stderr: 77600K .......... .......... .......... .......... .......... 46% 13.1M 5s 2024-03-29T04:19:19.415 INFO:teuthology.orchestra.run.smithi022.stderr: 77650K .......... .......... .......... .......... .......... 46% 16.3M 5s 2024-03-29T04:19:19.418 INFO:teuthology.orchestra.run.smithi022.stderr: 77700K .......... .......... .......... .......... .......... 46% 14.0M 5s 2024-03-29T04:19:19.423 INFO:teuthology.orchestra.run.smithi022.stderr: 77750K .......... .......... .......... .......... .......... 46% 11.2M 5s 2024-03-29T04:19:19.426 INFO:teuthology.orchestra.run.smithi022.stderr: 77800K .......... .......... .......... .......... .......... 46% 13.7M 5s 2024-03-29T04:19:19.429 INFO:teuthology.orchestra.run.smithi022.stderr: 77850K .......... .......... .......... .......... .......... 46% 18.4M 5s 2024-03-29T04:19:19.431 INFO:teuthology.orchestra.run.smithi022.stderr: 77900K .......... .......... .......... .......... .......... 46% 19.1M 5s 2024-03-29T04:19:19.434 INFO:teuthology.orchestra.run.smithi022.stderr: 77950K .......... .......... .......... .......... .......... 46% 17.7M 5s 2024-03-29T04:19:19.437 INFO:teuthology.orchestra.run.smithi022.stderr: 78000K .......... .......... .......... .......... .......... 46% 19.3M 5s 2024-03-29T04:19:19.439 INFO:teuthology.orchestra.run.smithi022.stderr: 78050K .......... .......... .......... .......... .......... 46% 19.1M 5s 2024-03-29T04:19:19.442 INFO:teuthology.orchestra.run.smithi022.stderr: 78100K .......... .......... .......... .......... .......... 46% 19.4M 5s 2024-03-29T04:19:19.444 INFO:teuthology.orchestra.run.smithi022.stderr: 78150K .......... .......... .......... .......... .......... 46% 17.9M 5s 2024-03-29T04:19:19.447 INFO:teuthology.orchestra.run.smithi022.stderr: 78200K .......... .......... .......... .......... .......... 46% 24.0M 5s 2024-03-29T04:19:19.448 INFO:teuthology.orchestra.run.smithi022.stderr: 78250K .......... .......... .......... .......... .......... 46% 27.2M 5s 2024-03-29T04:19:19.450 INFO:teuthology.orchestra.run.smithi022.stderr: 78300K .......... .......... .......... .......... .......... 46% 29.3M 5s 2024-03-29T04:19:19.452 INFO:teuthology.orchestra.run.smithi022.stderr: 78350K .......... .......... .......... .......... .......... 46% 24.9M 5s 2024-03-29T04:19:19.453 INFO:teuthology.orchestra.run.smithi022.stderr: 78400K .......... .......... .......... .......... .......... 46% 28.9M 5s 2024-03-29T04:19:19.455 INFO:teuthology.orchestra.run.smithi022.stderr: 78450K .......... .......... .......... .......... .......... 46% 28.7M 5s 2024-03-29T04:19:19.457 INFO:teuthology.orchestra.run.smithi022.stderr: 78500K .......... .......... .......... .......... .......... 46% 28.4M 5s 2024-03-29T04:19:19.459 INFO:teuthology.orchestra.run.smithi022.stderr: 78550K .......... .......... .......... .......... .......... 46% 24.9M 5s 2024-03-29T04:19:19.460 INFO:teuthology.orchestra.run.smithi022.stderr: 78600K .......... .......... .......... .......... .......... 46% 28.2M 5s 2024-03-29T04:19:19.463 INFO:teuthology.orchestra.run.smithi022.stderr: 78650K .......... .......... .......... .......... .......... 46% 28.5M 5s 2024-03-29T04:19:19.464 INFO:teuthology.orchestra.run.smithi022.stderr: 78700K .......... .......... .......... .......... .......... 46% 28.5M 5s 2024-03-29T04:19:19.466 INFO:teuthology.orchestra.run.smithi022.stderr: 78750K .......... .......... .......... .......... .......... 46% 24.5M 5s 2024-03-29T04:19:19.468 INFO:teuthology.orchestra.run.smithi022.stderr: 78800K .......... .......... .......... .......... .......... 46% 27.8M 5s 2024-03-29T04:19:19.469 INFO:teuthology.orchestra.run.smithi022.stderr: 78850K .......... .......... .......... .......... .......... 46% 28.2M 5s 2024-03-29T04:19:19.471 INFO:teuthology.orchestra.run.smithi022.stderr: 78900K .......... .......... .......... .......... .......... 46% 26.8M 5s 2024-03-29T04:19:19.473 INFO:teuthology.orchestra.run.smithi022.stderr: 78950K .......... .......... .......... .......... .......... 46% 23.5M 5s 2024-03-29T04:19:19.475 INFO:teuthology.orchestra.run.smithi022.stderr: 79000K .......... .......... .......... .......... .......... 46% 27.0M 5s 2024-03-29T04:19:19.477 INFO:teuthology.orchestra.run.smithi022.stderr: 79050K .......... .......... .......... .......... .......... 46% 26.4M 5s 2024-03-29T04:19:19.479 INFO:teuthology.orchestra.run.smithi022.stderr: 79100K .......... .......... .......... .......... .......... 46% 25.5M 5s 2024-03-29T04:19:19.481 INFO:teuthology.orchestra.run.smithi022.stderr: 79150K .......... .......... .......... .......... .......... 46% 22.7M 5s 2024-03-29T04:19:19.483 INFO:teuthology.orchestra.run.smithi022.stderr: 79200K .......... .......... .......... .......... .......... 47% 25.7M 5s 2024-03-29T04:19:19.485 INFO:teuthology.orchestra.run.smithi022.stderr: 79250K .......... .......... .......... .......... .......... 47% 24.7M 5s 2024-03-29T04:19:19.487 INFO:teuthology.orchestra.run.smithi022.stderr: 79300K .......... .......... .......... .......... .......... 47% 24.3M 5s 2024-03-29T04:19:19.489 INFO:teuthology.orchestra.run.smithi022.stderr: 79350K .......... .......... .......... .......... .......... 47% 21.1M 5s 2024-03-29T04:19:19.491 INFO:teuthology.orchestra.run.smithi022.stderr: 79400K .......... .......... .......... .......... .......... 47% 23.0M 5s 2024-03-29T04:19:19.494 INFO:teuthology.orchestra.run.smithi022.stderr: 79450K .......... .......... .......... .......... .......... 47% 23.8M 5s 2024-03-29T04:19:19.496 INFO:teuthology.orchestra.run.smithi022.stderr: 79500K .......... .......... .......... .......... .......... 47% 23.8M 5s 2024-03-29T04:19:19.498 INFO:teuthology.orchestra.run.smithi022.stderr: 79550K .......... .......... .......... .......... .......... 47% 20.2M 5s 2024-03-29T04:19:19.500 INFO:teuthology.orchestra.run.smithi022.stderr: 79600K .......... .......... .......... .......... .......... 47% 22.9M 5s 2024-03-29T04:19:19.503 INFO:teuthology.orchestra.run.smithi022.stderr: 79650K .......... .......... .......... .......... .......... 47% 22.1M 5s 2024-03-29T04:19:19.504 INFO:teuthology.orchestra.run.smithi022.stderr: 79700K .......... .......... .......... .......... .......... 47% 22.6M 5s 2024-03-29T04:19:19.507 INFO:teuthology.orchestra.run.smithi022.stderr: 79750K .......... .......... .......... .......... .......... 47% 19.5M 5s 2024-03-29T04:19:19.509 INFO:teuthology.orchestra.run.smithi022.stderr: 79800K .......... .......... .......... .......... .......... 47% 23.1M 5s 2024-03-29T04:19:19.511 INFO:teuthology.orchestra.run.smithi022.stderr: 79850K .......... .......... .......... .......... .......... 47% 22.5M 5s 2024-03-29T04:19:19.514 INFO:teuthology.orchestra.run.smithi022.stderr: 79900K .......... .......... .......... .......... .......... 47% 21.8M 5s 2024-03-29T04:19:19.516 INFO:teuthology.orchestra.run.smithi022.stderr: 79950K .......... .......... .......... .......... .......... 47% 19.2M 5s 2024-03-29T04:19:19.518 INFO:teuthology.orchestra.run.smithi022.stderr: 80000K .......... .......... .......... .......... .......... 47% 22.7M 5s 2024-03-29T04:19:19.521 INFO:teuthology.orchestra.run.smithi022.stderr: 80050K .......... .......... .......... .......... .......... 47% 21.8M 5s 2024-03-29T04:19:19.523 INFO:teuthology.orchestra.run.smithi022.stderr: 80100K .......... .......... .......... .......... .......... 47% 21.8M 5s 2024-03-29T04:19:19.525 INFO:teuthology.orchestra.run.smithi022.stderr: 80150K .......... .......... .......... .......... .......... 47% 18.7M 5s 2024-03-29T04:19:19.528 INFO:teuthology.orchestra.run.smithi022.stderr: 80200K .......... .......... .......... .......... .......... 47% 22.0M 5s 2024-03-29T04:19:19.530 INFO:teuthology.orchestra.run.smithi022.stderr: 80250K .......... .......... .......... .......... .......... 47% 22.5M 5s 2024-03-29T04:19:19.532 INFO:teuthology.orchestra.run.smithi022.stderr: 80300K .......... .......... .......... .......... .......... 47% 21.4M 5s 2024-03-29T04:19:19.535 INFO:teuthology.orchestra.run.smithi022.stderr: 80350K .......... .......... .......... .......... .......... 47% 18.3M 5s 2024-03-29T04:19:19.537 INFO:teuthology.orchestra.run.smithi022.stderr: 80400K .......... .......... .......... .......... .......... 47% 21.2M 5s 2024-03-29T04:19:19.539 INFO:teuthology.orchestra.run.smithi022.stderr: 80450K .......... .......... .......... .......... .......... 47% 21.3M 5s 2024-03-29T04:19:19.542 INFO:teuthology.orchestra.run.smithi022.stderr: 80500K .......... .......... .......... .......... .......... 47% 21.1M 5s 2024-03-29T04:19:19.544 INFO:teuthology.orchestra.run.smithi022.stderr: 80550K .......... .......... .......... .......... .......... 47% 17.8M 5s 2024-03-29T04:19:19.546 INFO:teuthology.orchestra.run.smithi022.stderr: 80600K .......... .......... .......... .......... .......... 47% 21.7M 5s 2024-03-29T04:19:19.549 INFO:teuthology.orchestra.run.smithi022.stderr: 80650K .......... .......... .......... .......... .......... 47% 20.5M 5s 2024-03-29T04:19:19.551 INFO:teuthology.orchestra.run.smithi022.stderr: 80700K .......... .......... .......... .......... .......... 47% 22.9M 5s 2024-03-29T04:19:19.554 INFO:teuthology.orchestra.run.smithi022.stderr: 80750K .......... .......... .......... .......... .......... 47% 18.9M 5s 2024-03-29T04:19:19.556 INFO:teuthology.orchestra.run.smithi022.stderr: 80800K .......... .......... .......... .......... .......... 47% 23.5M 5s 2024-03-29T04:19:19.558 INFO:teuthology.orchestra.run.smithi022.stderr: 80850K .......... .......... .......... .......... .......... 47% 22.0M 5s 2024-03-29T04:19:19.560 INFO:teuthology.orchestra.run.smithi022.stderr: 80900K .......... .......... .......... .......... .......... 48% 23.8M 5s 2024-03-29T04:19:19.562 INFO:teuthology.orchestra.run.smithi022.stderr: 80950K .......... .......... .......... .......... .......... 48% 22.4M 5s 2024-03-29T04:19:19.564 INFO:teuthology.orchestra.run.smithi022.stderr: 81000K .......... .......... .......... .......... .......... 48% 23.9M 5s 2024-03-29T04:19:19.566 INFO:teuthology.orchestra.run.smithi022.stderr: 81050K .......... .......... .......... .......... .......... 48% 22.8M 5s 2024-03-29T04:19:19.568 INFO:teuthology.orchestra.run.smithi022.stderr: 81100K .......... .......... .......... .......... .......... 48% 22.9M 5s 2024-03-29T04:19:19.571 INFO:teuthology.orchestra.run.smithi022.stderr: 81150K .......... .......... .......... .......... .......... 48% 19.7M 5s 2024-03-29T04:19:19.573 INFO:teuthology.orchestra.run.smithi022.stderr: 81200K .......... .......... .......... .......... .......... 48% 22.7M 5s 2024-03-29T04:19:19.576 INFO:teuthology.orchestra.run.smithi022.stderr: 81250K .......... .......... .......... .......... .......... 48% 20.8M 5s 2024-03-29T04:19:19.578 INFO:teuthology.orchestra.run.smithi022.stderr: 81300K .......... .......... .......... .......... .......... 48% 22.3M 5s 2024-03-29T04:19:19.580 INFO:teuthology.orchestra.run.smithi022.stderr: 81350K .......... .......... .......... .......... .......... 48% 18.7M 5s 2024-03-29T04:19:19.582 INFO:teuthology.orchestra.run.smithi022.stderr: 81400K .......... .......... .......... .......... .......... 48% 21.8M 5s 2024-03-29T04:19:19.585 INFO:teuthology.orchestra.run.smithi022.stderr: 81450K .......... .......... .......... .......... .......... 48% 21.7M 5s 2024-03-29T04:19:19.587 INFO:teuthology.orchestra.run.smithi022.stderr: 81500K .......... .......... .......... .......... .......... 48% 20.9M 5s 2024-03-29T04:19:19.590 INFO:teuthology.orchestra.run.smithi022.stderr: 81550K .......... .......... .......... .......... .......... 48% 18.7M 5s 2024-03-29T04:19:19.593 INFO:teuthology.orchestra.run.smithi022.stderr: 81600K .......... .......... .......... .......... .......... 48% 17.7M 5s 2024-03-29T04:19:19.596 INFO:teuthology.orchestra.run.smithi022.stderr: 81650K .......... .......... .......... .......... .......... 48% 16.2M 5s 2024-03-29T04:19:19.599 INFO:teuthology.orchestra.run.smithi022.stderr: 81700K .......... .......... .......... .......... .......... 48% 14.6M 5s 2024-03-29T04:19:19.603 INFO:teuthology.orchestra.run.smithi022.stderr: 81750K .......... .......... .......... .......... .......... 48% 13.5M 5s 2024-03-29T04:19:19.606 INFO:teuthology.orchestra.run.smithi022.stderr: 81800K .......... .......... .......... .......... .......... 48% 15.0M 5s 2024-03-29T04:19:19.609 INFO:teuthology.orchestra.run.smithi022.stderr: 81850K .......... .......... .......... .......... .......... 48% 14.4M 5s 2024-03-29T04:19:19.612 INFO:teuthology.orchestra.run.smithi022.stderr: 81900K .......... .......... .......... .......... .......... 48% 16.3M 5s 2024-03-29T04:19:19.615 INFO:teuthology.orchestra.run.smithi022.stderr: 81950K .......... .......... .......... .......... .......... 48% 21.1M 5s 2024-03-29T04:19:19.617 INFO:teuthology.orchestra.run.smithi022.stderr: 82000K .......... .......... .......... .......... .......... 48% 23.5M 5s 2024-03-29T04:19:19.618 INFO:teuthology.orchestra.run.smithi022.stderr: 82050K .......... .......... .......... .......... .......... 48% 24.3M 5s 2024-03-29T04:19:19.620 INFO:teuthology.orchestra.run.smithi022.stderr: 82100K .......... .......... .......... .......... .......... 48% 24.3M 5s 2024-03-29T04:19:19.623 INFO:teuthology.orchestra.run.smithi022.stderr: 82150K .......... .......... .......... .......... .......... 48% 19.8M 5s 2024-03-29T04:19:19.625 INFO:teuthology.orchestra.run.smithi022.stderr: 82200K .......... .......... .......... .......... .......... 48% 20.2M 5s 2024-03-29T04:19:19.628 INFO:teuthology.orchestra.run.smithi022.stderr: 82250K .......... .......... .......... .......... .......... 48% 21.1M 5s 2024-03-29T04:19:19.630 INFO:teuthology.orchestra.run.smithi022.stderr: 82300K .......... .......... .......... .......... .......... 48% 21.3M 5s 2024-03-29T04:19:19.632 INFO:teuthology.orchestra.run.smithi022.stderr: 82350K .......... .......... .......... .......... .......... 48% 22.0M 5s 2024-03-29T04:19:19.634 INFO:teuthology.orchestra.run.smithi022.stderr: 82400K .......... .......... .......... .......... .......... 48% 28.0M 5s 2024-03-29T04:19:19.636 INFO:teuthology.orchestra.run.smithi022.stderr: 82450K .......... .......... .......... .......... .......... 48% 26.8M 5s 2024-03-29T04:19:19.637 INFO:teuthology.orchestra.run.smithi022.stderr: 82500K .......... .......... .......... .......... .......... 48% 29.5M 5s 2024-03-29T04:19:19.639 INFO:teuthology.orchestra.run.smithi022.stderr: 82550K .......... .......... .......... .......... .......... 49% 24.9M 5s 2024-03-29T04:19:19.641 INFO:teuthology.orchestra.run.smithi022.stderr: 82600K .......... .......... .......... .......... .......... 49% 28.9M 5s 2024-03-29T04:19:19.643 INFO:teuthology.orchestra.run.smithi022.stderr: 82650K .......... .......... .......... .......... .......... 49% 28.0M 5s 2024-03-29T04:19:19.644 INFO:teuthology.orchestra.run.smithi022.stderr: 82700K .......... .......... .......... .......... .......... 49% 28.4M 5s 2024-03-29T04:19:19.646 INFO:teuthology.orchestra.run.smithi022.stderr: 82750K .......... .......... .......... .......... .......... 49% 24.7M 5s 2024-03-29T04:19:19.648 INFO:teuthology.orchestra.run.smithi022.stderr: 82800K .......... .......... .......... .......... .......... 49% 25.1M 5s 2024-03-29T04:19:19.650 INFO:teuthology.orchestra.run.smithi022.stderr: 82850K .......... .......... .......... .......... .......... 49% 23.8M 5s 2024-03-29T04:19:19.653 INFO:teuthology.orchestra.run.smithi022.stderr: 82900K .......... .......... .......... .......... .......... 49% 21.7M 5s 2024-03-29T04:19:19.656 INFO:teuthology.orchestra.run.smithi022.stderr: 82950K .......... .......... .......... .......... .......... 49% 17.8M 5s 2024-03-29T04:19:19.658 INFO:teuthology.orchestra.run.smithi022.stderr: 83000K .......... .......... .......... .......... .......... 49% 18.7M 5s 2024-03-29T04:19:19.661 INFO:teuthology.orchestra.run.smithi022.stderr: 83050K .......... .......... .......... .......... .......... 49% 20.4M 5s 2024-03-29T04:19:19.663 INFO:teuthology.orchestra.run.smithi022.stderr: 83100K .......... .......... .......... .......... .......... 49% 20.2M 5s 2024-03-29T04:19:19.666 INFO:teuthology.orchestra.run.smithi022.stderr: 83150K .......... .......... .......... .......... .......... 49% 17.7M 5s 2024-03-29T04:19:19.668 INFO:teuthology.orchestra.run.smithi022.stderr: 83200K .......... .......... .......... .......... .......... 49% 20.2M 5s 2024-03-29T04:19:19.671 INFO:teuthology.orchestra.run.smithi022.stderr: 83250K .......... .......... .......... .......... .......... 49% 19.8M 5s 2024-03-29T04:19:19.673 INFO:teuthology.orchestra.run.smithi022.stderr: 83300K .......... .......... .......... .......... .......... 49% 19.9M 5s 2024-03-29T04:19:19.678 INFO:teuthology.orchestra.run.smithi022.stderr: 83350K .......... .......... .......... .......... .......... 49% 16.8M 5s 2024-03-29T04:19:19.679 INFO:teuthology.orchestra.run.smithi022.stderr: 83400K .......... .......... .......... .......... .......... 49% 17.3M 5s 2024-03-29T04:19:19.681 INFO:teuthology.orchestra.run.smithi022.stderr: 83450K .......... .......... .......... .......... .......... 49% 19.2M 5s 2024-03-29T04:19:19.684 INFO:teuthology.orchestra.run.smithi022.stderr: 83500K .......... .......... .......... .......... .......... 49% 19.5M 5s 2024-03-29T04:19:19.687 INFO:teuthology.orchestra.run.smithi022.stderr: 83550K .......... .......... .......... .......... .......... 49% 16.8M 5s 2024-03-29T04:19:19.689 INFO:teuthology.orchestra.run.smithi022.stderr: 83600K .......... .......... .......... .......... .......... 49% 18.7M 5s 2024-03-29T04:19:19.691 INFO:teuthology.orchestra.run.smithi022.stderr: 83650K .......... .......... .......... .......... .......... 49% 22.2M 5s 2024-03-29T04:19:19.693 INFO:teuthology.orchestra.run.smithi022.stderr: 83700K .......... .......... .......... .......... .......... 49% 25.0M 5s 2024-03-29T04:19:19.696 INFO:teuthology.orchestra.run.smithi022.stderr: 83750K .......... .......... .......... .......... .......... 49% 20.8M 5s 2024-03-29T04:19:19.698 INFO:teuthology.orchestra.run.smithi022.stderr: 83800K .......... .......... .......... .......... .......... 49% 22.6M 5s 2024-03-29T04:19:19.700 INFO:teuthology.orchestra.run.smithi022.stderr: 83850K .......... .......... .......... .......... .......... 49% 24.3M 5s 2024-03-29T04:19:19.702 INFO:teuthology.orchestra.run.smithi022.stderr: 83900K .......... .......... .......... .......... .......... 49% 23.9M 5s 2024-03-29T04:19:19.704 INFO:teuthology.orchestra.run.smithi022.stderr: 83950K .......... .......... .......... .......... .......... 49% 20.5M 5s 2024-03-29T04:19:19.707 INFO:teuthology.orchestra.run.smithi022.stderr: 84000K .......... .......... .......... .......... .......... 49% 20.5M 5s 2024-03-29T04:19:19.709 INFO:teuthology.orchestra.run.smithi022.stderr: 84050K .......... .......... .......... .......... .......... 49% 23.4M 5s 2024-03-29T04:19:19.711 INFO:teuthology.orchestra.run.smithi022.stderr: 84100K .......... .......... .......... .......... .......... 49% 28.1M 5s 2024-03-29T04:19:19.713 INFO:teuthology.orchestra.run.smithi022.stderr: 84150K .......... .......... .......... .......... .......... 49% 22.1M 5s 2024-03-29T04:19:19.715 INFO:teuthology.orchestra.run.smithi022.stderr: 84200K .......... .......... .......... .......... .......... 49% 25.5M 5s 2024-03-29T04:19:19.717 INFO:teuthology.orchestra.run.smithi022.stderr: 84250K .......... .......... .......... .......... .......... 50% 25.0M 5s 2024-03-29T04:19:19.719 INFO:teuthology.orchestra.run.smithi022.stderr: 84300K .......... .......... .......... .......... .......... 50% 23.3M 5s 2024-03-29T04:19:19.721 INFO:teuthology.orchestra.run.smithi022.stderr: 84350K .......... .......... .......... .......... .......... 50% 21.5M 5s 2024-03-29T04:19:19.723 INFO:teuthology.orchestra.run.smithi022.stderr: 84400K .......... .......... .......... .......... .......... 50% 24.5M 5s 2024-03-29T04:19:19.725 INFO:teuthology.orchestra.run.smithi022.stderr: 84450K .......... .......... .......... .......... .......... 50% 24.3M 5s 2024-03-29T04:19:19.727 INFO:teuthology.orchestra.run.smithi022.stderr: 84500K .......... .......... .......... .......... .......... 50% 24.3M 5s 2024-03-29T04:19:19.729 INFO:teuthology.orchestra.run.smithi022.stderr: 84550K .......... .......... .......... .......... .......... 50% 19.8M 5s 2024-03-29T04:19:19.731 INFO:teuthology.orchestra.run.smithi022.stderr: 84600K .......... .......... .......... .......... .......... 50% 24.3M 5s 2024-03-29T04:19:19.733 INFO:teuthology.orchestra.run.smithi022.stderr: 84650K .......... .......... .......... .......... .......... 50% 24.5M 5s 2024-03-29T04:19:19.736 INFO:teuthology.orchestra.run.smithi022.stderr: 84700K .......... .......... .......... .......... .......... 50% 23.5M 5s 2024-03-29T04:19:19.738 INFO:teuthology.orchestra.run.smithi022.stderr: 84750K .......... .......... .......... .......... .......... 50% 18.3M 5s 2024-03-29T04:19:19.741 INFO:teuthology.orchestra.run.smithi022.stderr: 84800K .......... .......... .......... .......... .......... 50% 19.5M 5s 2024-03-29T04:19:19.745 INFO:teuthology.orchestra.run.smithi022.stderr: 84850K .......... .......... .......... .......... .......... 50% 13.5M 5s 2024-03-29T04:19:19.749 INFO:teuthology.orchestra.run.smithi022.stderr: 84900K .......... .......... .......... .......... .......... 50% 11.7M 5s 2024-03-29T04:19:19.754 INFO:teuthology.orchestra.run.smithi022.stderr: 84950K .......... .......... .......... .......... .......... 50% 9.70M 5s 2024-03-29T04:19:19.758 INFO:teuthology.orchestra.run.smithi022.stderr: 85000K .......... .......... .......... .......... .......... 50% 12.9M 5s 2024-03-29T04:19:19.762 INFO:teuthology.orchestra.run.smithi022.stderr: 85050K .......... .......... .......... .......... .......... 50% 12.4M 5s 2024-03-29T04:19:19.766 INFO:teuthology.orchestra.run.smithi022.stderr: 85100K .......... .......... .......... .......... .......... 50% 11.8M 5s 2024-03-29T04:19:19.771 INFO:teuthology.orchestra.run.smithi022.stderr: 85150K .......... .......... .......... .......... .......... 50% 10.2M 5s 2024-03-29T04:19:19.774 INFO:teuthology.orchestra.run.smithi022.stderr: 85200K .......... .......... .......... .......... .......... 50% 13.9M 5s 2024-03-29T04:19:19.776 INFO:teuthology.orchestra.run.smithi022.stderr: 85250K .......... .......... .......... .......... .......... 50% 20.3M 5s 2024-03-29T04:19:19.779 INFO:teuthology.orchestra.run.smithi022.stderr: 85300K .......... .......... .......... .......... .......... 50% 21.2M 5s 2024-03-29T04:19:19.782 INFO:teuthology.orchestra.run.smithi022.stderr: 85350K .......... .......... .......... .......... .......... 50% 15.1M 5s 2024-03-29T04:19:19.785 INFO:teuthology.orchestra.run.smithi022.stderr: 85400K .......... .......... .......... .......... .......... 50% 17.2M 5s 2024-03-29T04:19:19.787 INFO:teuthology.orchestra.run.smithi022.stderr: 85450K .......... .......... .......... .......... .......... 50% 17.8M 5s 2024-03-29T04:19:19.790 INFO:teuthology.orchestra.run.smithi022.stderr: 85500K .......... .......... .......... .......... .......... 50% 18.0M 5s 2024-03-29T04:19:19.793 INFO:teuthology.orchestra.run.smithi022.stderr: 85550K .......... .......... .......... .......... .......... 50% 15.2M 5s 2024-03-29T04:19:19.796 INFO:teuthology.orchestra.run.smithi022.stderr: 85600K .......... .......... .......... .......... .......... 50% 17.1M 5s 2024-03-29T04:19:19.799 INFO:teuthology.orchestra.run.smithi022.stderr: 85650K .......... .......... .......... .......... .......... 50% 17.9M 5s 2024-03-29T04:19:19.801 INFO:teuthology.orchestra.run.smithi022.stderr: 85700K .......... .......... .......... .......... .......... 50% 20.5M 5s 2024-03-29T04:19:19.804 INFO:teuthology.orchestra.run.smithi022.stderr: 85750K .......... .......... .......... .......... .......... 50% 17.6M 5s 2024-03-29T04:19:19.806 INFO:teuthology.orchestra.run.smithi022.stderr: 85800K .......... .......... .......... .......... .......... 50% 21.2M 5s 2024-03-29T04:19:19.809 INFO:teuthology.orchestra.run.smithi022.stderr: 85850K .......... .......... .......... .......... .......... 50% 21.0M 5s 2024-03-29T04:19:19.811 INFO:teuthology.orchestra.run.smithi022.stderr: 85900K .......... .......... .......... .......... .......... 50% 18.8M 5s 2024-03-29T04:19:19.814 INFO:teuthology.orchestra.run.smithi022.stderr: 85950K .......... .......... .......... .......... .......... 51% 15.3M 5s 2024-03-29T04:19:19.817 INFO:teuthology.orchestra.run.smithi022.stderr: 86000K .......... .......... .......... .......... .......... 51% 17.3M 5s 2024-03-29T04:19:19.820 INFO:teuthology.orchestra.run.smithi022.stderr: 86050K .......... .......... .......... .......... .......... 51% 18.1M 5s 2024-03-29T04:19:19.822 INFO:teuthology.orchestra.run.smithi022.stderr: 86100K .......... .......... .......... .......... .......... 51% 21.2M 5s 2024-03-29T04:19:19.825 INFO:teuthology.orchestra.run.smithi022.stderr: 86150K .......... .......... .......... .......... .......... 51% 19.2M 5s 2024-03-29T04:19:19.827 INFO:teuthology.orchestra.run.smithi022.stderr: 86200K .......... .......... .......... .......... .......... 51% 24.1M 5s 2024-03-29T04:19:19.829 INFO:teuthology.orchestra.run.smithi022.stderr: 86250K .......... .......... .......... .......... .......... 51% 23.2M 5s 2024-03-29T04:19:19.831 INFO:teuthology.orchestra.run.smithi022.stderr: 86300K .......... .......... .......... .......... .......... 51% 23.3M 5s 2024-03-29T04:19:19.833 INFO:teuthology.orchestra.run.smithi022.stderr: 86350K .......... .......... .......... .......... .......... 51% 26.7M 5s 2024-03-29T04:19:19.834 INFO:teuthology.orchestra.run.smithi022.stderr: 86400K .......... .......... .......... .......... .......... 51% 31.4M 5s 2024-03-29T04:19:19.836 INFO:teuthology.orchestra.run.smithi022.stderr: 86450K .......... .......... .......... .......... .......... 51% 31.6M 5s 2024-03-29T04:19:19.837 INFO:teuthology.orchestra.run.smithi022.stderr: 86500K .......... .......... .......... .......... .......... 51% 32.8M 5s 2024-03-29T04:19:19.839 INFO:teuthology.orchestra.run.smithi022.stderr: 86550K .......... .......... .......... .......... .......... 51% 24.4M 5s 2024-03-29T04:19:19.841 INFO:teuthology.orchestra.run.smithi022.stderr: 86600K .......... .......... .......... .......... .......... 51% 29.9M 5s 2024-03-29T04:19:19.843 INFO:teuthology.orchestra.run.smithi022.stderr: 86650K .......... .......... .......... .......... .......... 51% 29.5M 5s 2024-03-29T04:19:19.844 INFO:teuthology.orchestra.run.smithi022.stderr: 86700K .......... .......... .......... .......... .......... 51% 31.2M 5s 2024-03-29T04:19:19.847 INFO:teuthology.orchestra.run.smithi022.stderr: 86750K .......... .......... .......... .......... .......... 51% 20.4M 5s 2024-03-29T04:19:19.848 INFO:teuthology.orchestra.run.smithi022.stderr: 86800K .......... .......... .......... .......... .......... 51% 29.1M 5s 2024-03-29T04:19:19.850 INFO:teuthology.orchestra.run.smithi022.stderr: 86850K .......... .......... .......... .......... .......... 51% 24.0M 5s 2024-03-29T04:19:19.852 INFO:teuthology.orchestra.run.smithi022.stderr: 86900K .......... .......... .......... .......... .......... 51% 23.6M 5s 2024-03-29T04:19:19.855 INFO:teuthology.orchestra.run.smithi022.stderr: 86950K .......... .......... .......... .......... .......... 51% 20.2M 5s 2024-03-29T04:19:19.857 INFO:teuthology.orchestra.run.smithi022.stderr: 87000K .......... .......... .......... .......... .......... 51% 24.1M 5s 2024-03-29T04:19:19.859 INFO:teuthology.orchestra.run.smithi022.stderr: 87050K .......... .......... .......... .......... .......... 51% 23.6M 5s 2024-03-29T04:19:19.861 INFO:teuthology.orchestra.run.smithi022.stderr: 87100K .......... .......... .......... .......... .......... 51% 23.5M 5s 2024-03-29T04:19:19.864 INFO:teuthology.orchestra.run.smithi022.stderr: 87150K .......... .......... .......... .......... .......... 51% 19.2M 5s 2024-03-29T04:19:19.866 INFO:teuthology.orchestra.run.smithi022.stderr: 87200K .......... .......... .......... .......... .......... 51% 23.1M 5s 2024-03-29T04:19:19.868 INFO:teuthology.orchestra.run.smithi022.stderr: 87250K .......... .......... .......... .......... .......... 51% 22.4M 5s 2024-03-29T04:19:19.870 INFO:teuthology.orchestra.run.smithi022.stderr: 87300K .......... .......... .......... .......... .......... 51% 23.2M 5s 2024-03-29T04:19:19.872 INFO:teuthology.orchestra.run.smithi022.stderr: 87350K .......... .......... .......... .......... .......... 51% 19.6M 5s 2024-03-29T04:19:19.874 INFO:teuthology.orchestra.run.smithi022.stderr: 87400K .......... .......... .......... .......... .......... 51% 22.9M 5s 2024-03-29T04:19:19.877 INFO:teuthology.orchestra.run.smithi022.stderr: 87450K .......... .......... .......... .......... .......... 51% 21.5M 5s 2024-03-29T04:19:19.879 INFO:teuthology.orchestra.run.smithi022.stderr: 87500K .......... .......... .......... .......... .......... 51% 20.5M 5s 2024-03-29T04:19:19.882 INFO:teuthology.orchestra.run.smithi022.stderr: 87550K .......... .......... .......... .......... .......... 51% 17.7M 5s 2024-03-29T04:19:19.888 INFO:teuthology.orchestra.run.smithi022.stderr: 87600K .......... .......... .......... .......... .......... 51% 20.4M 5s 2024-03-29T04:19:19.888 INFO:teuthology.orchestra.run.smithi022.stderr: 87650K .......... .......... .......... .......... .......... 52% 22.6M 5s 2024-03-29T04:19:19.889 INFO:teuthology.orchestra.run.smithi022.stderr: 87700K .......... .......... .......... .......... .......... 52% 21.7M 5s 2024-03-29T04:19:19.891 INFO:teuthology.orchestra.run.smithi022.stderr: 87750K .......... .......... .......... .......... .......... 52% 17.9M 5s 2024-03-29T04:19:19.894 INFO:teuthology.orchestra.run.smithi022.stderr: 87800K .......... .......... .......... .......... .......... 52% 20.4M 5s 2024-03-29T04:19:19.896 INFO:teuthology.orchestra.run.smithi022.stderr: 87850K .......... .......... .......... .......... .......... 52% 20.1M 5s 2024-03-29T04:19:19.899 INFO:teuthology.orchestra.run.smithi022.stderr: 87900K .......... .......... .......... .......... .......... 52% 20.8M 5s 2024-03-29T04:19:19.901 INFO:teuthology.orchestra.run.smithi022.stderr: 87950K .......... .......... .......... .......... .......... 52% 17.8M 5s 2024-03-29T04:19:19.904 INFO:teuthology.orchestra.run.smithi022.stderr: 88000K .......... .......... .......... .......... .......... 52% 20.0M 5s 2024-03-29T04:19:19.906 INFO:teuthology.orchestra.run.smithi022.stderr: 88050K .......... .......... .......... .......... .......... 52% 24.4M 5s 2024-03-29T04:19:19.908 INFO:teuthology.orchestra.run.smithi022.stderr: 88100K .......... .......... .......... .......... .......... 52% 21.7M 5s 2024-03-29T04:19:19.911 INFO:teuthology.orchestra.run.smithi022.stderr: 88150K .......... .......... .......... .......... .......... 52% 16.9M 5s 2024-03-29T04:19:19.914 INFO:teuthology.orchestra.run.smithi022.stderr: 88200K .......... .......... .......... .......... .......... 52% 19.2M 5s 2024-03-29T04:19:19.916 INFO:teuthology.orchestra.run.smithi022.stderr: 88250K .......... .......... .......... .......... .......... 52% 19.5M 5s 2024-03-29T04:19:19.918 INFO:teuthology.orchestra.run.smithi022.stderr: 88300K .......... .......... .......... .......... .......... 52% 20.2M 5s 2024-03-29T04:19:19.921 INFO:teuthology.orchestra.run.smithi022.stderr: 88350K .......... .......... .......... .......... .......... 52% 16.5M 5s 2024-03-29T04:19:19.924 INFO:teuthology.orchestra.run.smithi022.stderr: 88400K .......... .......... .......... .......... .......... 52% 19.1M 5s 2024-03-29T04:19:19.928 INFO:teuthology.orchestra.run.smithi022.stderr: 88450K .......... .......... .......... .......... .......... 52% 16.0M 5s 2024-03-29T04:19:19.930 INFO:teuthology.orchestra.run.smithi022.stderr: 88500K .......... .......... .......... .......... .......... 52% 17.7M 5s 2024-03-29T04:19:19.933 INFO:teuthology.orchestra.run.smithi022.stderr: 88550K .......... .......... .......... .......... .......... 52% 15.7M 5s 2024-03-29T04:19:19.935 INFO:teuthology.orchestra.run.smithi022.stderr: 88600K .......... .......... .......... .......... .......... 52% 18.4M 5s 2024-03-29T04:19:19.938 INFO:teuthology.orchestra.run.smithi022.stderr: 88650K .......... .......... .......... .......... .......... 52% 18.6M 5s 2024-03-29T04:19:19.941 INFO:teuthology.orchestra.run.smithi022.stderr: 88700K .......... .......... .......... .......... .......... 52% 18.4M 5s 2024-03-29T04:19:19.944 INFO:teuthology.orchestra.run.smithi022.stderr: 88750K .......... .......... .......... .......... .......... 52% 16.0M 5s 2024-03-29T04:19:19.947 INFO:teuthology.orchestra.run.smithi022.stderr: 88800K .......... .......... .......... .......... .......... 52% 18.3M 5s 2024-03-29T04:19:19.949 INFO:teuthology.orchestra.run.smithi022.stderr: 88850K .......... .......... .......... .......... .......... 52% 17.8M 5s 2024-03-29T04:19:19.952 INFO:teuthology.orchestra.run.smithi022.stderr: 88900K .......... .......... .......... .......... .......... 52% 17.9M 5s 2024-03-29T04:19:19.955 INFO:teuthology.orchestra.run.smithi022.stderr: 88950K .......... .......... .......... .......... .......... 52% 14.9M 5s 2024-03-29T04:19:19.958 INFO:teuthology.orchestra.run.smithi022.stderr: 89000K .......... .......... .......... .......... .......... 52% 17.3M 5s 2024-03-29T04:19:19.961 INFO:teuthology.orchestra.run.smithi022.stderr: 89050K .......... .......... .......... .......... .......... 52% 17.1M 5s 2024-03-29T04:19:19.964 INFO:teuthology.orchestra.run.smithi022.stderr: 89100K .......... .......... .......... .......... .......... 52% 16.8M 5s 2024-03-29T04:19:19.968 INFO:teuthology.orchestra.run.smithi022.stderr: 89150K .......... .......... .......... .......... .......... 52% 14.0M 5s 2024-03-29T04:19:19.970 INFO:teuthology.orchestra.run.smithi022.stderr: 89200K .......... .......... .......... .......... .......... 52% 17.4M 5s 2024-03-29T04:19:19.973 INFO:teuthology.orchestra.run.smithi022.stderr: 89250K .......... .......... .......... .......... .......... 52% 17.2M 5s 2024-03-29T04:19:19.976 INFO:teuthology.orchestra.run.smithi022.stderr: 89300K .......... .......... .......... .......... .......... 53% 16.4M 5s 2024-03-29T04:19:19.980 INFO:teuthology.orchestra.run.smithi022.stderr: 89350K .......... .......... .......... .......... .......... 53% 14.1M 5s 2024-03-29T04:19:19.983 INFO:teuthology.orchestra.run.smithi022.stderr: 89400K .......... .......... .......... .......... .......... 53% 17.5M 5s 2024-03-29T04:19:19.985 INFO:teuthology.orchestra.run.smithi022.stderr: 89450K .......... .......... .......... .......... .......... 53% 16.7M 5s 2024-03-29T04:19:19.988 INFO:teuthology.orchestra.run.smithi022.stderr: 89500K .......... .......... .......... .......... .......... 53% 16.1M 5s 2024-03-29T04:19:19.992 INFO:teuthology.orchestra.run.smithi022.stderr: 89550K .......... .......... .......... .......... .......... 53% 14.2M 5s 2024-03-29T04:19:19.995 INFO:teuthology.orchestra.run.smithi022.stderr: 89600K .......... .......... .......... .......... .......... 53% 16.6M 5s 2024-03-29T04:19:19.998 INFO:teuthology.orchestra.run.smithi022.stderr: 89650K .......... .......... .......... .......... .......... 53% 15.8M 5s 2024-03-29T04:19:20.001 INFO:teuthology.orchestra.run.smithi022.stderr: 89700K .......... .......... .......... .......... .......... 53% 16.1M 4s 2024-03-29T04:19:20.005 INFO:teuthology.orchestra.run.smithi022.stderr: 89750K .......... .......... .......... .......... .......... 53% 13.6M 4s 2024-03-29T04:19:20.008 INFO:teuthology.orchestra.run.smithi022.stderr: 89800K .......... .......... .......... .......... .......... 53% 15.4M 4s 2024-03-29T04:19:20.011 INFO:teuthology.orchestra.run.smithi022.stderr: 89850K .......... .......... .......... .......... .......... 53% 14.1M 4s 2024-03-29T04:19:20.015 INFO:teuthology.orchestra.run.smithi022.stderr: 89900K .......... .......... .......... .......... .......... 53% 12.6M 4s 2024-03-29T04:19:20.020 INFO:teuthology.orchestra.run.smithi022.stderr: 89950K .......... .......... .......... .......... .......... 53% 11.0M 4s 2024-03-29T04:19:20.023 INFO:teuthology.orchestra.run.smithi022.stderr: 90000K .......... .......... .......... .......... .......... 53% 14.6M 4s 2024-03-29T04:19:20.026 INFO:teuthology.orchestra.run.smithi022.stderr: 90050K .......... .......... .......... .......... .......... 53% 13.7M 4s 2024-03-29T04:19:20.029 INFO:teuthology.orchestra.run.smithi022.stderr: 90100K .......... .......... .......... .......... .......... 53% 15.3M 4s 2024-03-29T04:19:20.032 INFO:teuthology.orchestra.run.smithi022.stderr: 90150K .......... .......... .......... .......... .......... 53% 20.3M 4s 2024-03-29T04:19:20.034 INFO:teuthology.orchestra.run.smithi022.stderr: 90200K .......... .......... .......... .......... .......... 53% 21.2M 4s 2024-03-29T04:19:20.036 INFO:teuthology.orchestra.run.smithi022.stderr: 90250K .......... .......... .......... .......... .......... 53% 20.6M 4s 2024-03-29T04:19:20.039 INFO:teuthology.orchestra.run.smithi022.stderr: 90300K .......... .......... .......... .......... .......... 53% 19.7M 4s 2024-03-29T04:19:20.042 INFO:teuthology.orchestra.run.smithi022.stderr: 90350K .......... .......... .......... .......... .......... 53% 15.9M 4s 2024-03-29T04:19:20.045 INFO:teuthology.orchestra.run.smithi022.stderr: 90400K .......... .......... .......... .......... .......... 53% 18.0M 4s 2024-03-29T04:19:20.051 INFO:teuthology.orchestra.run.smithi022.stderr: 90450K .......... .......... .......... .......... .......... 53% 7.84M 4s 2024-03-29T04:19:20.055 INFO:teuthology.orchestra.run.smithi022.stderr: 90500K .......... .......... .......... .......... .......... 53% 13.7M 4s 2024-03-29T04:19:20.058 INFO:teuthology.orchestra.run.smithi022.stderr: 90550K .......... .......... .......... .......... .......... 53% 12.4M 4s 2024-03-29T04:19:20.061 INFO:teuthology.orchestra.run.smithi022.stderr: 90600K .......... .......... .......... .......... .......... 53% 18.0M 4s 2024-03-29T04:19:20.064 INFO:teuthology.orchestra.run.smithi022.stderr: 90650K .......... .......... .......... .......... .......... 53% 17.9M 4s 2024-03-29T04:19:20.067 INFO:teuthology.orchestra.run.smithi022.stderr: 90700K .......... .......... .......... .......... .......... 53% 18.3M 4s 2024-03-29T04:19:20.070 INFO:teuthology.orchestra.run.smithi022.stderr: 90750K .......... .......... .......... .......... .......... 53% 15.3M 4s 2024-03-29T04:19:20.072 INFO:teuthology.orchestra.run.smithi022.stderr: 90800K .......... .......... .......... .......... .......... 53% 18.5M 4s 2024-03-29T04:19:20.075 INFO:teuthology.orchestra.run.smithi022.stderr: 90850K .......... .......... .......... .......... .......... 53% 19.0M 4s 2024-03-29T04:19:20.077 INFO:teuthology.orchestra.run.smithi022.stderr: 90900K .......... .......... .......... .......... .......... 53% 22.2M 4s 2024-03-29T04:19:20.080 INFO:teuthology.orchestra.run.smithi022.stderr: 90950K .......... .......... .......... .......... .......... 53% 18.5M 4s 2024-03-29T04:19:20.082 INFO:teuthology.orchestra.run.smithi022.stderr: 91000K .......... .......... .......... .......... .......... 54% 22.2M 4s 2024-03-29T04:19:20.084 INFO:teuthology.orchestra.run.smithi022.stderr: 91050K .......... .......... .......... .......... .......... 54% 22.1M 4s 2024-03-29T04:19:20.086 INFO:teuthology.orchestra.run.smithi022.stderr: 91100K .......... .......... .......... .......... .......... 54% 22.1M 4s 2024-03-29T04:19:20.091 INFO:teuthology.orchestra.run.smithi022.stderr: 91150K .......... .......... .......... .......... .......... 54% 18.8M 4s 2024-03-29T04:19:20.091 INFO:teuthology.orchestra.run.smithi022.stderr: 91200K .......... .......... .......... .......... .......... 54% 21.5M 4s 2024-03-29T04:19:20.093 INFO:teuthology.orchestra.run.smithi022.stderr: 91250K .......... .......... .......... .......... .......... 54% 21.5M 4s 2024-03-29T04:19:20.096 INFO:teuthology.orchestra.run.smithi022.stderr: 91300K .......... .......... .......... .......... .......... 54% 21.4M 4s 2024-03-29T04:19:20.098 INFO:teuthology.orchestra.run.smithi022.stderr: 91350K .......... .......... .......... .......... .......... 54% 18.5M 4s 2024-03-29T04:19:20.101 INFO:teuthology.orchestra.run.smithi022.stderr: 91400K .......... .......... .......... .......... .......... 54% 21.1M 4s 2024-03-29T04:19:20.103 INFO:teuthology.orchestra.run.smithi022.stderr: 91450K .......... .......... .......... .......... .......... 54% 19.6M 4s 2024-03-29T04:19:20.106 INFO:teuthology.orchestra.run.smithi022.stderr: 91500K .......... .......... .......... .......... .......... 54% 16.0M 4s 2024-03-29T04:19:20.110 INFO:teuthology.orchestra.run.smithi022.stderr: 91550K .......... .......... .......... .......... .......... 54% 13.6M 4s 2024-03-29T04:19:20.113 INFO:teuthology.orchestra.run.smithi022.stderr: 91600K .......... .......... .......... .......... .......... 54% 15.9M 4s 2024-03-29T04:19:20.116 INFO:teuthology.orchestra.run.smithi022.stderr: 91650K .......... .......... .......... .......... .......... 54% 16.4M 4s 2024-03-29T04:19:20.119 INFO:teuthology.orchestra.run.smithi022.stderr: 91700K .......... .......... .......... .......... .......... 54% 16.2M 4s 2024-03-29T04:19:20.122 INFO:teuthology.orchestra.run.smithi022.stderr: 91750K .......... .......... .......... .......... .......... 54% 14.3M 4s 2024-03-29T04:19:20.125 INFO:teuthology.orchestra.run.smithi022.stderr: 91800K .......... .......... .......... .......... .......... 54% 16.2M 4s 2024-03-29T04:19:20.128 INFO:teuthology.orchestra.run.smithi022.stderr: 91850K .......... .......... .......... .......... .......... 54% 17.5M 4s 2024-03-29T04:19:20.131 INFO:teuthology.orchestra.run.smithi022.stderr: 91900K .......... .......... .......... .......... .......... 54% 17.3M 4s 2024-03-29T04:19:20.134 INFO:teuthology.orchestra.run.smithi022.stderr: 91950K .......... .......... .......... .......... .......... 54% 14.5M 4s 2024-03-29T04:19:20.137 INFO:teuthology.orchestra.run.smithi022.stderr: 92000K .......... .......... .......... .......... .......... 54% 16.9M 4s 2024-03-29T04:19:20.141 INFO:teuthology.orchestra.run.smithi022.stderr: 92050K .......... .......... .......... .......... .......... 54% 15.7M 4s 2024-03-29T04:19:20.143 INFO:teuthology.orchestra.run.smithi022.stderr: 92100K .......... .......... .......... .......... .......... 54% 16.2M 4s 2024-03-29T04:19:20.147 INFO:teuthology.orchestra.run.smithi022.stderr: 92150K .......... .......... .......... .......... .......... 54% 15.0M 4s 2024-03-29T04:19:20.149 INFO:teuthology.orchestra.run.smithi022.stderr: 92200K .......... .......... .......... .......... .......... 54% 16.9M 4s 2024-03-29T04:19:20.153 INFO:teuthology.orchestra.run.smithi022.stderr: 92250K .......... .......... .......... .......... .......... 54% 16.3M 4s 2024-03-29T04:19:20.155 INFO:teuthology.orchestra.run.smithi022.stderr: 92300K .......... .......... .......... .......... .......... 54% 16.9M 4s 2024-03-29T04:19:20.159 INFO:teuthology.orchestra.run.smithi022.stderr: 92350K .......... .......... .......... .......... .......... 54% 14.1M 4s 2024-03-29T04:19:20.162 INFO:teuthology.orchestra.run.smithi022.stderr: 92400K .......... .......... .......... .......... .......... 54% 15.0M 4s 2024-03-29T04:19:20.165 INFO:teuthology.orchestra.run.smithi022.stderr: 92450K .......... .......... .......... .......... .......... 54% 15.1M 4s 2024-03-29T04:19:20.169 INFO:teuthology.orchestra.run.smithi022.stderr: 92500K .......... .......... .......... .......... .......... 54% 13.4M 4s 2024-03-29T04:19:20.173 INFO:teuthology.orchestra.run.smithi022.stderr: 92550K .......... .......... .......... .......... .......... 54% 11.8M 4s 2024-03-29T04:19:20.176 INFO:teuthology.orchestra.run.smithi022.stderr: 92600K .......... .......... .......... .......... .......... 54% 18.3M 4s 2024-03-29T04:19:20.178 INFO:teuthology.orchestra.run.smithi022.stderr: 92650K .......... .......... .......... .......... .......... 54% 21.8M 4s 2024-03-29T04:19:20.180 INFO:teuthology.orchestra.run.smithi022.stderr: 92700K .......... .......... .......... .......... .......... 55% 20.5M 4s 2024-03-29T04:19:20.183 INFO:teuthology.orchestra.run.smithi022.stderr: 92750K .......... .......... .......... .......... .......... 55% 18.3M 4s 2024-03-29T04:19:20.186 INFO:teuthology.orchestra.run.smithi022.stderr: 92800K .......... .......... .......... .......... .......... 55% 20.5M 4s 2024-03-29T04:19:20.188 INFO:teuthology.orchestra.run.smithi022.stderr: 92850K .......... .......... .......... .......... .......... 55% 18.2M 4s 2024-03-29T04:19:20.191 INFO:teuthology.orchestra.run.smithi022.stderr: 92900K .......... .......... .......... .......... .......... 55% 18.2M 4s 2024-03-29T04:19:20.194 INFO:teuthology.orchestra.run.smithi022.stderr: 92950K .......... .......... .......... .......... .......... 55% 16.2M 4s 2024-03-29T04:19:20.196 INFO:teuthology.orchestra.run.smithi022.stderr: 93000K .......... .......... .......... .......... .......... 55% 19.6M 4s 2024-03-29T04:19:20.199 INFO:teuthology.orchestra.run.smithi022.stderr: 93050K .......... .......... .......... .......... .......... 55% 18.0M 4s 2024-03-29T04:19:20.201 INFO:teuthology.orchestra.run.smithi022.stderr: 93100K .......... .......... .......... .......... .......... 55% 19.8M 4s 2024-03-29T04:19:20.204 INFO:teuthology.orchestra.run.smithi022.stderr: 93150K .......... .......... .......... .......... .......... 55% 17.3M 4s 2024-03-29T04:19:20.207 INFO:teuthology.orchestra.run.smithi022.stderr: 93200K .......... .......... .......... .......... .......... 55% 19.2M 4s 2024-03-29T04:19:20.211 INFO:teuthology.orchestra.run.smithi022.stderr: 93250K .......... .......... .......... .......... .......... 55% 13.8M 4s 2024-03-29T04:19:20.215 INFO:teuthology.orchestra.run.smithi022.stderr: 93300K .......... .......... .......... .......... .......... 55% 12.3M 4s 2024-03-29T04:19:20.219 INFO:teuthology.orchestra.run.smithi022.stderr: 93350K .......... .......... .......... .......... .......... 55% 10.4M 4s 2024-03-29T04:19:20.223 INFO:teuthology.orchestra.run.smithi022.stderr: 93400K .......... .......... .......... .......... .......... 55% 12.5M 4s 2024-03-29T04:19:20.228 INFO:teuthology.orchestra.run.smithi022.stderr: 93450K .......... .......... .......... .......... .......... 55% 11.2M 4s 2024-03-29T04:19:20.231 INFO:teuthology.orchestra.run.smithi022.stderr: 93500K .......... .......... .......... .......... .......... 55% 14.6M 4s 2024-03-29T04:19:20.234 INFO:teuthology.orchestra.run.smithi022.stderr: 93550K .......... .......... .......... .......... .......... 55% 16.0M 4s 2024-03-29T04:19:20.236 INFO:teuthology.orchestra.run.smithi022.stderr: 93600K .......... .......... .......... .......... .......... 55% 18.3M 4s 2024-03-29T04:19:20.239 INFO:teuthology.orchestra.run.smithi022.stderr: 93650K .......... .......... .......... .......... .......... 55% 18.7M 4s 2024-03-29T04:19:20.242 INFO:teuthology.orchestra.run.smithi022.stderr: 93700K .......... .......... .......... .......... .......... 55% 19.1M 4s 2024-03-29T04:19:20.245 INFO:teuthology.orchestra.run.smithi022.stderr: 93750K .......... .......... .......... .......... .......... 55% 15.4M 4s 2024-03-29T04:19:20.247 INFO:teuthology.orchestra.run.smithi022.stderr: 93800K .......... .......... .......... .......... .......... 55% 18.9M 4s 2024-03-29T04:19:20.250 INFO:teuthology.orchestra.run.smithi022.stderr: 93850K .......... .......... .......... .......... .......... 55% 17.9M 4s 2024-03-29T04:19:20.254 INFO:teuthology.orchestra.run.smithi022.stderr: 93900K .......... .......... .......... .......... .......... 55% 14.4M 4s 2024-03-29T04:19:20.259 INFO:teuthology.orchestra.run.smithi022.stderr: 93950K .......... .......... .......... .......... .......... 55% 8.70M 4s 2024-03-29T04:19:20.265 INFO:teuthology.orchestra.run.smithi022.stderr: 94000K .......... .......... .......... .......... .......... 55% 9.04M 4s 2024-03-29T04:19:20.271 INFO:teuthology.orchestra.run.smithi022.stderr: 94050K .......... .......... .......... .......... .......... 55% 8.85M 4s 2024-03-29T04:19:20.276 INFO:teuthology.orchestra.run.smithi022.stderr: 94100K .......... .......... .......... .......... .......... 55% 8.48M 4s 2024-03-29T04:19:20.283 INFO:teuthology.orchestra.run.smithi022.stderr: 94150K .......... .......... .......... .......... .......... 55% 7.42M 4s 2024-03-29T04:19:20.286 INFO:teuthology.orchestra.run.smithi022.stderr: 94200K .......... .......... .......... .......... .......... 55% 12.3M 4s 2024-03-29T04:19:20.289 INFO:teuthology.orchestra.run.smithi022.stderr: 94250K .......... .......... .......... .......... .......... 55% 17.9M 4s 2024-03-29T04:19:20.295 INFO:teuthology.orchestra.run.smithi022.stderr: 94300K .......... .......... .......... .......... .......... 55% 19.2M 4s 2024-03-29T04:19:20.295 INFO:teuthology.orchestra.run.smithi022.stderr: 94350K .......... .......... .......... .......... .......... 56% 17.7M 4s 2024-03-29T04:19:20.297 INFO:teuthology.orchestra.run.smithi022.stderr: 94400K .......... .......... .......... .......... .......... 56% 20.2M 4s 2024-03-29T04:19:20.299 INFO:teuthology.orchestra.run.smithi022.stderr: 94450K .......... .......... .......... .......... .......... 56% 19.3M 4s 2024-03-29T04:19:20.303 INFO:teuthology.orchestra.run.smithi022.stderr: 94500K .......... .......... .......... .......... .......... 56% 14.7M 4s 2024-03-29T04:19:20.307 INFO:teuthology.orchestra.run.smithi022.stderr: 94550K .......... .......... .......... .......... .......... 56% 12.4M 4s 2024-03-29T04:19:20.310 INFO:teuthology.orchestra.run.smithi022.stderr: 94600K .......... .......... .......... .......... .......... 56% 15.3M 4s 2024-03-29T04:19:20.313 INFO:teuthology.orchestra.run.smithi022.stderr: 94650K .......... .......... .......... .......... .......... 56% 16.0M 4s 2024-03-29T04:19:20.316 INFO:teuthology.orchestra.run.smithi022.stderr: 94700K .......... .......... .......... .......... .......... 56% 17.2M 4s 2024-03-29T04:19:20.320 INFO:teuthology.orchestra.run.smithi022.stderr: 94750K .......... .......... .......... .......... .......... 56% 12.9M 4s 2024-03-29T04:19:20.323 INFO:teuthology.orchestra.run.smithi022.stderr: 94800K .......... .......... .......... .......... .......... 56% 14.8M 4s 2024-03-29T04:19:20.326 INFO:teuthology.orchestra.run.smithi022.stderr: 94850K .......... .......... .......... .......... .......... 56% 14.9M 4s 2024-03-29T04:19:20.330 INFO:teuthology.orchestra.run.smithi022.stderr: 94900K .......... .......... .......... .......... .......... 56% 14.7M 4s 2024-03-29T04:19:20.333 INFO:teuthology.orchestra.run.smithi022.stderr: 94950K .......... .......... .......... .......... .......... 56% 12.9M 4s 2024-03-29T04:19:20.337 INFO:teuthology.orchestra.run.smithi022.stderr: 95000K .......... .......... .......... .......... .......... 56% 15.1M 4s 2024-03-29T04:19:20.340 INFO:teuthology.orchestra.run.smithi022.stderr: 95050K .......... .......... .......... .......... .......... 56% 14.7M 4s 2024-03-29T04:19:20.343 INFO:teuthology.orchestra.run.smithi022.stderr: 95100K .......... .......... .......... .......... .......... 56% 14.5M 4s 2024-03-29T04:19:20.347 INFO:teuthology.orchestra.run.smithi022.stderr: 95150K .......... .......... .......... .......... .......... 56% 12.2M 4s 2024-03-29T04:19:20.351 INFO:teuthology.orchestra.run.smithi022.stderr: 95200K .......... .......... .......... .......... .......... 56% 14.0M 4s 2024-03-29T04:19:20.354 INFO:teuthology.orchestra.run.smithi022.stderr: 95250K .......... .......... .......... .......... .......... 56% 14.4M 4s 2024-03-29T04:19:20.358 INFO:teuthology.orchestra.run.smithi022.stderr: 95300K .......... .......... .......... .......... .......... 56% 13.9M 4s 2024-03-29T04:19:20.362 INFO:teuthology.orchestra.run.smithi022.stderr: 95350K .......... .......... .......... .......... .......... 56% 10.8M 4s 2024-03-29T04:19:20.366 INFO:teuthology.orchestra.run.smithi022.stderr: 95400K .......... .......... .......... .......... .......... 56% 14.0M 4s 2024-03-29T04:19:20.369 INFO:teuthology.orchestra.run.smithi022.stderr: 95450K .......... .......... .......... .......... .......... 56% 13.9M 4s 2024-03-29T04:19:20.373 INFO:teuthology.orchestra.run.smithi022.stderr: 95500K .......... .......... .......... .......... .......... 56% 14.1M 4s 2024-03-29T04:19:20.377 INFO:teuthology.orchestra.run.smithi022.stderr: 95550K .......... .......... .......... .......... .......... 56% 11.9M 4s 2024-03-29T04:19:20.380 INFO:teuthology.orchestra.run.smithi022.stderr: 95600K .......... .......... .......... .......... .......... 56% 13.4M 4s 2024-03-29T04:19:20.384 INFO:teuthology.orchestra.run.smithi022.stderr: 95650K .......... .......... .......... .......... .......... 56% 13.2M 4s 2024-03-29T04:19:20.387 INFO:teuthology.orchestra.run.smithi022.stderr: 95700K .......... .......... .......... .......... .......... 56% 14.6M 4s 2024-03-29T04:19:20.391 INFO:teuthology.orchestra.run.smithi022.stderr: 95750K .......... .......... .......... .......... .......... 56% 12.6M 4s 2024-03-29T04:19:20.395 INFO:teuthology.orchestra.run.smithi022.stderr: 95800K .......... .......... .......... .......... .......... 56% 14.7M 4s 2024-03-29T04:19:20.398 INFO:teuthology.orchestra.run.smithi022.stderr: 95850K .......... .......... .......... .......... .......... 56% 14.6M 4s 2024-03-29T04:19:20.401 INFO:teuthology.orchestra.run.smithi022.stderr: 95900K .......... .......... .......... .......... .......... 56% 14.2M 4s 2024-03-29T04:19:20.405 INFO:teuthology.orchestra.run.smithi022.stderr: 95950K .......... .......... .......... .......... .......... 56% 12.2M 4s 2024-03-29T04:19:20.409 INFO:teuthology.orchestra.run.smithi022.stderr: 96000K .......... .......... .......... .......... .......... 56% 12.9M 4s 2024-03-29T04:19:20.413 INFO:teuthology.orchestra.run.smithi022.stderr: 96050K .......... .......... .......... .......... .......... 57% 13.1M 4s 2024-03-29T04:19:20.417 INFO:teuthology.orchestra.run.smithi022.stderr: 96100K .......... .......... .......... .......... .......... 57% 12.7M 4s 2024-03-29T04:19:20.421 INFO:teuthology.orchestra.run.smithi022.stderr: 96150K .......... .......... .......... .......... .......... 57% 10.8M 4s 2024-03-29T04:19:20.425 INFO:teuthology.orchestra.run.smithi022.stderr: 96200K .......... .......... .......... .......... .......... 57% 12.7M 4s 2024-03-29T04:19:20.429 INFO:teuthology.orchestra.run.smithi022.stderr: 96250K .......... .......... .......... .......... .......... 57% 11.4M 4s 2024-03-29T04:19:20.434 INFO:teuthology.orchestra.run.smithi022.stderr: 96300K .......... .......... .......... .......... .......... 57% 11.6M 4s 2024-03-29T04:19:20.439 INFO:teuthology.orchestra.run.smithi022.stderr: 96350K .......... .......... .......... .......... .......... 57% 9.95M 4s 2024-03-29T04:19:20.443 INFO:teuthology.orchestra.run.smithi022.stderr: 96400K .......... .......... .......... .......... .......... 57% 11.6M 4s 2024-03-29T04:19:20.447 INFO:teuthology.orchestra.run.smithi022.stderr: 96450K .......... .......... .......... .......... .......... 57% 10.8M 4s 2024-03-29T04:19:20.452 INFO:teuthology.orchestra.run.smithi022.stderr: 96500K .......... .......... .......... .......... .......... 57% 10.8M 4s 2024-03-29T04:19:20.457 INFO:teuthology.orchestra.run.smithi022.stderr: 96550K .......... .......... .......... .......... .......... 57% 9.84M 4s 2024-03-29T04:19:20.461 INFO:teuthology.orchestra.run.smithi022.stderr: 96600K .......... .......... .......... .......... .......... 57% 12.4M 4s 2024-03-29T04:19:20.465 INFO:teuthology.orchestra.run.smithi022.stderr: 96650K .......... .......... .......... .......... .......... 57% 10.9M 4s 2024-03-29T04:19:20.469 INFO:teuthology.orchestra.run.smithi022.stderr: 96700K .......... .......... .......... .......... .......... 57% 11.5M 4s 2024-03-29T04:19:20.475 INFO:teuthology.orchestra.run.smithi022.stderr: 96750K .......... .......... .......... .......... .......... 57% 9.65M 4s 2024-03-29T04:19:20.479 INFO:teuthology.orchestra.run.smithi022.stderr: 96800K .......... .......... .......... .......... .......... 57% 11.2M 4s 2024-03-29T04:19:20.483 INFO:teuthology.orchestra.run.smithi022.stderr: 96850K .......... .......... .......... .......... .......... 57% 11.0M 4s 2024-03-29T04:19:20.488 INFO:teuthology.orchestra.run.smithi022.stderr: 96900K .......... .......... .......... .......... .......... 57% 11.3M 4s 2024-03-29T04:19:20.493 INFO:teuthology.orchestra.run.smithi022.stderr: 96950K .......... .......... .......... .......... .......... 57% 9.58M 4s 2024-03-29T04:19:20.499 INFO:teuthology.orchestra.run.smithi022.stderr: 97000K .......... .......... .......... .......... .......... 57% 11.3M 4s 2024-03-29T04:19:20.501 INFO:teuthology.orchestra.run.smithi022.stderr: 97050K .......... .......... .......... .......... .......... 57% 11.3M 4s 2024-03-29T04:19:20.506 INFO:teuthology.orchestra.run.smithi022.stderr: 97100K .......... .......... .......... .......... .......... 57% 10.7M 4s 2024-03-29T04:19:20.511 INFO:teuthology.orchestra.run.smithi022.stderr: 97150K .......... .......... .......... .......... .......... 57% 9.33M 4s 2024-03-29T04:19:20.516 INFO:teuthology.orchestra.run.smithi022.stderr: 97200K .......... .......... .......... .......... .......... 57% 10.8M 4s 2024-03-29T04:19:20.520 INFO:teuthology.orchestra.run.smithi022.stderr: 97250K .......... .......... .......... .......... .......... 57% 11.1M 4s 2024-03-29T04:19:20.524 INFO:teuthology.orchestra.run.smithi022.stderr: 97300K .......... .......... .......... .......... .......... 57% 11.0M 4s 2024-03-29T04:19:20.530 INFO:teuthology.orchestra.run.smithi022.stderr: 97350K .......... .......... .......... .......... .......... 57% 9.26M 4s 2024-03-29T04:19:20.534 INFO:teuthology.orchestra.run.smithi022.stderr: 97400K .......... .......... .......... .......... .......... 57% 11.2M 4s 2024-03-29T04:19:20.539 INFO:teuthology.orchestra.run.smithi022.stderr: 97450K .......... .......... .......... .......... .......... 57% 10.9M 4s 2024-03-29T04:19:20.543 INFO:teuthology.orchestra.run.smithi022.stderr: 97500K .......... .......... .......... .......... .......... 57% 11.0M 4s 2024-03-29T04:19:20.549 INFO:teuthology.orchestra.run.smithi022.stderr: 97550K .......... .......... .......... .......... .......... 57% 8.85M 4s 2024-03-29T04:19:20.553 INFO:teuthology.orchestra.run.smithi022.stderr: 97600K .......... .......... .......... .......... .......... 57% 10.6M 4s 2024-03-29T04:19:20.558 INFO:teuthology.orchestra.run.smithi022.stderr: 97650K .......... .......... .......... .......... .......... 57% 9.61M 4s 2024-03-29T04:19:20.564 INFO:teuthology.orchestra.run.smithi022.stderr: 97700K .......... .......... .......... .......... .......... 57% 9.60M 4s 2024-03-29T04:19:20.571 INFO:teuthology.orchestra.run.smithi022.stderr: 97750K .......... .......... .......... .......... .......... 58% 7.02M 4s 2024-03-29T04:19:20.577 INFO:teuthology.orchestra.run.smithi022.stderr: 97800K .......... .......... .......... .......... .......... 58% 7.79M 4s 2024-03-29T04:19:20.582 INFO:teuthology.orchestra.run.smithi022.stderr: 97850K .......... .......... .......... .......... .......... 58% 8.87M 4s 2024-03-29T04:19:20.588 INFO:teuthology.orchestra.run.smithi022.stderr: 97900K .......... .......... .......... .......... .......... 58% 8.50M 4s 2024-03-29T04:19:20.594 INFO:teuthology.orchestra.run.smithi022.stderr: 97950K .......... .......... .......... .......... .......... 58% 7.61M 4s 2024-03-29T04:19:20.600 INFO:teuthology.orchestra.run.smithi022.stderr: 98000K .......... .......... .......... .......... .......... 58% 9.07M 4s 2024-03-29T04:19:20.604 INFO:teuthology.orchestra.run.smithi022.stderr: 98050K .......... .......... .......... .......... .......... 58% 10.3M 4s 2024-03-29T04:19:20.607 INFO:teuthology.orchestra.run.smithi022.stderr: 98100K .......... .......... .......... .......... .......... 58% 16.0M 4s 2024-03-29T04:19:20.611 INFO:teuthology.orchestra.run.smithi022.stderr: 98150K .......... .......... .......... .......... .......... 58% 13.7M 4s 2024-03-29T04:19:20.614 INFO:teuthology.orchestra.run.smithi022.stderr: 98200K .......... .......... .......... .......... .......... 58% 16.1M 4s 2024-03-29T04:19:20.617 INFO:teuthology.orchestra.run.smithi022.stderr: 98250K .......... .......... .......... .......... .......... 58% 16.1M 4s 2024-03-29T04:19:20.619 INFO:teuthology.orchestra.run.smithi022.stderr: 98300K .......... .......... .......... .......... .......... 58% 21.1M 4s 2024-03-29T04:19:20.620 INFO:teuthology.orchestra.run.smithi022.stderr: 98350K .......... .......... .......... .......... .......... 58% 27.7M 4s 2024-03-29T04:19:20.622 INFO:teuthology.orchestra.run.smithi022.stderr: 98400K .......... .......... .......... .......... .......... 58% 30.6M 4s 2024-03-29T04:19:20.624 INFO:teuthology.orchestra.run.smithi022.stderr: 98450K .......... .......... .......... .......... .......... 58% 30.7M 4s 2024-03-29T04:19:20.626 INFO:teuthology.orchestra.run.smithi022.stderr: 98500K .......... .......... .......... .......... .......... 58% 20.0M 4s 2024-03-29T04:19:20.630 INFO:teuthology.orchestra.run.smithi022.stderr: 98550K .......... .......... .......... .......... .......... 58% 14.9M 4s 2024-03-29T04:19:20.632 INFO:teuthology.orchestra.run.smithi022.stderr: 98600K .......... .......... .......... .......... .......... 58% 20.7M 4s 2024-03-29T04:19:20.634 INFO:teuthology.orchestra.run.smithi022.stderr: 98650K .......... .......... .......... .......... .......... 58% 26.4M 4s 2024-03-29T04:19:20.635 INFO:teuthology.orchestra.run.smithi022.stderr: 98700K .......... .......... .......... .......... .......... 58% 26.7M 4s 2024-03-29T04:19:20.638 INFO:teuthology.orchestra.run.smithi022.stderr: 98750K .......... .......... .......... .......... .......... 58% 23.3M 4s 2024-03-29T04:19:20.639 INFO:teuthology.orchestra.run.smithi022.stderr: 98800K .......... .......... .......... .......... .......... 58% 26.1M 4s 2024-03-29T04:19:20.641 INFO:teuthology.orchestra.run.smithi022.stderr: 98850K .......... .......... .......... .......... .......... 58% 27.8M 4s 2024-03-29T04:19:20.643 INFO:teuthology.orchestra.run.smithi022.stderr: 98900K .......... .......... .......... .......... .......... 58% 22.9M 4s 2024-03-29T04:19:20.645 INFO:teuthology.orchestra.run.smithi022.stderr: 98950K .......... .......... .......... .......... .......... 58% 23.5M 4s 2024-03-29T04:19:20.648 INFO:teuthology.orchestra.run.smithi022.stderr: 99000K .......... .......... .......... .......... .......... 58% 24.7M 4s 2024-03-29T04:19:20.649 INFO:teuthology.orchestra.run.smithi022.stderr: 99050K .......... .......... .......... .......... .......... 58% 27.5M 4s 2024-03-29T04:19:20.651 INFO:teuthology.orchestra.run.smithi022.stderr: 99100K .......... .......... .......... .......... .......... 58% 24.7M 4s 2024-03-29T04:19:20.653 INFO:teuthology.orchestra.run.smithi022.stderr: 99150K .......... .......... .......... .......... .......... 58% 23.2M 4s 2024-03-29T04:19:20.655 INFO:teuthology.orchestra.run.smithi022.stderr: 99200K .......... .......... .......... .......... .......... 58% 26.7M 4s 2024-03-29T04:19:20.657 INFO:teuthology.orchestra.run.smithi022.stderr: 99250K .......... .......... .......... .......... .......... 58% 25.8M 4s 2024-03-29T04:19:20.659 INFO:teuthology.orchestra.run.smithi022.stderr: 99300K .......... .......... .......... .......... .......... 58% 24.0M 4s 2024-03-29T04:19:20.661 INFO:teuthology.orchestra.run.smithi022.stderr: 99350K .......... .......... .......... .......... .......... 58% 22.4M 4s 2024-03-29T04:19:20.663 INFO:teuthology.orchestra.run.smithi022.stderr: 99400K .......... .......... .......... .......... .......... 58% 25.6M 4s 2024-03-29T04:19:20.665 INFO:teuthology.orchestra.run.smithi022.stderr: 99450K .......... .......... .......... .......... .......... 59% 26.0M 4s 2024-03-29T04:19:20.667 INFO:teuthology.orchestra.run.smithi022.stderr: 99500K .......... .......... .......... .......... .......... 59% 25.6M 4s 2024-03-29T04:19:20.669 INFO:teuthology.orchestra.run.smithi022.stderr: 99550K .......... .......... .......... .......... .......... 59% 21.2M 4s 2024-03-29T04:19:20.672 INFO:teuthology.orchestra.run.smithi022.stderr: 99600K .......... .......... .......... .......... .......... 59% 18.4M 4s 2024-03-29T04:19:20.674 INFO:teuthology.orchestra.run.smithi022.stderr: 99650K .......... .......... .......... .......... .......... 59% 21.2M 4s 2024-03-29T04:19:20.677 INFO:teuthology.orchestra.run.smithi022.stderr: 99700K .......... .......... .......... .......... .......... 59% 21.0M 4s 2024-03-29T04:19:20.679 INFO:teuthology.orchestra.run.smithi022.stderr: 99750K .......... .......... .......... .......... .......... 59% 18.3M 4s 2024-03-29T04:19:20.682 INFO:teuthology.orchestra.run.smithi022.stderr: 99800K .......... .......... .......... .......... .......... 59% 21.3M 4s 2024-03-29T04:19:20.684 INFO:teuthology.orchestra.run.smithi022.stderr: 99850K .......... .......... .......... .......... .......... 59% 21.1M 4s 2024-03-29T04:19:20.686 INFO:teuthology.orchestra.run.smithi022.stderr: 99900K .......... .......... .......... .......... .......... 59% 19.4M 4s 2024-03-29T04:19:20.690 INFO:teuthology.orchestra.run.smithi022.stderr: 99950K .......... .......... .......... .......... .......... 59% 15.1M 4s 2024-03-29T04:19:20.693 INFO:teuthology.orchestra.run.smithi022.stderr:100000K .......... .......... .......... .......... .......... 59% 17.0M 4s 2024-03-29T04:19:20.696 INFO:teuthology.orchestra.run.smithi022.stderr:100050K .......... .......... .......... .......... .......... 59% 16.8M 4s 2024-03-29T04:19:20.703 INFO:teuthology.orchestra.run.smithi022.stderr:100100K .......... .......... .......... .......... .......... 59% 15.6M 4s 2024-03-29T04:19:20.704 INFO:teuthology.orchestra.run.smithi022.stderr:100150K .......... .......... .......... .......... .......... 59% 13.4M 4s 2024-03-29T04:19:20.706 INFO:teuthology.orchestra.run.smithi022.stderr:100200K .......... .......... .......... .......... .......... 59% 15.8M 4s 2024-03-29T04:19:20.708 INFO:teuthology.orchestra.run.smithi022.stderr:100250K .......... .......... .......... .......... .......... 59% 16.4M 4s 2024-03-29T04:19:20.711 INFO:teuthology.orchestra.run.smithi022.stderr:100300K .......... .......... .......... .......... .......... 59% 18.0M 4s 2024-03-29T04:19:20.714 INFO:teuthology.orchestra.run.smithi022.stderr:100350K .......... .......... .......... .......... .......... 59% 15.7M 4s 2024-03-29T04:19:20.717 INFO:teuthology.orchestra.run.smithi022.stderr:100400K .......... .......... .......... .......... .......... 59% 18.9M 4s 2024-03-29T04:19:20.719 INFO:teuthology.orchestra.run.smithi022.stderr:100450K .......... .......... .......... .......... .......... 59% 18.0M 4s 2024-03-29T04:19:20.721 INFO:teuthology.orchestra.run.smithi022.stderr:100500K .......... .......... .......... .......... .......... 59% 22.9M 4s 2024-03-29T04:19:20.724 INFO:teuthology.orchestra.run.smithi022.stderr:100550K .......... .......... .......... .......... .......... 59% 21.1M 4s 2024-03-29T04:19:20.726 INFO:teuthology.orchestra.run.smithi022.stderr:100600K .......... .......... .......... .......... .......... 59% 23.3M 4s 2024-03-29T04:19:20.729 INFO:teuthology.orchestra.run.smithi022.stderr:100650K .......... .......... .......... .......... .......... 59% 14.9M 4s 2024-03-29T04:19:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:100700K .......... .......... .......... .......... .......... 59% 18.3M 4s 2024-03-29T04:19:20.735 INFO:teuthology.orchestra.run.smithi022.stderr:100750K .......... .......... .......... .......... .......... 59% 16.4M 4s 2024-03-29T04:19:20.737 INFO:teuthology.orchestra.run.smithi022.stderr:100800K .......... .......... .......... .......... .......... 59% 19.7M 4s 2024-03-29T04:19:20.740 INFO:teuthology.orchestra.run.smithi022.stderr:100850K .......... .......... .......... .......... .......... 59% 19.2M 4s 2024-03-29T04:19:20.743 INFO:teuthology.orchestra.run.smithi022.stderr:100900K .......... .......... .......... .......... .......... 59% 19.2M 4s 2024-03-29T04:19:20.745 INFO:teuthology.orchestra.run.smithi022.stderr:100950K .......... .......... .......... .......... .......... 59% 16.4M 4s 2024-03-29T04:19:20.748 INFO:teuthology.orchestra.run.smithi022.stderr:101000K .......... .......... .......... .......... .......... 59% 19.5M 4s 2024-03-29T04:19:20.751 INFO:teuthology.orchestra.run.smithi022.stderr:101050K .......... .......... .......... .......... .......... 59% 18.6M 4s 2024-03-29T04:19:20.753 INFO:teuthology.orchestra.run.smithi022.stderr:101100K .......... .......... .......... .......... .......... 60% 18.3M 4s 2024-03-29T04:19:20.756 INFO:teuthology.orchestra.run.smithi022.stderr:101150K .......... .......... .......... .......... .......... 60% 16.2M 4s 2024-03-29T04:19:20.759 INFO:teuthology.orchestra.run.smithi022.stderr:101200K .......... .......... .......... .......... .......... 60% 18.5M 4s 2024-03-29T04:19:20.762 INFO:teuthology.orchestra.run.smithi022.stderr:101250K .......... .......... .......... .......... .......... 60% 18.8M 4s 2024-03-29T04:19:20.764 INFO:teuthology.orchestra.run.smithi022.stderr:101300K .......... .......... .......... .......... .......... 60% 20.4M 4s 2024-03-29T04:19:20.766 INFO:teuthology.orchestra.run.smithi022.stderr:101350K .......... .......... .......... .......... .......... 60% 18.4M 4s 2024-03-29T04:19:20.769 INFO:teuthology.orchestra.run.smithi022.stderr:101400K .......... .......... .......... .......... .......... 60% 21.0M 4s 2024-03-29T04:19:20.771 INFO:teuthology.orchestra.run.smithi022.stderr:101450K .......... .......... .......... .......... .......... 60% 20.0M 4s 2024-03-29T04:19:20.774 INFO:teuthology.orchestra.run.smithi022.stderr:101500K .......... .......... .......... .......... .......... 60% 20.7M 4s 2024-03-29T04:19:20.776 INFO:teuthology.orchestra.run.smithi022.stderr:101550K .......... .......... .......... .......... .......... 60% 17.6M 4s 2024-03-29T04:19:20.779 INFO:teuthology.orchestra.run.smithi022.stderr:101600K .......... .......... .......... .......... .......... 60% 20.8M 4s 2024-03-29T04:19:20.781 INFO:teuthology.orchestra.run.smithi022.stderr:101650K .......... .......... .......... .......... .......... 60% 20.8M 4s 2024-03-29T04:19:20.783 INFO:teuthology.orchestra.run.smithi022.stderr:101700K .......... .......... .......... .......... .......... 60% 20.0M 4s 2024-03-29T04:19:20.786 INFO:teuthology.orchestra.run.smithi022.stderr:101750K .......... .......... .......... .......... .......... 60% 17.8M 4s 2024-03-29T04:19:20.789 INFO:teuthology.orchestra.run.smithi022.stderr:101800K .......... .......... .......... .......... .......... 60% 20.5M 4s 2024-03-29T04:19:20.791 INFO:teuthology.orchestra.run.smithi022.stderr:101850K .......... .......... .......... .......... .......... 60% 20.6M 4s 2024-03-29T04:19:20.793 INFO:teuthology.orchestra.run.smithi022.stderr:101900K .......... .......... .......... .......... .......... 60% 19.9M 4s 2024-03-29T04:19:20.796 INFO:teuthology.orchestra.run.smithi022.stderr:101950K .......... .......... .......... .......... .......... 60% 17.6M 4s 2024-03-29T04:19:20.799 INFO:teuthology.orchestra.run.smithi022.stderr:102000K .......... .......... .......... .......... .......... 60% 21.0M 4s 2024-03-29T04:19:20.801 INFO:teuthology.orchestra.run.smithi022.stderr:102050K .......... .......... .......... .......... .......... 60% 20.7M 4s 2024-03-29T04:19:20.803 INFO:teuthology.orchestra.run.smithi022.stderr:102100K .......... .......... .......... .......... .......... 60% 20.2M 4s 2024-03-29T04:19:20.806 INFO:teuthology.orchestra.run.smithi022.stderr:102150K .......... .......... .......... .......... .......... 60% 17.9M 4s 2024-03-29T04:19:20.809 INFO:teuthology.orchestra.run.smithi022.stderr:102200K .......... .......... .......... .......... .......... 60% 19.6M 4s 2024-03-29T04:19:20.812 INFO:teuthology.orchestra.run.smithi022.stderr:102250K .......... .......... .......... .......... .......... 60% 14.7M 4s 2024-03-29T04:19:20.816 INFO:teuthology.orchestra.run.smithi022.stderr:102300K .......... .......... .......... .......... .......... 60% 12.7M 4s 2024-03-29T04:19:20.820 INFO:teuthology.orchestra.run.smithi022.stderr:102350K .......... .......... .......... .......... .......... 60% 11.2M 4s 2024-03-29T04:19:20.823 INFO:teuthology.orchestra.run.smithi022.stderr:102400K .......... .......... .......... .......... .......... 60% 16.3M 4s 2024-03-29T04:19:20.827 INFO:teuthology.orchestra.run.smithi022.stderr:102450K .......... .......... .......... .......... .......... 60% 14.0M 4s 2024-03-29T04:19:20.830 INFO:teuthology.orchestra.run.smithi022.stderr:102500K .......... .......... .......... .......... .......... 60% 16.2M 4s 2024-03-29T04:19:20.833 INFO:teuthology.orchestra.run.smithi022.stderr:102550K .......... .......... .......... .......... .......... 60% 16.7M 4s 2024-03-29T04:19:20.835 INFO:teuthology.orchestra.run.smithi022.stderr:102600K .......... .......... .......... .......... .......... 60% 18.0M 4s 2024-03-29T04:19:20.837 INFO:teuthology.orchestra.run.smithi022.stderr:102650K .......... .......... .......... .......... .......... 60% 22.7M 4s 2024-03-29T04:19:20.839 INFO:teuthology.orchestra.run.smithi022.stderr:102700K .......... .......... .......... .......... .......... 60% 24.1M 4s 2024-03-29T04:19:20.841 INFO:teuthology.orchestra.run.smithi022.stderr:102750K .......... .......... .......... .......... .......... 60% 24.1M 4s 2024-03-29T04:19:20.843 INFO:teuthology.orchestra.run.smithi022.stderr:102800K .......... .......... .......... .......... .......... 61% 28.3M 4s 2024-03-29T04:19:20.845 INFO:teuthology.orchestra.run.smithi022.stderr:102850K .......... .......... .......... .......... .......... 61% 28.4M 4s 2024-03-29T04:19:20.847 INFO:teuthology.orchestra.run.smithi022.stderr:102900K .......... .......... .......... .......... .......... 61% 26.6M 4s 2024-03-29T04:19:20.849 INFO:teuthology.orchestra.run.smithi022.stderr:102950K .......... .......... .......... .......... .......... 61% 20.6M 4s 2024-03-29T04:19:20.851 INFO:teuthology.orchestra.run.smithi022.stderr:103000K .......... .......... .......... .......... .......... 61% 24.4M 4s 2024-03-29T04:19:20.853 INFO:teuthology.orchestra.run.smithi022.stderr:103050K .......... .......... .......... .......... .......... 61% 24.6M 4s 2024-03-29T04:19:20.855 INFO:teuthology.orchestra.run.smithi022.stderr:103100K .......... .......... .......... .......... .......... 61% 22.8M 4s 2024-03-29T04:19:20.857 INFO:teuthology.orchestra.run.smithi022.stderr:103150K .......... .......... .......... .......... .......... 61% 20.1M 4s 2024-03-29T04:19:20.859 INFO:teuthology.orchestra.run.smithi022.stderr:103200K .......... .......... .......... .......... .......... 61% 24.3M 4s 2024-03-29T04:19:20.861 INFO:teuthology.orchestra.run.smithi022.stderr:103250K .......... .......... .......... .......... .......... 61% 27.2M 4s 2024-03-29T04:19:20.863 INFO:teuthology.orchestra.run.smithi022.stderr:103300K .......... .......... .......... .......... .......... 61% 26.2M 4s 2024-03-29T04:19:20.865 INFO:teuthology.orchestra.run.smithi022.stderr:103350K .......... .......... .......... .......... .......... 61% 22.4M 4s 2024-03-29T04:19:20.867 INFO:teuthology.orchestra.run.smithi022.stderr:103400K .......... .......... .......... .......... .......... 61% 26.8M 4s 2024-03-29T04:19:20.869 INFO:teuthology.orchestra.run.smithi022.stderr:103450K .......... .......... .......... .......... .......... 61% 26.5M 4s 2024-03-29T04:19:20.871 INFO:teuthology.orchestra.run.smithi022.stderr:103500K .......... .......... .......... .......... .......... 61% 26.4M 4s 2024-03-29T04:19:20.873 INFO:teuthology.orchestra.run.smithi022.stderr:103550K .......... .......... .......... .......... .......... 61% 22.7M 4s 2024-03-29T04:19:20.875 INFO:teuthology.orchestra.run.smithi022.stderr:103600K .......... .......... .......... .......... .......... 61% 24.3M 4s 2024-03-29T04:19:20.877 INFO:teuthology.orchestra.run.smithi022.stderr:103650K .......... .......... .......... .......... .......... 61% 24.6M 4s 2024-03-29T04:19:20.879 INFO:teuthology.orchestra.run.smithi022.stderr:103700K .......... .......... .......... .......... .......... 61% 24.5M 4s 2024-03-29T04:19:20.881 INFO:teuthology.orchestra.run.smithi022.stderr:103750K .......... .......... .......... .......... .......... 61% 21.0M 4s 2024-03-29T04:19:20.883 INFO:teuthology.orchestra.run.smithi022.stderr:103800K .......... .......... .......... .......... .......... 61% 24.4M 4s 2024-03-29T04:19:20.885 INFO:teuthology.orchestra.run.smithi022.stderr:103850K .......... .......... .......... .......... .......... 61% 23.5M 4s 2024-03-29T04:19:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:103900K .......... .......... .......... .......... .......... 61% 24.2M 4s 2024-03-29T04:19:20.890 INFO:teuthology.orchestra.run.smithi022.stderr:103950K .......... .......... .......... .......... .......... 61% 21.4M 4s 2024-03-29T04:19:20.892 INFO:teuthology.orchestra.run.smithi022.stderr:104000K .......... .......... .......... .......... .......... 61% 24.0M 4s 2024-03-29T04:19:20.894 INFO:teuthology.orchestra.run.smithi022.stderr:104050K .......... .......... .......... .......... .......... 61% 24.8M 4s 2024-03-29T04:19:20.896 INFO:teuthology.orchestra.run.smithi022.stderr:104100K .......... .......... .......... .......... .......... 61% 21.4M 4s 2024-03-29T04:19:20.898 INFO:teuthology.orchestra.run.smithi022.stderr:104150K .......... .......... .......... .......... .......... 61% 20.4M 4s 2024-03-29T04:19:20.900 INFO:teuthology.orchestra.run.smithi022.stderr:104200K .......... .......... .......... .......... .......... 61% 24.7M 4s 2024-03-29T04:19:20.902 INFO:teuthology.orchestra.run.smithi022.stderr:104250K .......... .......... .......... .......... .......... 61% 25.6M 4s 2024-03-29T04:19:20.904 INFO:teuthology.orchestra.run.smithi022.stderr:104300K .......... .......... .......... .......... .......... 61% 24.6M 4s 2024-03-29T04:19:20.907 INFO:teuthology.orchestra.run.smithi022.stderr:104350K .......... .......... .......... .......... .......... 61% 19.4M 4s 2024-03-29T04:19:20.909 INFO:teuthology.orchestra.run.smithi022.stderr:104400K .......... .......... .......... .......... .......... 61% 24.2M 4s 2024-03-29T04:19:20.911 INFO:teuthology.orchestra.run.smithi022.stderr:104450K .......... .......... .......... .......... .......... 61% 22.7M 4s 2024-03-29T04:19:20.913 INFO:teuthology.orchestra.run.smithi022.stderr:104500K .......... .......... .......... .......... .......... 62% 22.2M 4s 2024-03-29T04:19:20.916 INFO:teuthology.orchestra.run.smithi022.stderr:104550K .......... .......... .......... .......... .......... 62% 18.2M 4s 2024-03-29T04:19:20.918 INFO:teuthology.orchestra.run.smithi022.stderr:104600K .......... .......... .......... .......... .......... 62% 22.1M 4s 2024-03-29T04:19:20.920 INFO:teuthology.orchestra.run.smithi022.stderr:104650K .......... .......... .......... .......... .......... 62% 21.7M 4s 2024-03-29T04:19:20.923 INFO:teuthology.orchestra.run.smithi022.stderr:104700K .......... .......... .......... .......... .......... 62% 21.2M 4s 2024-03-29T04:19:20.925 INFO:teuthology.orchestra.run.smithi022.stderr:104750K .......... .......... .......... .......... .......... 62% 18.5M 4s 2024-03-29T04:19:20.927 INFO:teuthology.orchestra.run.smithi022.stderr:104800K .......... .......... .......... .......... .......... 62% 22.3M 4s 2024-03-29T04:19:20.929 INFO:teuthology.orchestra.run.smithi022.stderr:104850K .......... .......... .......... .......... .......... 62% 23.5M 4s 2024-03-29T04:19:20.932 INFO:teuthology.orchestra.run.smithi022.stderr:104900K .......... .......... .......... .......... .......... 62% 22.8M 4s 2024-03-29T04:19:20.934 INFO:teuthology.orchestra.run.smithi022.stderr:104950K .......... .......... .......... .......... .......... 62% 20.3M 4s 2024-03-29T04:19:20.936 INFO:teuthology.orchestra.run.smithi022.stderr:105000K .......... .......... .......... .......... .......... 62% 22.9M 4s 2024-03-29T04:19:20.938 INFO:teuthology.orchestra.run.smithi022.stderr:105050K .......... .......... .......... .......... .......... 62% 23.2M 4s 2024-03-29T04:19:20.940 INFO:teuthology.orchestra.run.smithi022.stderr:105100K .......... .......... .......... .......... .......... 62% 23.2M 4s 2024-03-29T04:19:20.943 INFO:teuthology.orchestra.run.smithi022.stderr:105150K .......... .......... .......... .......... .......... 62% 19.8M 4s 2024-03-29T04:19:20.945 INFO:teuthology.orchestra.run.smithi022.stderr:105200K .......... .......... .......... .......... .......... 62% 23.2M 4s 2024-03-29T04:19:20.947 INFO:teuthology.orchestra.run.smithi022.stderr:105250K .......... .......... .......... .......... .......... 62% 22.6M 4s 2024-03-29T04:19:20.949 INFO:teuthology.orchestra.run.smithi022.stderr:105300K .......... .......... .......... .......... .......... 62% 22.9M 4s 2024-03-29T04:19:20.952 INFO:teuthology.orchestra.run.smithi022.stderr:105350K .......... .......... .......... .......... .......... 62% 19.2M 4s 2024-03-29T04:19:20.954 INFO:teuthology.orchestra.run.smithi022.stderr:105400K .......... .......... .......... .......... .......... 62% 22.5M 4s 2024-03-29T04:19:20.956 INFO:teuthology.orchestra.run.smithi022.stderr:105450K .......... .......... .......... .......... .......... 62% 22.5M 4s 2024-03-29T04:19:20.958 INFO:teuthology.orchestra.run.smithi022.stderr:105500K .......... .......... .......... .......... .......... 62% 23.8M 4s 2024-03-29T04:19:20.961 INFO:teuthology.orchestra.run.smithi022.stderr:105550K .......... .......... .......... .......... .......... 62% 18.1M 4s 2024-03-29T04:19:20.963 INFO:teuthology.orchestra.run.smithi022.stderr:105600K .......... .......... .......... .......... .......... 62% 21.2M 4s 2024-03-29T04:19:20.966 INFO:teuthology.orchestra.run.smithi022.stderr:105650K .......... .......... .......... .......... .......... 62% 20.4M 4s 2024-03-29T04:19:20.968 INFO:teuthology.orchestra.run.smithi022.stderr:105700K .......... .......... .......... .......... .......... 62% 19.7M 4s 2024-03-29T04:19:20.971 INFO:teuthology.orchestra.run.smithi022.stderr:105750K .......... .......... .......... .......... .......... 62% 17.3M 4s 2024-03-29T04:19:20.973 INFO:teuthology.orchestra.run.smithi022.stderr:105800K .......... .......... .......... .......... .......... 62% 20.3M 4s 2024-03-29T04:19:20.976 INFO:teuthology.orchestra.run.smithi022.stderr:105850K .......... .......... .......... .......... .......... 62% 19.9M 4s 2024-03-29T04:19:20.979 INFO:teuthology.orchestra.run.smithi022.stderr:105900K .......... .......... .......... .......... .......... 62% 19.1M 4s 2024-03-29T04:19:20.982 INFO:teuthology.orchestra.run.smithi022.stderr:105950K .......... .......... .......... .......... .......... 62% 13.3M 4s 2024-03-29T04:19:20.985 INFO:teuthology.orchestra.run.smithi022.stderr:106000K .......... .......... .......... .......... .......... 62% 16.9M 4s 2024-03-29T04:19:20.988 INFO:teuthology.orchestra.run.smithi022.stderr:106050K .......... .......... .......... .......... .......... 62% 17.4M 4s 2024-03-29T04:19:20.990 INFO:teuthology.orchestra.run.smithi022.stderr:106100K .......... .......... .......... .......... .......... 62% 20.0M 4s 2024-03-29T04:19:20.993 INFO:teuthology.orchestra.run.smithi022.stderr:106150K .......... .......... .......... .......... .......... 63% 18.7M 4s 2024-03-29T04:19:20.995 INFO:teuthology.orchestra.run.smithi022.stderr:106200K .......... .......... .......... .......... .......... 63% 22.3M 4s 2024-03-29T04:19:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:106250K .......... .......... .......... .......... .......... 63% 22.4M 4s 2024-03-29T04:19:20.999 INFO:teuthology.orchestra.run.smithi022.stderr:106300K .......... .......... .......... .......... .......... 63% 21.6M 4s 2024-03-29T04:19:21.002 INFO:teuthology.orchestra.run.smithi022.stderr:106350K .......... .......... .......... .......... .......... 63% 18.7M 4s 2024-03-29T04:19:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:106400K .......... .......... .......... .......... .......... 63% 21.8M 4s 2024-03-29T04:19:21.006 INFO:teuthology.orchestra.run.smithi022.stderr:106450K .......... .......... .......... .......... .......... 63% 22.3M 4s 2024-03-29T04:19:21.008 INFO:teuthology.orchestra.run.smithi022.stderr:106500K .......... .......... .......... .......... .......... 63% 24.7M 4s 2024-03-29T04:19:21.011 INFO:teuthology.orchestra.run.smithi022.stderr:106550K .......... .......... .......... .......... .......... 63% 23.7M 4s 2024-03-29T04:19:21.013 INFO:teuthology.orchestra.run.smithi022.stderr:106600K .......... .......... .......... .......... .......... 63% 20.7M 4s 2024-03-29T04:19:21.015 INFO:teuthology.orchestra.run.smithi022.stderr:106650K .......... .......... .......... .......... .......... 63% 23.9M 4s 2024-03-29T04:19:21.016 INFO:teuthology.orchestra.run.smithi022.stderr:106700K .......... .......... .......... .......... .......... 63% 32.4M 4s 2024-03-29T04:19:21.018 INFO:teuthology.orchestra.run.smithi022.stderr:106750K .......... .......... .......... .......... .......... 63% 28.8M 4s 2024-03-29T04:19:21.019 INFO:teuthology.orchestra.run.smithi022.stderr:106800K .......... .......... .......... .......... .......... 63% 31.5M 4s 2024-03-29T04:19:21.021 INFO:teuthology.orchestra.run.smithi022.stderr:106850K .......... .......... .......... .......... .......... 63% 32.4M 4s 2024-03-29T04:19:21.023 INFO:teuthology.orchestra.run.smithi022.stderr:106900K .......... .......... .......... .......... .......... 63% 32.9M 4s 2024-03-29T04:19:21.024 INFO:teuthology.orchestra.run.smithi022.stderr:106950K .......... .......... .......... .......... .......... 63% 28.1M 4s 2024-03-29T04:19:21.026 INFO:teuthology.orchestra.run.smithi022.stderr:107000K .......... .......... .......... .......... .......... 63% 32.7M 4s 2024-03-29T04:19:21.027 INFO:teuthology.orchestra.run.smithi022.stderr:107050K .......... .......... .......... .......... .......... 63% 31.1M 4s 2024-03-29T04:19:21.029 INFO:teuthology.orchestra.run.smithi022.stderr:107100K .......... .......... .......... .......... .......... 63% 31.5M 4s 2024-03-29T04:19:21.031 INFO:teuthology.orchestra.run.smithi022.stderr:107150K .......... .......... .......... .......... .......... 63% 26.4M 4s 2024-03-29T04:19:21.033 INFO:teuthology.orchestra.run.smithi022.stderr:107200K .......... .......... .......... .......... .......... 63% 27.4M 4s 2024-03-29T04:19:21.034 INFO:teuthology.orchestra.run.smithi022.stderr:107250K .......... .......... .......... .......... .......... 63% 28.3M 4s 2024-03-29T04:19:21.036 INFO:teuthology.orchestra.run.smithi022.stderr:107300K .......... .......... .......... .......... .......... 63% 25.9M 4s 2024-03-29T04:19:21.038 INFO:teuthology.orchestra.run.smithi022.stderr:107350K .......... .......... .......... .......... .......... 63% 22.4M 4s 2024-03-29T04:19:21.040 INFO:teuthology.orchestra.run.smithi022.stderr:107400K .......... .......... .......... .......... .......... 63% 25.6M 4s 2024-03-29T04:19:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:107450K .......... .......... .......... .......... .......... 63% 24.6M 4s 2024-03-29T04:19:21.044 INFO:teuthology.orchestra.run.smithi022.stderr:107500K .......... .......... .......... .......... .......... 63% 32.1M 3s 2024-03-29T04:19:21.046 INFO:teuthology.orchestra.run.smithi022.stderr:107550K .......... .......... .......... .......... .......... 63% 28.3M 3s 2024-03-29T04:19:21.047 INFO:teuthology.orchestra.run.smithi022.stderr:107600K .......... .......... .......... .......... .......... 63% 32.3M 3s 2024-03-29T04:19:21.048 INFO:teuthology.orchestra.run.smithi022.stderr:107650K .......... .......... .......... .......... .......... 63% 32.2M 3s 2024-03-29T04:19:21.050 INFO:teuthology.orchestra.run.smithi022.stderr:107700K .......... .......... .......... .......... .......... 63% 31.3M 3s 2024-03-29T04:19:21.052 INFO:teuthology.orchestra.run.smithi022.stderr:107750K .......... .......... .......... .......... .......... 63% 26.5M 3s 2024-03-29T04:19:21.054 INFO:teuthology.orchestra.run.smithi022.stderr:107800K .......... .......... .......... .......... .......... 63% 32.2M 3s 2024-03-29T04:19:21.055 INFO:teuthology.orchestra.run.smithi022.stderr:107850K .......... .......... .......... .......... .......... 64% 31.6M 3s 2024-03-29T04:19:21.056 INFO:teuthology.orchestra.run.smithi022.stderr:107900K .......... .......... .......... .......... .......... 64% 31.8M 3s 2024-03-29T04:19:21.059 INFO:teuthology.orchestra.run.smithi022.stderr:107950K .......... .......... .......... .......... .......... 64% 24.0M 3s 2024-03-29T04:19:21.061 INFO:teuthology.orchestra.run.smithi022.stderr:108000K .......... .......... .......... .......... .......... 64% 26.5M 3s 2024-03-29T04:19:21.062 INFO:teuthology.orchestra.run.smithi022.stderr:108050K .......... .......... .......... .......... .......... 64% 27.8M 3s 2024-03-29T04:19:21.064 INFO:teuthology.orchestra.run.smithi022.stderr:108100K .......... .......... .......... .......... .......... 64% 26.9M 3s 2024-03-29T04:19:21.066 INFO:teuthology.orchestra.run.smithi022.stderr:108150K .......... .......... .......... .......... .......... 64% 24.2M 3s 2024-03-29T04:19:21.068 INFO:teuthology.orchestra.run.smithi022.stderr:108200K .......... .......... .......... .......... .......... 64% 28.6M 3s 2024-03-29T04:19:21.069 INFO:teuthology.orchestra.run.smithi022.stderr:108250K .......... .......... .......... .......... .......... 64% 29.1M 3s 2024-03-29T04:19:21.071 INFO:teuthology.orchestra.run.smithi022.stderr:108300K .......... .......... .......... .......... .......... 64% 27.6M 3s 2024-03-29T04:19:21.073 INFO:teuthology.orchestra.run.smithi022.stderr:108350K .......... .......... .......... .......... .......... 64% 23.8M 3s 2024-03-29T04:19:21.075 INFO:teuthology.orchestra.run.smithi022.stderr:108400K .......... .......... .......... .......... .......... 64% 28.0M 3s 2024-03-29T04:19:21.077 INFO:teuthology.orchestra.run.smithi022.stderr:108450K .......... .......... .......... .......... .......... 64% 24.0M 3s 2024-03-29T04:19:21.079 INFO:teuthology.orchestra.run.smithi022.stderr:108500K .......... .......... .......... .......... .......... 64% 19.7M 3s 2024-03-29T04:19:21.082 INFO:teuthology.orchestra.run.smithi022.stderr:108550K .......... .......... .......... .......... .......... 64% 17.3M 3s 2024-03-29T04:19:21.085 INFO:teuthology.orchestra.run.smithi022.stderr:108600K .......... .......... .......... .......... .......... 64% 20.8M 3s 2024-03-29T04:19:21.087 INFO:teuthology.orchestra.run.smithi022.stderr:108650K .......... .......... .......... .......... .......... 64% 20.9M 3s 2024-03-29T04:19:21.089 INFO:teuthology.orchestra.run.smithi022.stderr:108700K .......... .......... .......... .......... .......... 64% 21.0M 3s 2024-03-29T04:19:21.092 INFO:teuthology.orchestra.run.smithi022.stderr:108750K .......... .......... .......... .......... .......... 64% 17.3M 3s 2024-03-29T04:19:21.095 INFO:teuthology.orchestra.run.smithi022.stderr:108800K .......... .......... .......... .......... .......... 64% 20.7M 3s 2024-03-29T04:19:21.097 INFO:teuthology.orchestra.run.smithi022.stderr:108850K .......... .......... .......... .......... .......... 64% 20.5M 3s 2024-03-29T04:19:21.100 INFO:teuthology.orchestra.run.smithi022.stderr:108900K .......... .......... .......... .......... .......... 64% 18.8M 3s 2024-03-29T04:19:21.103 INFO:teuthology.orchestra.run.smithi022.stderr:108950K .......... .......... .......... .......... .......... 64% 14.9M 3s 2024-03-29T04:19:21.105 INFO:teuthology.orchestra.run.smithi022.stderr:109000K .......... .......... .......... .......... .......... 64% 25.0M 3s 2024-03-29T04:19:21.107 INFO:teuthology.orchestra.run.smithi022.stderr:109050K .......... .......... .......... .......... .......... 64% 20.4M 3s 2024-03-29T04:19:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:109100K .......... .......... .......... .......... .......... 64% 19.3M 3s 2024-03-29T04:19:21.113 INFO:teuthology.orchestra.run.smithi022.stderr:109150K .......... .......... .......... .......... .......... 64% 16.9M 3s 2024-03-29T04:19:21.115 INFO:teuthology.orchestra.run.smithi022.stderr:109200K .......... .......... .......... .......... .......... 64% 19.9M 3s 2024-03-29T04:19:21.118 INFO:teuthology.orchestra.run.smithi022.stderr:109250K .......... .......... .......... .......... .......... 64% 19.2M 3s 2024-03-29T04:19:21.120 INFO:teuthology.orchestra.run.smithi022.stderr:109300K .......... .......... .......... .......... .......... 64% 18.8M 3s 2024-03-29T04:19:21.123 INFO:teuthology.orchestra.run.smithi022.stderr:109350K .......... .......... .......... .......... .......... 64% 15.3M 3s 2024-03-29T04:19:21.126 INFO:teuthology.orchestra.run.smithi022.stderr:109400K .......... .......... .......... .......... .......... 64% 19.0M 3s 2024-03-29T04:19:21.128 INFO:teuthology.orchestra.run.smithi022.stderr:109450K .......... .......... .......... .......... .......... 64% 20.8M 3s 2024-03-29T04:19:21.130 INFO:teuthology.orchestra.run.smithi022.stderr:109500K .......... .......... .......... .......... .......... 64% 20.9M 3s 2024-03-29T04:19:21.133 INFO:teuthology.orchestra.run.smithi022.stderr:109550K .......... .......... .......... .......... .......... 65% 17.6M 3s 2024-03-29T04:19:21.136 INFO:teuthology.orchestra.run.smithi022.stderr:109600K .......... .......... .......... .......... .......... 65% 20.6M 3s 2024-03-29T04:19:21.138 INFO:teuthology.orchestra.run.smithi022.stderr:109650K .......... .......... .......... .......... .......... 65% 20.9M 3s 2024-03-29T04:19:21.140 INFO:teuthology.orchestra.run.smithi022.stderr:109700K .......... .......... .......... .......... .......... 65% 20.7M 3s 2024-03-29T04:19:21.143 INFO:teuthology.orchestra.run.smithi022.stderr:109750K .......... .......... .......... .......... .......... 65% 17.5M 3s 2024-03-29T04:19:21.146 INFO:teuthology.orchestra.run.smithi022.stderr:109800K .......... .......... .......... .......... .......... 65% 20.6M 3s 2024-03-29T04:19:21.148 INFO:teuthology.orchestra.run.smithi022.stderr:109850K .......... .......... .......... .......... .......... 65% 20.2M 3s 2024-03-29T04:19:21.150 INFO:teuthology.orchestra.run.smithi022.stderr:109900K .......... .......... .......... .......... .......... 65% 20.3M 3s 2024-03-29T04:19:21.153 INFO:teuthology.orchestra.run.smithi022.stderr:109950K .......... .......... .......... .......... .......... 65% 17.3M 3s 2024-03-29T04:19:21.156 INFO:teuthology.orchestra.run.smithi022.stderr:110000K .......... .......... .......... .......... .......... 65% 20.0M 3s 2024-03-29T04:19:21.158 INFO:teuthology.orchestra.run.smithi022.stderr:110050K .......... .......... .......... .......... .......... 65% 20.4M 3s 2024-03-29T04:19:21.161 INFO:teuthology.orchestra.run.smithi022.stderr:110100K .......... .......... .......... .......... .......... 65% 19.1M 3s 2024-03-29T04:19:21.164 INFO:teuthology.orchestra.run.smithi022.stderr:110150K .......... .......... .......... .......... .......... 65% 15.8M 3s 2024-03-29T04:19:21.166 INFO:teuthology.orchestra.run.smithi022.stderr:110200K .......... .......... .......... .......... .......... 65% 19.1M 3s 2024-03-29T04:19:21.169 INFO:teuthology.orchestra.run.smithi022.stderr:110250K .......... .......... .......... .......... .......... 65% 18.7M 3s 2024-03-29T04:19:21.172 INFO:teuthology.orchestra.run.smithi022.stderr:110300K .......... .......... .......... .......... .......... 65% 18.6M 3s 2024-03-29T04:19:21.175 INFO:teuthology.orchestra.run.smithi022.stderr:110350K .......... .......... .......... .......... .......... 65% 14.4M 3s 2024-03-29T04:19:21.178 INFO:teuthology.orchestra.run.smithi022.stderr:110400K .......... .......... .......... .......... .......... 65% 16.6M 3s 2024-03-29T04:19:21.181 INFO:teuthology.orchestra.run.smithi022.stderr:110450K .......... .......... .......... .......... .......... 65% 16.8M 3s 2024-03-29T04:19:21.184 INFO:teuthology.orchestra.run.smithi022.stderr:110500K .......... .......... .......... .......... .......... 65% 16.6M 3s 2024-03-29T04:19:21.188 INFO:teuthology.orchestra.run.smithi022.stderr:110550K .......... .......... .......... .......... .......... 65% 14.4M 3s 2024-03-29T04:19:21.191 INFO:teuthology.orchestra.run.smithi022.stderr:110600K .......... .......... .......... .......... .......... 65% 15.5M 3s 2024-03-29T04:19:21.193 INFO:teuthology.orchestra.run.smithi022.stderr:110650K .......... .......... .......... .......... .......... 65% 19.9M 3s 2024-03-29T04:19:21.195 INFO:teuthology.orchestra.run.smithi022.stderr:110700K .......... .......... .......... .......... .......... 65% 19.8M 3s 2024-03-29T04:19:21.198 INFO:teuthology.orchestra.run.smithi022.stderr:110750K .......... .......... .......... .......... .......... 65% 18.6M 3s 2024-03-29T04:19:21.200 INFO:teuthology.orchestra.run.smithi022.stderr:110800K .......... .......... .......... .......... .......... 65% 21.8M 3s 2024-03-29T04:19:21.202 INFO:teuthology.orchestra.run.smithi022.stderr:110850K .......... .......... .......... .......... .......... 65% 22.3M 3s 2024-03-29T04:19:21.205 INFO:teuthology.orchestra.run.smithi022.stderr:110900K .......... .......... .......... .......... .......... 65% 20.4M 3s 2024-03-29T04:19:21.207 INFO:teuthology.orchestra.run.smithi022.stderr:110950K .......... .......... .......... .......... .......... 65% 18.6M 3s 2024-03-29T04:19:21.210 INFO:teuthology.orchestra.run.smithi022.stderr:111000K .......... .......... .......... .......... .......... 65% 21.2M 3s 2024-03-29T04:19:21.212 INFO:teuthology.orchestra.run.smithi022.stderr:111050K .......... .......... .......... .......... .......... 65% 20.4M 3s 2024-03-29T04:19:21.214 INFO:teuthology.orchestra.run.smithi022.stderr:111100K .......... .......... .......... .......... .......... 65% 21.8M 3s 2024-03-29T04:19:21.217 INFO:teuthology.orchestra.run.smithi022.stderr:111150K .......... .......... .......... .......... .......... 65% 16.7M 3s 2024-03-29T04:19:21.219 INFO:teuthology.orchestra.run.smithi022.stderr:111200K .......... .......... .......... .......... .......... 65% 21.2M 3s 2024-03-29T04:19:21.222 INFO:teuthology.orchestra.run.smithi022.stderr:111250K .......... .......... .......... .......... .......... 66% 21.7M 3s 2024-03-29T04:19:21.224 INFO:teuthology.orchestra.run.smithi022.stderr:111300K .......... .......... .......... .......... .......... 66% 20.9M 3s 2024-03-29T04:19:21.227 INFO:teuthology.orchestra.run.smithi022.stderr:111350K .......... .......... .......... .......... .......... 66% 17.5M 3s 2024-03-29T04:19:21.230 INFO:teuthology.orchestra.run.smithi022.stderr:111400K .......... .......... .......... .......... .......... 66% 16.2M 3s 2024-03-29T04:19:21.233 INFO:teuthology.orchestra.run.smithi022.stderr:111450K .......... .......... .......... .......... .......... 66% 15.5M 3s 2024-03-29T04:19:21.236 INFO:teuthology.orchestra.run.smithi022.stderr:111500K .......... .......... .......... .......... .......... 66% 16.8M 3s 2024-03-29T04:19:21.239 INFO:teuthology.orchestra.run.smithi022.stderr:111550K .......... .......... .......... .......... .......... 66% 14.7M 3s 2024-03-29T04:19:21.242 INFO:teuthology.orchestra.run.smithi022.stderr:111600K .......... .......... .......... .......... .......... 66% 15.6M 3s 2024-03-29T04:19:21.246 INFO:teuthology.orchestra.run.smithi022.stderr:111650K .......... .......... .......... .......... .......... 66% 15.6M 3s 2024-03-29T04:19:21.249 INFO:teuthology.orchestra.run.smithi022.stderr:111700K .......... .......... .......... .......... .......... 66% 14.2M 3s 2024-03-29T04:19:21.253 INFO:teuthology.orchestra.run.smithi022.stderr:111750K .......... .......... .......... .......... .......... 66% 13.9M 3s 2024-03-29T04:19:21.255 INFO:teuthology.orchestra.run.smithi022.stderr:111800K .......... .......... .......... .......... .......... 66% 24.8M 3s 2024-03-29T04:19:21.256 INFO:teuthology.orchestra.run.smithi022.stderr:111850K .......... .......... .......... .......... .......... 66% 26.7M 3s 2024-03-29T04:19:21.258 INFO:teuthology.orchestra.run.smithi022.stderr:111900K .......... .......... .......... .......... .......... 66% 27.1M 3s 2024-03-29T04:19:21.260 INFO:teuthology.orchestra.run.smithi022.stderr:111950K .......... .......... .......... .......... .......... 66% 23.3M 3s 2024-03-29T04:19:21.262 INFO:teuthology.orchestra.run.smithi022.stderr:112000K .......... .......... .......... .......... .......... 66% 26.9M 3s 2024-03-29T04:19:21.264 INFO:teuthology.orchestra.run.smithi022.stderr:112050K .......... .......... .......... .......... .......... 66% 26.3M 3s 2024-03-29T04:19:21.266 INFO:teuthology.orchestra.run.smithi022.stderr:112100K .......... .......... .......... .......... .......... 66% 26.2M 3s 2024-03-29T04:19:21.268 INFO:teuthology.orchestra.run.smithi022.stderr:112150K .......... .......... .......... .......... .......... 66% 22.1M 3s 2024-03-29T04:19:21.270 INFO:teuthology.orchestra.run.smithi022.stderr:112200K .......... .......... .......... .......... .......... 66% 26.5M 3s 2024-03-29T04:19:21.272 INFO:teuthology.orchestra.run.smithi022.stderr:112250K .......... .......... .......... .......... .......... 66% 26.1M 3s 2024-03-29T04:19:21.273 INFO:teuthology.orchestra.run.smithi022.stderr:112300K .......... .......... .......... .......... .......... 66% 26.1M 3s 2024-03-29T04:19:21.276 INFO:teuthology.orchestra.run.smithi022.stderr:112350K .......... .......... .......... .......... .......... 66% 21.9M 3s 2024-03-29T04:19:21.278 INFO:teuthology.orchestra.run.smithi022.stderr:112400K .......... .......... .......... .......... .......... 66% 24.9M 3s 2024-03-29T04:19:21.280 INFO:teuthology.orchestra.run.smithi022.stderr:112450K .......... .......... .......... .......... .......... 66% 25.3M 3s 2024-03-29T04:19:21.281 INFO:teuthology.orchestra.run.smithi022.stderr:112500K .......... .......... .......... .......... .......... 66% 25.6M 3s 2024-03-29T04:19:21.284 INFO:teuthology.orchestra.run.smithi022.stderr:112550K .......... .......... .......... .......... .......... 66% 21.6M 3s 2024-03-29T04:19:21.286 INFO:teuthology.orchestra.run.smithi022.stderr:112600K .......... .......... .......... .......... .......... 66% 25.6M 3s 2024-03-29T04:19:21.288 INFO:teuthology.orchestra.run.smithi022.stderr:112650K .......... .......... .......... .......... .......... 66% 24.5M 3s 2024-03-29T04:19:21.290 INFO:teuthology.orchestra.run.smithi022.stderr:112700K .......... .......... .......... .......... .......... 66% 25.6M 3s 2024-03-29T04:19:21.292 INFO:teuthology.orchestra.run.smithi022.stderr:112750K .......... .......... .......... .......... .......... 66% 21.4M 3s 2024-03-29T04:19:21.294 INFO:teuthology.orchestra.run.smithi022.stderr:112800K .......... .......... .......... .......... .......... 66% 24.9M 3s 2024-03-29T04:19:21.296 INFO:teuthology.orchestra.run.smithi022.stderr:112850K .......... .......... .......... .......... .......... 66% 24.4M 3s 2024-03-29T04:19:21.298 INFO:teuthology.orchestra.run.smithi022.stderr:112900K .......... .......... .......... .......... .......... 67% 24.6M 3s 2024-03-29T04:19:21.300 INFO:teuthology.orchestra.run.smithi022.stderr:112950K .......... .......... .......... .......... .......... 67% 21.1M 3s 2024-03-29T04:19:21.302 INFO:teuthology.orchestra.run.smithi022.stderr:113000K .......... .......... .......... .......... .......... 67% 24.7M 3s 2024-03-29T04:19:21.304 INFO:teuthology.orchestra.run.smithi022.stderr:113050K .......... .......... .......... .......... .......... 67% 24.4M 3s 2024-03-29T04:19:21.306 INFO:teuthology.orchestra.run.smithi022.stderr:113100K .......... .......... .......... .......... .......... 67% 24.4M 3s 2024-03-29T04:19:21.308 INFO:teuthology.orchestra.run.smithi022.stderr:113150K .......... .......... .......... .......... .......... 67% 20.7M 3s 2024-03-29T04:19:21.310 INFO:teuthology.orchestra.run.smithi022.stderr:113200K .......... .......... .......... .......... .......... 67% 25.4M 3s 2024-03-29T04:19:21.316 INFO:teuthology.orchestra.run.smithi022.stderr:113250K .......... .......... .......... .......... .......... 67% 25.1M 3s 2024-03-29T04:19:21.316 INFO:teuthology.orchestra.run.smithi022.stderr:113300K .......... .......... .......... .......... .......... 67% 25.2M 3s 2024-03-29T04:19:21.317 INFO:teuthology.orchestra.run.smithi022.stderr:113350K .......... .......... .......... .......... .......... 67% 20.6M 3s 2024-03-29T04:19:21.319 INFO:teuthology.orchestra.run.smithi022.stderr:113400K .......... .......... .......... .......... .......... 67% 23.7M 3s 2024-03-29T04:19:21.321 INFO:teuthology.orchestra.run.smithi022.stderr:113450K .......... .......... .......... .......... .......... 67% 24.0M 3s 2024-03-29T04:19:21.323 INFO:teuthology.orchestra.run.smithi022.stderr:113500K .......... .......... .......... .......... .......... 67% 24.1M 3s 2024-03-29T04:19:21.325 INFO:teuthology.orchestra.run.smithi022.stderr:113550K .......... .......... .......... .......... .......... 67% 18.5M 3s 2024-03-29T04:19:21.328 INFO:teuthology.orchestra.run.smithi022.stderr:113600K .......... .......... .......... .......... .......... 67% 17.9M 3s 2024-03-29T04:19:21.331 INFO:teuthology.orchestra.run.smithi022.stderr:113650K .......... .......... .......... .......... .......... 67% 20.3M 3s 2024-03-29T04:19:21.333 INFO:teuthology.orchestra.run.smithi022.stderr:113700K .......... .......... .......... .......... .......... 67% 20.0M 3s 2024-03-29T04:19:21.336 INFO:teuthology.orchestra.run.smithi022.stderr:113750K .......... .......... .......... .......... .......... 67% 15.9M 3s 2024-03-29T04:19:21.339 INFO:teuthology.orchestra.run.smithi022.stderr:113800K .......... .......... .......... .......... .......... 67% 17.7M 3s 2024-03-29T04:19:21.342 INFO:teuthology.orchestra.run.smithi022.stderr:113850K .......... .......... .......... .......... .......... 67% 18.1M 3s 2024-03-29T04:19:21.344 INFO:teuthology.orchestra.run.smithi022.stderr:113900K .......... .......... .......... .......... .......... 67% 18.0M 3s 2024-03-29T04:19:21.347 INFO:teuthology.orchestra.run.smithi022.stderr:113950K .......... .......... .......... .......... .......... 67% 15.8M 3s 2024-03-29T04:19:21.350 INFO:teuthology.orchestra.run.smithi022.stderr:114000K .......... .......... .......... .......... .......... 67% 16.8M 3s 2024-03-29T04:19:21.353 INFO:teuthology.orchestra.run.smithi022.stderr:114050K .......... .......... .......... .......... .......... 67% 16.0M 3s 2024-03-29T04:19:21.356 INFO:teuthology.orchestra.run.smithi022.stderr:114100K .......... .......... .......... .......... .......... 67% 15.8M 3s 2024-03-29T04:19:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:114150K .......... .......... .......... .......... .......... 67% 13.3M 3s 2024-03-29T04:19:21.364 INFO:teuthology.orchestra.run.smithi022.stderr:114200K .......... .......... .......... .......... .......... 67% 14.3M 3s 2024-03-29T04:19:21.368 INFO:teuthology.orchestra.run.smithi022.stderr:114250K .......... .......... .......... .......... .......... 67% 11.1M 3s 2024-03-29T04:19:21.373 INFO:teuthology.orchestra.run.smithi022.stderr:114300K .......... .......... .......... .......... .......... 67% 10.1M 3s 2024-03-29T04:19:21.378 INFO:teuthology.orchestra.run.smithi022.stderr:114350K .......... .......... .......... .......... .......... 67% 9.57M 3s 2024-03-29T04:19:21.382 INFO:teuthology.orchestra.run.smithi022.stderr:114400K .......... .......... .......... .......... .......... 67% 11.8M 3s 2024-03-29T04:19:21.386 INFO:teuthology.orchestra.run.smithi022.stderr:114450K .......... .......... .......... .......... .......... 67% 12.0M 3s 2024-03-29T04:19:21.391 INFO:teuthology.orchestra.run.smithi022.stderr:114500K .......... .......... .......... .......... .......... 67% 10.4M 3s 2024-03-29T04:19:21.396 INFO:teuthology.orchestra.run.smithi022.stderr:114550K .......... .......... .......... .......... .......... 67% 9.18M 3s 2024-03-29T04:19:21.402 INFO:teuthology.orchestra.run.smithi022.stderr:114600K .......... .......... .......... .......... .......... 68% 9.26M 3s 2024-03-29T04:19:21.407 INFO:teuthology.orchestra.run.smithi022.stderr:114650K .......... .......... .......... .......... .......... 68% 9.29M 3s 2024-03-29T04:19:21.411 INFO:teuthology.orchestra.run.smithi022.stderr:114700K .......... .......... .......... .......... .......... 68% 12.4M 3s 2024-03-29T04:19:21.415 INFO:teuthology.orchestra.run.smithi022.stderr:114750K .......... .......... .......... .......... .......... 68% 11.2M 3s 2024-03-29T04:19:21.419 INFO:teuthology.orchestra.run.smithi022.stderr:114800K .......... .......... .......... .......... .......... 68% 10.7M 3s 2024-03-29T04:19:21.423 INFO:teuthology.orchestra.run.smithi022.stderr:114850K .......... .......... .......... .......... .......... 68% 15.6M 3s 2024-03-29T04:19:21.426 INFO:teuthology.orchestra.run.smithi022.stderr:114900K .......... .......... .......... .......... .......... 68% 17.0M 3s 2024-03-29T04:19:21.429 INFO:teuthology.orchestra.run.smithi022.stderr:114950K .......... .......... .......... .......... .......... 68% 14.7M 3s 2024-03-29T04:19:21.432 INFO:teuthology.orchestra.run.smithi022.stderr:115000K .......... .......... .......... .......... .......... 68% 16.2M 3s 2024-03-29T04:19:21.435 INFO:teuthology.orchestra.run.smithi022.stderr:115050K .......... .......... .......... .......... .......... 68% 17.1M 3s 2024-03-29T04:19:21.438 INFO:teuthology.orchestra.run.smithi022.stderr:115100K .......... .......... .......... .......... .......... 68% 16.9M 3s 2024-03-29T04:19:21.441 INFO:teuthology.orchestra.run.smithi022.stderr:115150K .......... .......... .......... .......... .......... 68% 14.4M 3s 2024-03-29T04:19:21.444 INFO:teuthology.orchestra.run.smithi022.stderr:115200K .......... .......... .......... .......... .......... 68% 15.3M 3s 2024-03-29T04:19:21.447 INFO:teuthology.orchestra.run.smithi022.stderr:115250K .......... .......... .......... .......... .......... 68% 16.7M 3s 2024-03-29T04:19:21.450 INFO:teuthology.orchestra.run.smithi022.stderr:115300K .......... .......... .......... .......... .......... 68% 16.4M 3s 2024-03-29T04:19:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:115350K .......... .......... .......... .......... .......... 68% 12.7M 3s 2024-03-29T04:19:21.457 INFO:teuthology.orchestra.run.smithi022.stderr:115400K .......... .......... .......... .......... .......... 68% 16.5M 3s 2024-03-29T04:19:21.460 INFO:teuthology.orchestra.run.smithi022.stderr:115450K .......... .......... .......... .......... .......... 68% 16.1M 3s 2024-03-29T04:19:21.462 INFO:teuthology.orchestra.run.smithi022.stderr:115500K .......... .......... .......... .......... .......... 68% 19.0M 3s 2024-03-29T04:19:21.465 INFO:teuthology.orchestra.run.smithi022.stderr:115550K .......... .......... .......... .......... .......... 68% 17.3M 3s 2024-03-29T04:19:21.468 INFO:teuthology.orchestra.run.smithi022.stderr:115600K .......... .......... .......... .......... .......... 68% 20.0M 3s 2024-03-29T04:19:21.470 INFO:teuthology.orchestra.run.smithi022.stderr:115650K .......... .......... .......... .......... .......... 68% 20.0M 3s 2024-03-29T04:19:21.473 INFO:teuthology.orchestra.run.smithi022.stderr:115700K .......... .......... .......... .......... .......... 68% 19.6M 3s 2024-03-29T04:19:21.475 INFO:teuthology.orchestra.run.smithi022.stderr:115750K .......... .......... .......... .......... .......... 68% 16.9M 3s 2024-03-29T04:19:21.478 INFO:teuthology.orchestra.run.smithi022.stderr:115800K .......... .......... .......... .......... .......... 68% 20.0M 3s 2024-03-29T04:19:21.480 INFO:teuthology.orchestra.run.smithi022.stderr:115850K .......... .......... .......... .......... .......... 68% 19.8M 3s 2024-03-29T04:19:21.483 INFO:teuthology.orchestra.run.smithi022.stderr:115900K .......... .......... .......... .......... .......... 68% 19.9M 3s 2024-03-29T04:19:21.486 INFO:teuthology.orchestra.run.smithi022.stderr:115950K .......... .......... .......... .......... .......... 68% 17.2M 3s 2024-03-29T04:19:21.488 INFO:teuthology.orchestra.run.smithi022.stderr:116000K .......... .......... .......... .......... .......... 68% 19.5M 3s 2024-03-29T04:19:21.490 INFO:teuthology.orchestra.run.smithi022.stderr:116050K .......... .......... .......... .......... .......... 68% 20.2M 3s 2024-03-29T04:19:21.492 INFO:teuthology.orchestra.run.smithi022.stderr:116100K .......... .......... .......... .......... .......... 68% 25.8M 3s 2024-03-29T04:19:21.495 INFO:teuthology.orchestra.run.smithi022.stderr:116150K .......... .......... .......... .......... .......... 68% 21.7M 3s 2024-03-29T04:19:21.496 INFO:teuthology.orchestra.run.smithi022.stderr:116200K .......... .......... .......... .......... .......... 68% 25.6M 3s 2024-03-29T04:19:21.498 INFO:teuthology.orchestra.run.smithi022.stderr:116250K .......... .......... .......... .......... .......... 68% 25.8M 3s 2024-03-29T04:19:21.500 INFO:teuthology.orchestra.run.smithi022.stderr:116300K .......... .......... .......... .......... .......... 69% 24.6M 3s 2024-03-29T04:19:21.503 INFO:teuthology.orchestra.run.smithi022.stderr:116350K .......... .......... .......... .......... .......... 69% 21.4M 3s 2024-03-29T04:19:21.505 INFO:teuthology.orchestra.run.smithi022.stderr:116400K .......... .......... .......... .......... .......... 69% 24.7M 3s 2024-03-29T04:19:21.507 INFO:teuthology.orchestra.run.smithi022.stderr:116450K .......... .......... .......... .......... .......... 69% 24.2M 3s 2024-03-29T04:19:21.509 INFO:teuthology.orchestra.run.smithi022.stderr:116500K .......... .......... .......... .......... .......... 69% 25.1M 3s 2024-03-29T04:19:21.511 INFO:teuthology.orchestra.run.smithi022.stderr:116550K .......... .......... .......... .......... .......... 69% 20.8M 3s 2024-03-29T04:19:21.513 INFO:teuthology.orchestra.run.smithi022.stderr:116600K .......... .......... .......... .......... .......... 69% 24.3M 3s 2024-03-29T04:19:21.515 INFO:teuthology.orchestra.run.smithi022.stderr:116650K .......... .......... .......... .......... .......... 69% 22.0M 3s 2024-03-29T04:19:21.517 INFO:teuthology.orchestra.run.smithi022.stderr:116700K .......... .......... .......... .......... .......... 69% 24.1M 3s 2024-03-29T04:19:21.520 INFO:teuthology.orchestra.run.smithi022.stderr:116750K .......... .......... .......... .......... .......... 69% 20.3M 3s 2024-03-29T04:19:21.522 INFO:teuthology.orchestra.run.smithi022.stderr:116800K .......... .......... .......... .......... .......... 69% 22.7M 3s 2024-03-29T04:19:21.524 INFO:teuthology.orchestra.run.smithi022.stderr:116850K .......... .......... .......... .......... .......... 69% 22.9M 3s 2024-03-29T04:19:21.526 INFO:teuthology.orchestra.run.smithi022.stderr:116900K .......... .......... .......... .......... .......... 69% 23.5M 3s 2024-03-29T04:19:21.528 INFO:teuthology.orchestra.run.smithi022.stderr:116950K .......... .......... .......... .......... .......... 69% 19.8M 3s 2024-03-29T04:19:21.530 INFO:teuthology.orchestra.run.smithi022.stderr:117000K .......... .......... .......... .......... .......... 69% 23.6M 3s 2024-03-29T04:19:21.533 INFO:teuthology.orchestra.run.smithi022.stderr:117050K .......... .......... .......... .......... .......... 69% 23.4M 3s 2024-03-29T04:19:21.536 INFO:teuthology.orchestra.run.smithi022.stderr:117100K .......... .......... .......... .......... .......... 69% 19.5M 3s 2024-03-29T04:19:21.540 INFO:teuthology.orchestra.run.smithi022.stderr:117150K .......... .......... .......... .......... .......... 69% 10.4M 3s 2024-03-29T04:19:21.545 INFO:teuthology.orchestra.run.smithi022.stderr:117200K .......... .......... .......... .......... .......... 69% 11.0M 3s 2024-03-29T04:19:21.549 INFO:teuthology.orchestra.run.smithi022.stderr:117250K .......... .......... .......... .......... .......... 69% 11.7M 3s 2024-03-29T04:19:21.553 INFO:teuthology.orchestra.run.smithi022.stderr:117300K .......... .......... .......... .......... .......... 69% 12.7M 3s 2024-03-29T04:19:21.557 INFO:teuthology.orchestra.run.smithi022.stderr:117350K .......... .......... .......... .......... .......... 69% 11.1M 3s 2024-03-29T04:19:21.561 INFO:teuthology.orchestra.run.smithi022.stderr:117400K .......... .......... .......... .......... .......... 69% 12.1M 3s 2024-03-29T04:19:21.566 INFO:teuthology.orchestra.run.smithi022.stderr:117450K .......... .......... .......... .......... .......... 69% 11.5M 3s 2024-03-29T04:19:21.570 INFO:teuthology.orchestra.run.smithi022.stderr:117500K .......... .......... .......... .......... .......... 69% 11.3M 3s 2024-03-29T04:19:21.575 INFO:teuthology.orchestra.run.smithi022.stderr:117550K .......... .......... .......... .......... .......... 69% 9.25M 3s 2024-03-29T04:19:21.580 INFO:teuthology.orchestra.run.smithi022.stderr:117600K .......... .......... .......... .......... .......... 69% 9.78M 3s 2024-03-29T04:19:21.585 INFO:teuthology.orchestra.run.smithi022.stderr:117650K .......... .......... .......... .......... .......... 69% 10.5M 3s 2024-03-29T04:19:21.589 INFO:teuthology.orchestra.run.smithi022.stderr:117700K .......... .......... .......... .......... .......... 69% 12.2M 3s 2024-03-29T04:19:21.593 INFO:teuthology.orchestra.run.smithi022.stderr:117750K .......... .......... .......... .......... .......... 69% 12.0M 3s 2024-03-29T04:19:21.597 INFO:teuthology.orchestra.run.smithi022.stderr:117800K .......... .......... .......... .......... .......... 69% 12.8M 3s 2024-03-29T04:19:21.600 INFO:teuthology.orchestra.run.smithi022.stderr:117850K .......... .......... .......... .......... .......... 69% 14.1M 3s 2024-03-29T04:19:21.603 INFO:teuthology.orchestra.run.smithi022.stderr:117900K .......... .......... .......... .......... .......... 69% 13.8M 3s 2024-03-29T04:19:21.608 INFO:teuthology.orchestra.run.smithi022.stderr:117950K .......... .......... .......... .......... .......... 70% 11.6M 3s 2024-03-29T04:19:21.612 INFO:teuthology.orchestra.run.smithi022.stderr:118000K .......... .......... .......... .......... .......... 70% 12.9M 3s 2024-03-29T04:19:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:118050K .......... .......... .......... .......... .......... 70% 13.7M 3s 2024-03-29T04:19:21.618 INFO:teuthology.orchestra.run.smithi022.stderr:118100K .......... .......... .......... .......... .......... 70% 16.3M 3s 2024-03-29T04:19:21.621 INFO:teuthology.orchestra.run.smithi022.stderr:118150K .......... .......... .......... .......... .......... 70% 14.4M 3s 2024-03-29T04:19:21.624 INFO:teuthology.orchestra.run.smithi022.stderr:118200K .......... .......... .......... .......... .......... 70% 17.5M 3s 2024-03-29T04:19:21.627 INFO:teuthology.orchestra.run.smithi022.stderr:118250K .......... .......... .......... .......... .......... 70% 17.6M 3s 2024-03-29T04:19:21.630 INFO:teuthology.orchestra.run.smithi022.stderr:118300K .......... .......... .......... .......... .......... 70% 16.2M 3s 2024-03-29T04:19:21.633 INFO:teuthology.orchestra.run.smithi022.stderr:118350K .......... .......... .......... .......... .......... 70% 16.5M 3s 2024-03-29T04:19:21.635 INFO:teuthology.orchestra.run.smithi022.stderr:118400K .......... .......... .......... .......... .......... 70% 23.5M 3s 2024-03-29T04:19:21.637 INFO:teuthology.orchestra.run.smithi022.stderr:118450K .......... .......... .......... .......... .......... 70% 22.7M 3s 2024-03-29T04:19:21.639 INFO:teuthology.orchestra.run.smithi022.stderr:118500K .......... .......... .......... .......... .......... 70% 20.7M 3s 2024-03-29T04:19:21.642 INFO:teuthology.orchestra.run.smithi022.stderr:118550K .......... .......... .......... .......... .......... 70% 18.1M 3s 2024-03-29T04:19:21.644 INFO:teuthology.orchestra.run.smithi022.stderr:118600K .......... .......... .......... .......... .......... 70% 21.8M 3s 2024-03-29T04:19:21.646 INFO:teuthology.orchestra.run.smithi022.stderr:118650K .......... .......... .......... .......... .......... 70% 24.0M 3s 2024-03-29T04:19:21.648 INFO:teuthology.orchestra.run.smithi022.stderr:118700K .......... .......... .......... .......... .......... 70% 23.2M 3s 2024-03-29T04:19:21.651 INFO:teuthology.orchestra.run.smithi022.stderr:118750K .......... .......... .......... .......... .......... 70% 19.2M 3s 2024-03-29T04:19:21.652 INFO:teuthology.orchestra.run.smithi022.stderr:118800K .......... .......... .......... .......... .......... 70% 31.4M 3s 2024-03-29T04:19:21.654 INFO:teuthology.orchestra.run.smithi022.stderr:118850K .......... .......... .......... .......... .......... 70% 27.2M 3s 2024-03-29T04:19:21.656 INFO:teuthology.orchestra.run.smithi022.stderr:118900K .......... .......... .......... .......... .......... 70% 27.6M 3s 2024-03-29T04:19:21.658 INFO:teuthology.orchestra.run.smithi022.stderr:118950K .......... .......... .......... .......... .......... 70% 26.8M 3s 2024-03-29T04:19:21.659 INFO:teuthology.orchestra.run.smithi022.stderr:119000K .......... .......... .......... .......... .......... 70% 30.5M 3s 2024-03-29T04:19:21.661 INFO:teuthology.orchestra.run.smithi022.stderr:119050K .......... .......... .......... .......... .......... 70% 28.1M 3s 2024-03-29T04:19:21.664 INFO:teuthology.orchestra.run.smithi022.stderr:119100K .......... .......... .......... .......... .......... 70% 20.9M 3s 2024-03-29T04:19:21.666 INFO:teuthology.orchestra.run.smithi022.stderr:119150K .......... .......... .......... .......... .......... 70% 16.7M 3s 2024-03-29T04:19:21.669 INFO:teuthology.orchestra.run.smithi022.stderr:119200K .......... .......... .......... .......... .......... 70% 21.0M 3s 2024-03-29T04:19:21.672 INFO:teuthology.orchestra.run.smithi022.stderr:119250K .......... .......... .......... .......... .......... 70% 16.3M 3s 2024-03-29T04:19:21.674 INFO:teuthology.orchestra.run.smithi022.stderr:119300K .......... .......... .......... .......... .......... 70% 21.7M 3s 2024-03-29T04:19:21.676 INFO:teuthology.orchestra.run.smithi022.stderr:119350K .......... .......... .......... .......... .......... 70% 22.6M 3s 2024-03-29T04:19:21.678 INFO:teuthology.orchestra.run.smithi022.stderr:119400K .......... .......... .......... .......... .......... 70% 26.1M 3s 2024-03-29T04:19:21.680 INFO:teuthology.orchestra.run.smithi022.stderr:119450K .......... .......... .......... .......... .......... 70% 23.0M 3s 2024-03-29T04:19:21.682 INFO:teuthology.orchestra.run.smithi022.stderr:119500K .......... .......... .......... .......... .......... 70% 25.7M 3s 2024-03-29T04:19:21.684 INFO:teuthology.orchestra.run.smithi022.stderr:119550K .......... .......... .......... .......... .......... 70% 22.9M 3s 2024-03-29T04:19:21.686 INFO:teuthology.orchestra.run.smithi022.stderr:119600K .......... .......... .......... .......... .......... 70% 23.9M 3s 2024-03-29T04:19:21.688 INFO:teuthology.orchestra.run.smithi022.stderr:119650K .......... .......... .......... .......... .......... 71% 24.6M 3s 2024-03-29T04:19:21.690 INFO:teuthology.orchestra.run.smithi022.stderr:119700K .......... .......... .......... .......... .......... 71% 25.2M 3s 2024-03-29T04:19:21.692 INFO:teuthology.orchestra.run.smithi022.stderr:119750K .......... .......... .......... .......... .......... 71% 22.1M 3s 2024-03-29T04:19:21.695 INFO:teuthology.orchestra.run.smithi022.stderr:119800K .......... .......... .......... .......... .......... 71% 25.7M 3s 2024-03-29T04:19:21.697 INFO:teuthology.orchestra.run.smithi022.stderr:119850K .......... .......... .......... .......... .......... 71% 21.4M 3s 2024-03-29T04:19:21.699 INFO:teuthology.orchestra.run.smithi022.stderr:119900K .......... .......... .......... .......... .......... 71% 22.2M 3s 2024-03-29T04:19:21.701 INFO:teuthology.orchestra.run.smithi022.stderr:119950K .......... .......... .......... .......... .......... 71% 18.9M 3s 2024-03-29T04:19:21.703 INFO:teuthology.orchestra.run.smithi022.stderr:120000K .......... .......... .......... .......... .......... 71% 22.8M 3s 2024-03-29T04:19:21.706 INFO:teuthology.orchestra.run.smithi022.stderr:120050K .......... .......... .......... .......... .......... 71% 21.7M 3s 2024-03-29T04:19:21.708 INFO:teuthology.orchestra.run.smithi022.stderr:120100K .......... .......... .......... .......... .......... 71% 18.3M 3s 2024-03-29T04:19:21.711 INFO:teuthology.orchestra.run.smithi022.stderr:120150K .......... .......... .......... .......... .......... 71% 16.5M 3s 2024-03-29T04:19:21.714 INFO:teuthology.orchestra.run.smithi022.stderr:120200K .......... .......... .......... .......... .......... 71% 18.6M 3s 2024-03-29T04:19:21.717 INFO:teuthology.orchestra.run.smithi022.stderr:120250K .......... .......... .......... .......... .......... 71% 19.0M 3s 2024-03-29T04:19:21.723 INFO:teuthology.orchestra.run.smithi022.stderr:120300K .......... .......... .......... .......... .......... 71% 18.5M 3s 2024-03-29T04:19:21.723 INFO:teuthology.orchestra.run.smithi022.stderr:120350K .......... .......... .......... .......... .......... 71% 16.1M 3s 2024-03-29T04:19:21.725 INFO:teuthology.orchestra.run.smithi022.stderr:120400K .......... .......... .......... .......... .......... 71% 20.1M 3s 2024-03-29T04:19:21.727 INFO:teuthology.orchestra.run.smithi022.stderr:120450K .......... .......... .......... .......... .......... 71% 19.8M 3s 2024-03-29T04:19:21.730 INFO:teuthology.orchestra.run.smithi022.stderr:120500K .......... .......... .......... .......... .......... 71% 19.1M 3s 2024-03-29T04:19:21.733 INFO:teuthology.orchestra.run.smithi022.stderr:120550K .......... .......... .......... .......... .......... 71% 14.3M 3s 2024-03-29T04:19:21.736 INFO:teuthology.orchestra.run.smithi022.stderr:120600K .......... .......... .......... .......... .......... 71% 16.5M 3s 2024-03-29T04:19:21.739 INFO:teuthology.orchestra.run.smithi022.stderr:120650K .......... .......... .......... .......... .......... 71% 16.4M 3s 2024-03-29T04:19:21.743 INFO:teuthology.orchestra.run.smithi022.stderr:120700K .......... .......... .......... .......... .......... 71% 13.1M 3s 2024-03-29T04:19:21.747 INFO:teuthology.orchestra.run.smithi022.stderr:120750K .......... .......... .......... .......... .......... 71% 12.5M 3s 2024-03-29T04:19:21.750 INFO:teuthology.orchestra.run.smithi022.stderr:120800K .......... .......... .......... .......... .......... 71% 14.2M 3s 2024-03-29T04:19:21.754 INFO:teuthology.orchestra.run.smithi022.stderr:120850K .......... .......... .......... .......... .......... 71% 13.6M 3s 2024-03-29T04:19:21.757 INFO:teuthology.orchestra.run.smithi022.stderr:120900K .......... .......... .......... .......... .......... 71% 14.1M 3s 2024-03-29T04:19:21.761 INFO:teuthology.orchestra.run.smithi022.stderr:120950K .......... .......... .......... .......... .......... 71% 12.1M 3s 2024-03-29T04:19:21.764 INFO:teuthology.orchestra.run.smithi022.stderr:121000K .......... .......... .......... .......... .......... 71% 15.3M 3s 2024-03-29T04:19:21.768 INFO:teuthology.orchestra.run.smithi022.stderr:121050K .......... .......... .......... .......... .......... 71% 15.2M 3s 2024-03-29T04:19:21.771 INFO:teuthology.orchestra.run.smithi022.stderr:121100K .......... .......... .......... .......... .......... 71% 14.6M 3s 2024-03-29T04:19:21.775 INFO:teuthology.orchestra.run.smithi022.stderr:121150K .......... .......... .......... .......... .......... 71% 12.0M 3s 2024-03-29T04:19:21.779 INFO:teuthology.orchestra.run.smithi022.stderr:121200K .......... .......... .......... .......... .......... 71% 13.7M 3s 2024-03-29T04:19:21.783 INFO:teuthology.orchestra.run.smithi022.stderr:121250K .......... .......... .......... .......... .......... 71% 13.2M 3s 2024-03-29T04:19:21.787 INFO:teuthology.orchestra.run.smithi022.stderr:121300K .......... .......... .......... .......... .......... 71% 13.1M 3s 2024-03-29T04:19:21.791 INFO:teuthology.orchestra.run.smithi022.stderr:121350K .......... .......... .......... .......... .......... 72% 11.4M 3s 2024-03-29T04:19:21.794 INFO:teuthology.orchestra.run.smithi022.stderr:121400K .......... .......... .......... .......... .......... 72% 13.9M 3s 2024-03-29T04:19:21.797 INFO:teuthology.orchestra.run.smithi022.stderr:121450K .......... .......... .......... .......... .......... 72% 14.9M 3s 2024-03-29T04:19:21.801 INFO:teuthology.orchestra.run.smithi022.stderr:121500K .......... .......... .......... .......... .......... 72% 14.8M 3s 2024-03-29T04:19:21.804 INFO:teuthology.orchestra.run.smithi022.stderr:121550K .......... .......... .......... .......... .......... 72% 12.3M 3s 2024-03-29T04:19:21.808 INFO:teuthology.orchestra.run.smithi022.stderr:121600K .......... .......... .......... .......... .......... 72% 14.4M 3s 2024-03-29T04:19:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:121650K .......... .......... .......... .......... .......... 72% 14.2M 3s 2024-03-29T04:19:21.815 INFO:teuthology.orchestra.run.smithi022.stderr:121700K .......... .......... .......... .......... .......... 72% 13.4M 3s 2024-03-29T04:19:21.820 INFO:teuthology.orchestra.run.smithi022.stderr:121750K .......... .......... .......... .......... .......... 72% 10.5M 3s 2024-03-29T04:19:21.824 INFO:teuthology.orchestra.run.smithi022.stderr:121800K .......... .......... .......... .......... .......... 72% 13.0M 3s 2024-03-29T04:19:21.827 INFO:teuthology.orchestra.run.smithi022.stderr:121850K .......... .......... .......... .......... .......... 72% 13.1M 3s 2024-03-29T04:19:21.831 INFO:teuthology.orchestra.run.smithi022.stderr:121900K .......... .......... .......... .......... .......... 72% 12.6M 3s 2024-03-29T04:19:21.836 INFO:teuthology.orchestra.run.smithi022.stderr:121950K .......... .......... .......... .......... .......... 72% 10.9M 3s 2024-03-29T04:19:21.840 INFO:teuthology.orchestra.run.smithi022.stderr:122000K .......... .......... .......... .......... .......... 72% 11.9M 3s 2024-03-29T04:19:21.844 INFO:teuthology.orchestra.run.smithi022.stderr:122050K .......... .......... .......... .......... .......... 72% 11.3M 3s 2024-03-29T04:19:21.848 INFO:teuthology.orchestra.run.smithi022.stderr:122100K .......... .......... .......... .......... .......... 72% 11.7M 3s 2024-03-29T04:19:21.853 INFO:teuthology.orchestra.run.smithi022.stderr:122150K .......... .......... .......... .......... .......... 72% 9.69M 3s 2024-03-29T04:19:21.858 INFO:teuthology.orchestra.run.smithi022.stderr:122200K .......... .......... .......... .......... .......... 72% 10.9M 3s 2024-03-29T04:19:21.860 INFO:teuthology.orchestra.run.smithi022.stderr:122250K .......... .......... .......... .......... .......... 72% 17.6M 3s 2024-03-29T04:19:21.863 INFO:teuthology.orchestra.run.smithi022.stderr:122300K .......... .......... .......... .......... .......... 72% 17.8M 3s 2024-03-29T04:19:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:122350K .......... .......... .......... .......... .......... 72% 13.4M 3s 2024-03-29T04:19:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:122400K .......... .......... .......... .......... .......... 72% 16.3M 3s 2024-03-29T04:19:21.873 INFO:teuthology.orchestra.run.smithi022.stderr:122450K .......... .......... .......... .......... .......... 72% 14.7M 3s 2024-03-29T04:19:21.876 INFO:teuthology.orchestra.run.smithi022.stderr:122500K .......... .......... .......... .......... .......... 72% 15.4M 3s 2024-03-29T04:19:21.879 INFO:teuthology.orchestra.run.smithi022.stderr:122550K .......... .......... .......... .......... .......... 72% 18.4M 3s 2024-03-29T04:19:21.881 INFO:teuthology.orchestra.run.smithi022.stderr:122600K .......... .......... .......... .......... .......... 72% 20.0M 3s 2024-03-29T04:19:21.884 INFO:teuthology.orchestra.run.smithi022.stderr:122650K .......... .......... .......... .......... .......... 72% 20.2M 3s 2024-03-29T04:19:21.886 INFO:teuthology.orchestra.run.smithi022.stderr:122700K .......... .......... .......... .......... .......... 72% 20.2M 3s 2024-03-29T04:19:21.889 INFO:teuthology.orchestra.run.smithi022.stderr:122750K .......... .......... .......... .......... .......... 72% 17.3M 3s 2024-03-29T04:19:21.891 INFO:teuthology.orchestra.run.smithi022.stderr:122800K .......... .......... .......... .......... .......... 72% 20.0M 3s 2024-03-29T04:19:21.894 INFO:teuthology.orchestra.run.smithi022.stderr:122850K .......... .......... .......... .......... .......... 72% 17.7M 3s 2024-03-29T04:19:21.896 INFO:teuthology.orchestra.run.smithi022.stderr:122900K .......... .......... .......... .......... .......... 72% 23.1M 3s 2024-03-29T04:19:21.899 INFO:teuthology.orchestra.run.smithi022.stderr:122950K .......... .......... .......... .......... .......... 72% 19.5M 3s 2024-03-29T04:19:21.901 INFO:teuthology.orchestra.run.smithi022.stderr:123000K .......... .......... .......... .......... .......... 72% 23.3M 3s 2024-03-29T04:19:21.903 INFO:teuthology.orchestra.run.smithi022.stderr:123050K .......... .......... .......... .......... .......... 73% 21.0M 3s 2024-03-29T04:19:21.905 INFO:teuthology.orchestra.run.smithi022.stderr:123100K .......... .......... .......... .......... .......... 73% 20.6M 3s 2024-03-29T04:19:21.908 INFO:teuthology.orchestra.run.smithi022.stderr:123150K .......... .......... .......... .......... .......... 73% 18.4M 3s 2024-03-29T04:19:21.910 INFO:teuthology.orchestra.run.smithi022.stderr:123200K .......... .......... .......... .......... .......... 73% 20.6M 3s 2024-03-29T04:19:21.913 INFO:teuthology.orchestra.run.smithi022.stderr:123250K .......... .......... .......... .......... .......... 73% 23.5M 3s 2024-03-29T04:19:21.914 INFO:teuthology.orchestra.run.smithi022.stderr:123300K .......... .......... .......... .......... .......... 73% 29.2M 3s 2024-03-29T04:19:21.916 INFO:teuthology.orchestra.run.smithi022.stderr:123350K .......... .......... .......... .......... .......... 73% 25.8M 3s 2024-03-29T04:19:21.918 INFO:teuthology.orchestra.run.smithi022.stderr:123400K .......... .......... .......... .......... .......... 73% 25.9M 3s 2024-03-29T04:19:21.920 INFO:teuthology.orchestra.run.smithi022.stderr:123450K .......... .......... .......... .......... .......... 73% 25.4M 3s 2024-03-29T04:19:21.922 INFO:teuthology.orchestra.run.smithi022.stderr:123500K .......... .......... .......... .......... .......... 73% 27.0M 3s 2024-03-29T04:19:21.928 INFO:teuthology.orchestra.run.smithi022.stderr:123550K .......... .......... .......... .......... .......... 73% 22.1M 3s 2024-03-29T04:19:21.928 INFO:teuthology.orchestra.run.smithi022.stderr:123600K .......... .......... .......... .......... .......... 73% 26.2M 3s 2024-03-29T04:19:21.928 INFO:teuthology.orchestra.run.smithi022.stderr:123650K .......... .......... .......... .......... .......... 73% 23.4M 3s 2024-03-29T04:19:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:123700K .......... .......... .......... .......... .......... 73% 21.4M 3s 2024-03-29T04:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:123750K .......... .......... .......... .......... .......... 73% 18.1M 3s 2024-03-29T04:19:21.935 INFO:teuthology.orchestra.run.smithi022.stderr:123800K .......... .......... .......... .......... .......... 73% 21.5M 3s 2024-03-29T04:19:21.937 INFO:teuthology.orchestra.run.smithi022.stderr:123850K .......... .......... .......... .......... .......... 73% 22.8M 3s 2024-03-29T04:19:21.939 INFO:teuthology.orchestra.run.smithi022.stderr:123900K .......... .......... .......... .......... .......... 73% 22.0M 3s 2024-03-29T04:19:21.942 INFO:teuthology.orchestra.run.smithi022.stderr:123950K .......... .......... .......... .......... .......... 73% 20.0M 3s 2024-03-29T04:19:21.944 INFO:teuthology.orchestra.run.smithi022.stderr:124000K .......... .......... .......... .......... .......... 73% 22.6M 3s 2024-03-29T04:19:21.947 INFO:teuthology.orchestra.run.smithi022.stderr:124050K .......... .......... .......... .......... .......... 73% 20.9M 3s 2024-03-29T04:19:21.949 INFO:teuthology.orchestra.run.smithi022.stderr:124100K .......... .......... .......... .......... .......... 73% 20.2M 3s 2024-03-29T04:19:21.951 INFO:teuthology.orchestra.run.smithi022.stderr:124150K .......... .......... .......... .......... .......... 73% 19.7M 3s 2024-03-29T04:19:21.954 INFO:teuthology.orchestra.run.smithi022.stderr:124200K .......... .......... .......... .......... .......... 73% 22.5M 3s 2024-03-29T04:19:21.956 INFO:teuthology.orchestra.run.smithi022.stderr:124250K .......... .......... .......... .......... .......... 73% 22.0M 3s 2024-03-29T04:19:21.958 INFO:teuthology.orchestra.run.smithi022.stderr:124300K .......... .......... .......... .......... .......... 73% 20.5M 3s 2024-03-29T04:19:21.961 INFO:teuthology.orchestra.run.smithi022.stderr:124350K .......... .......... .......... .......... .......... 73% 18.5M 3s 2024-03-29T04:19:21.963 INFO:teuthology.orchestra.run.smithi022.stderr:124400K .......... .......... .......... .......... .......... 73% 22.1M 3s 2024-03-29T04:19:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:124450K .......... .......... .......... .......... .......... 73% 21.8M 3s 2024-03-29T04:19:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:124500K .......... .......... .......... .......... .......... 73% 20.5M 3s 2024-03-29T04:19:21.970 INFO:teuthology.orchestra.run.smithi022.stderr:124550K .......... .......... .......... .......... .......... 73% 18.9M 3s 2024-03-29T04:19:21.972 INFO:teuthology.orchestra.run.smithi022.stderr:124600K .......... .......... .......... .......... .......... 73% 22.4M 2s 2024-03-29T04:19:21.974 INFO:teuthology.orchestra.run.smithi022.stderr:124650K .......... .......... .......... .......... .......... 73% 22.3M 2s 2024-03-29T04:19:21.977 INFO:teuthology.orchestra.run.smithi022.stderr:124700K .......... .......... .......... .......... .......... 74% 21.8M 2s 2024-03-29T04:19:21.979 INFO:teuthology.orchestra.run.smithi022.stderr:124750K .......... .......... .......... .......... .......... 74% 19.6M 2s 2024-03-29T04:19:21.981 INFO:teuthology.orchestra.run.smithi022.stderr:124800K .......... .......... .......... .......... .......... 74% 24.6M 2s 2024-03-29T04:19:21.983 INFO:teuthology.orchestra.run.smithi022.stderr:124850K .......... .......... .......... .......... .......... 74% 24.4M 2s 2024-03-29T04:19:21.985 INFO:teuthology.orchestra.run.smithi022.stderr:124900K .......... .......... .......... .......... .......... 74% 24.8M 2s 2024-03-29T04:19:21.988 INFO:teuthology.orchestra.run.smithi022.stderr:124950K .......... .......... .......... .......... .......... 74% 20.7M 2s 2024-03-29T04:19:21.990 INFO:teuthology.orchestra.run.smithi022.stderr:125000K .......... .......... .......... .......... .......... 74% 24.3M 2s 2024-03-29T04:19:21.992 INFO:teuthology.orchestra.run.smithi022.stderr:125050K .......... .......... .......... .......... .......... 74% 24.1M 2s 2024-03-29T04:19:21.993 INFO:teuthology.orchestra.run.smithi022.stderr:125100K .......... .......... .......... .......... .......... 74% 24.1M 2s 2024-03-29T04:19:21.996 INFO:teuthology.orchestra.run.smithi022.stderr:125150K .......... .......... .......... .......... .......... 74% 20.3M 2s 2024-03-29T04:19:21.998 INFO:teuthology.orchestra.run.smithi022.stderr:125200K .......... .......... .......... .......... .......... 74% 23.8M 2s 2024-03-29T04:19:22.000 INFO:teuthology.orchestra.run.smithi022.stderr:125250K .......... .......... .......... .......... .......... 74% 23.5M 2s 2024-03-29T04:19:22.002 INFO:teuthology.orchestra.run.smithi022.stderr:125300K .......... .......... .......... .......... .......... 74% 23.6M 2s 2024-03-29T04:19:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:125350K .......... .......... .......... .......... .......... 74% 18.5M 2s 2024-03-29T04:19:22.008 INFO:teuthology.orchestra.run.smithi022.stderr:125400K .......... .......... .......... .......... .......... 74% 18.2M 2s 2024-03-29T04:19:22.010 INFO:teuthology.orchestra.run.smithi022.stderr:125450K .......... .......... .......... .......... .......... 74% 18.3M 2s 2024-03-29T04:19:22.013 INFO:teuthology.orchestra.run.smithi022.stderr:125500K .......... .......... .......... .......... .......... 74% 18.2M 2s 2024-03-29T04:19:22.016 INFO:teuthology.orchestra.run.smithi022.stderr:125550K .......... .......... .......... .......... .......... 74% 16.4M 2s 2024-03-29T04:19:22.018 INFO:teuthology.orchestra.run.smithi022.stderr:125600K .......... .......... .......... .......... .......... 74% 21.1M 2s 2024-03-29T04:19:22.020 INFO:teuthology.orchestra.run.smithi022.stderr:125650K .......... .......... .......... .......... .......... 74% 21.6M 2s 2024-03-29T04:19:22.023 INFO:teuthology.orchestra.run.smithi022.stderr:125700K .......... .......... .......... .......... .......... 74% 21.3M 2s 2024-03-29T04:19:22.025 INFO:teuthology.orchestra.run.smithi022.stderr:125750K .......... .......... .......... .......... .......... 74% 18.4M 2s 2024-03-29T04:19:22.028 INFO:teuthology.orchestra.run.smithi022.stderr:125800K .......... .......... .......... .......... .......... 74% 21.0M 2s 2024-03-29T04:19:22.030 INFO:teuthology.orchestra.run.smithi022.stderr:125850K .......... .......... .......... .......... .......... 74% 21.5M 2s 2024-03-29T04:19:22.032 INFO:teuthology.orchestra.run.smithi022.stderr:125900K .......... .......... .......... .......... .......... 74% 20.3M 2s 2024-03-29T04:19:22.035 INFO:teuthology.orchestra.run.smithi022.stderr:125950K .......... .......... .......... .......... .......... 74% 17.8M 2s 2024-03-29T04:19:22.038 INFO:teuthology.orchestra.run.smithi022.stderr:126000K .......... .......... .......... .......... .......... 74% 20.4M 2s 2024-03-29T04:19:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:126050K .......... .......... .......... .......... .......... 74% 21.2M 2s 2024-03-29T04:19:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:126100K .......... .......... .......... .......... .......... 74% 20.7M 2s 2024-03-29T04:19:22.045 INFO:teuthology.orchestra.run.smithi022.stderr:126150K .......... .......... .......... .......... .......... 74% 17.9M 2s 2024-03-29T04:19:22.047 INFO:teuthology.orchestra.run.smithi022.stderr:126200K .......... .......... .......... .......... .......... 74% 20.5M 2s 2024-03-29T04:19:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:126250K .......... .......... .......... .......... .......... 74% 18.3M 2s 2024-03-29T04:19:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:126300K .......... .......... .......... .......... .......... 74% 17.2M 2s 2024-03-29T04:19:22.056 INFO:teuthology.orchestra.run.smithi022.stderr:126350K .......... .......... .......... .......... .......... 74% 14.3M 2s 2024-03-29T04:19:22.060 INFO:teuthology.orchestra.run.smithi022.stderr:126400K .......... .......... .......... .......... .......... 75% 15.1M 2s 2024-03-29T04:19:22.063 INFO:teuthology.orchestra.run.smithi022.stderr:126450K .......... .......... .......... .......... .......... 75% 15.4M 2s 2024-03-29T04:19:22.066 INFO:teuthology.orchestra.run.smithi022.stderr:126500K .......... .......... .......... .......... .......... 75% 15.6M 2s 2024-03-29T04:19:22.069 INFO:teuthology.orchestra.run.smithi022.stderr:126550K .......... .......... .......... .......... .......... 75% 14.3M 2s 2024-03-29T04:19:22.073 INFO:teuthology.orchestra.run.smithi022.stderr:126600K .......... .......... .......... .......... .......... 75% 14.8M 2s 2024-03-29T04:19:22.076 INFO:teuthology.orchestra.run.smithi022.stderr:126650K .......... .......... .......... .......... .......... 75% 15.4M 2s 2024-03-29T04:19:22.079 INFO:teuthology.orchestra.run.smithi022.stderr:126700K .......... .......... .......... .......... .......... 75% 14.5M 2s 2024-03-29T04:19:22.082 INFO:teuthology.orchestra.run.smithi022.stderr:126750K .......... .......... .......... .......... .......... 75% 14.7M 2s 2024-03-29T04:19:22.085 INFO:teuthology.orchestra.run.smithi022.stderr:126800K .......... .......... .......... .......... .......... 75% 17.3M 2s 2024-03-29T04:19:22.088 INFO:teuthology.orchestra.run.smithi022.stderr:126850K .......... .......... .......... .......... .......... 75% 16.9M 2s 2024-03-29T04:19:22.091 INFO:teuthology.orchestra.run.smithi022.stderr:126900K .......... .......... .......... .......... .......... 75% 17.5M 2s 2024-03-29T04:19:22.094 INFO:teuthology.orchestra.run.smithi022.stderr:126950K .......... .......... .......... .......... .......... 75% 15.5M 2s 2024-03-29T04:19:22.095 INFO:teuthology.orchestra.run.smithi022.stderr:127000K .......... .......... .......... .......... .......... 75% 31.0M 2s 2024-03-29T04:19:22.097 INFO:teuthology.orchestra.run.smithi022.stderr:127050K .......... .......... .......... .......... .......... 75% 29.7M 2s 2024-03-29T04:19:22.100 INFO:teuthology.orchestra.run.smithi022.stderr:127100K .......... .......... .......... .......... .......... 75% 19.8M 2s 2024-03-29T04:19:22.102 INFO:teuthology.orchestra.run.smithi022.stderr:127150K .......... .......... .......... .......... .......... 75% 17.4M 2s 2024-03-29T04:19:22.104 INFO:teuthology.orchestra.run.smithi022.stderr:127200K .......... .......... .......... .......... .......... 75% 24.2M 2s 2024-03-29T04:19:22.106 INFO:teuthology.orchestra.run.smithi022.stderr:127250K .......... .......... .......... .......... .......... 75% 26.4M 2s 2024-03-29T04:19:22.109 INFO:teuthology.orchestra.run.smithi022.stderr:127300K .......... .......... .......... .......... .......... 75% 24.3M 2s 2024-03-29T04:19:22.110 INFO:teuthology.orchestra.run.smithi022.stderr:127350K .......... .......... .......... .......... .......... 75% 22.6M 2s 2024-03-29T04:19:22.112 INFO:teuthology.orchestra.run.smithi022.stderr:127400K .......... .......... .......... .......... .......... 75% 28.9M 2s 2024-03-29T04:19:22.114 INFO:teuthology.orchestra.run.smithi022.stderr:127450K .......... .......... .......... .......... .......... 75% 30.7M 2s 2024-03-29T04:19:22.115 INFO:teuthology.orchestra.run.smithi022.stderr:127500K .......... .......... .......... .......... .......... 75% 29.7M 2s 2024-03-29T04:19:22.117 INFO:teuthology.orchestra.run.smithi022.stderr:127550K .......... .......... .......... .......... .......... 75% 25.1M 2s 2024-03-29T04:19:22.119 INFO:teuthology.orchestra.run.smithi022.stderr:127600K .......... .......... .......... .......... .......... 75% 30.2M 2s 2024-03-29T04:19:22.120 INFO:teuthology.orchestra.run.smithi022.stderr:127650K .......... .......... .......... .......... .......... 75% 30.3M 2s 2024-03-29T04:19:22.122 INFO:teuthology.orchestra.run.smithi022.stderr:127700K .......... .......... .......... .......... .......... 75% 30.0M 2s 2024-03-29T04:19:22.124 INFO:teuthology.orchestra.run.smithi022.stderr:127750K .......... .......... .......... .......... .......... 75% 25.6M 2s 2024-03-29T04:19:22.126 INFO:teuthology.orchestra.run.smithi022.stderr:127800K .......... .......... .......... .......... .......... 75% 29.7M 2s 2024-03-29T04:19:22.128 INFO:teuthology.orchestra.run.smithi022.stderr:127850K .......... .......... .......... .......... .......... 75% 29.3M 2s 2024-03-29T04:19:22.129 INFO:teuthology.orchestra.run.smithi022.stderr:127900K .......... .......... .......... .......... .......... 75% 29.7M 2s 2024-03-29T04:19:22.131 INFO:teuthology.orchestra.run.smithi022.stderr:127950K .......... .......... .......... .......... .......... 75% 24.6M 2s 2024-03-29T04:19:22.133 INFO:teuthology.orchestra.run.smithi022.stderr:128000K .......... .......... .......... .......... .......... 75% 28.8M 2s 2024-03-29T04:19:22.134 INFO:teuthology.orchestra.run.smithi022.stderr:128050K .......... .......... .......... .......... .......... 75% 28.8M 2s 2024-03-29T04:19:22.136 INFO:teuthology.orchestra.run.smithi022.stderr:128100K .......... .......... .......... .......... .......... 76% 28.4M 2s 2024-03-29T04:19:22.138 INFO:teuthology.orchestra.run.smithi022.stderr:128150K .......... .......... .......... .......... .......... 76% 24.4M 2s 2024-03-29T04:19:22.140 INFO:teuthology.orchestra.run.smithi022.stderr:128200K .......... .......... .......... .......... .......... 76% 28.5M 2s 2024-03-29T04:19:22.141 INFO:teuthology.orchestra.run.smithi022.stderr:128250K .......... .......... .......... .......... .......... 76% 28.3M 2s 2024-03-29T04:19:22.143 INFO:teuthology.orchestra.run.smithi022.stderr:128300K .......... .......... .......... .......... .......... 76% 27.9M 2s 2024-03-29T04:19:22.145 INFO:teuthology.orchestra.run.smithi022.stderr:128350K .......... .......... .......... .......... .......... 76% 24.3M 2s 2024-03-29T04:19:22.147 INFO:teuthology.orchestra.run.smithi022.stderr:128400K .......... .......... .......... .......... .......... 76% 27.8M 2s 2024-03-29T04:19:22.149 INFO:teuthology.orchestra.run.smithi022.stderr:128450K .......... .......... .......... .......... .......... 76% 27.6M 2s 2024-03-29T04:19:22.151 INFO:teuthology.orchestra.run.smithi022.stderr:128500K .......... .......... .......... .......... .......... 76% 27.5M 2s 2024-03-29T04:19:22.153 INFO:teuthology.orchestra.run.smithi022.stderr:128550K .......... .......... .......... .......... .......... 76% 23.0M 2s 2024-03-29T04:19:22.155 INFO:teuthology.orchestra.run.smithi022.stderr:128600K .......... .......... .......... .......... .......... 76% 25.9M 2s 2024-03-29T04:19:22.156 INFO:teuthology.orchestra.run.smithi022.stderr:128650K .......... .......... .......... .......... .......... 76% 27.4M 2s 2024-03-29T04:19:22.158 INFO:teuthology.orchestra.run.smithi022.stderr:128700K .......... .......... .......... .......... .......... 76% 26.8M 2s 2024-03-29T04:19:22.160 INFO:teuthology.orchestra.run.smithi022.stderr:128750K .......... .......... .......... .......... .......... 76% 23.0M 2s 2024-03-29T04:19:22.162 INFO:teuthology.orchestra.run.smithi022.stderr:128800K .......... .......... .......... .......... .......... 76% 26.6M 2s 2024-03-29T04:19:22.164 INFO:teuthology.orchestra.run.smithi022.stderr:128850K .......... .......... .......... .......... .......... 76% 25.6M 2s 2024-03-29T04:19:22.166 INFO:teuthology.orchestra.run.smithi022.stderr:128900K .......... .......... .......... .......... .......... 76% 27.1M 2s 2024-03-29T04:19:22.168 INFO:teuthology.orchestra.run.smithi022.stderr:128950K .......... .......... .......... .......... .......... 76% 20.9M 2s 2024-03-29T04:19:22.170 INFO:teuthology.orchestra.run.smithi022.stderr:129000K .......... .......... .......... .......... .......... 76% 23.4M 2s 2024-03-29T04:19:22.172 INFO:teuthology.orchestra.run.smithi022.stderr:129050K .......... .......... .......... .......... .......... 76% 22.9M 2s 2024-03-29T04:19:22.175 INFO:teuthology.orchestra.run.smithi022.stderr:129100K .......... .......... .......... .......... .......... 76% 22.8M 2s 2024-03-29T04:19:22.177 INFO:teuthology.orchestra.run.smithi022.stderr:129150K .......... .......... .......... .......... .......... 76% 19.5M 2s 2024-03-29T04:19:22.179 INFO:teuthology.orchestra.run.smithi022.stderr:129200K .......... .......... .......... .......... .......... 76% 22.8M 2s 2024-03-29T04:19:22.181 INFO:teuthology.orchestra.run.smithi022.stderr:129250K .......... .......... .......... .......... .......... 76% 22.9M 2s 2024-03-29T04:19:22.184 INFO:teuthology.orchestra.run.smithi022.stderr:129300K .......... .......... .......... .......... .......... 76% 22.2M 2s 2024-03-29T04:19:22.186 INFO:teuthology.orchestra.run.smithi022.stderr:129350K .......... .......... .......... .......... .......... 76% 19.5M 2s 2024-03-29T04:19:22.188 INFO:teuthology.orchestra.run.smithi022.stderr:129400K .......... .......... .......... .......... .......... 76% 22.4M 2s 2024-03-29T04:19:22.191 INFO:teuthology.orchestra.run.smithi022.stderr:129450K .......... .......... .......... .......... .......... 76% 22.3M 2s 2024-03-29T04:19:22.193 INFO:teuthology.orchestra.run.smithi022.stderr:129500K .......... .......... .......... .......... .......... 76% 22.5M 2s 2024-03-29T04:19:22.195 INFO:teuthology.orchestra.run.smithi022.stderr:129550K .......... .......... .......... .......... .......... 76% 19.0M 2s 2024-03-29T04:19:22.197 INFO:teuthology.orchestra.run.smithi022.stderr:129600K .......... .......... .......... .......... .......... 76% 22.3M 2s 2024-03-29T04:19:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:129650K .......... .......... .......... .......... .......... 76% 21.4M 2s 2024-03-29T04:19:22.202 INFO:teuthology.orchestra.run.smithi022.stderr:129700K .......... .......... .......... .......... .......... 76% 21.3M 2s 2024-03-29T04:19:22.205 INFO:teuthology.orchestra.run.smithi022.stderr:129750K .......... .......... .......... .......... .......... 77% 18.2M 2s 2024-03-29T04:19:22.207 INFO:teuthology.orchestra.run.smithi022.stderr:129800K .......... .......... .......... .......... .......... 77% 21.5M 2s 2024-03-29T04:19:22.211 INFO:teuthology.orchestra.run.smithi022.stderr:129850K .......... .......... .......... .......... .......... 77% 14.1M 2s 2024-03-29T04:19:22.214 INFO:teuthology.orchestra.run.smithi022.stderr:129900K .......... .......... .......... .......... .......... 77% 14.5M 2s 2024-03-29T04:19:22.218 INFO:teuthology.orchestra.run.smithi022.stderr:129950K .......... .......... .......... .......... .......... 77% 11.6M 2s 2024-03-29T04:19:22.222 INFO:teuthology.orchestra.run.smithi022.stderr:130000K .......... .......... .......... .......... .......... 77% 14.3M 2s 2024-03-29T04:19:22.225 INFO:teuthology.orchestra.run.smithi022.stderr:130050K .......... .......... .......... .......... .......... 77% 14.0M 2s 2024-03-29T04:19:22.229 INFO:teuthology.orchestra.run.smithi022.stderr:130100K .......... .......... .......... .......... .......... 77% 14.1M 2s 2024-03-29T04:19:22.233 INFO:teuthology.orchestra.run.smithi022.stderr:130150K .......... .......... .......... .......... .......... 77% 11.8M 2s 2024-03-29T04:19:22.236 INFO:teuthology.orchestra.run.smithi022.stderr:130200K .......... .......... .......... .......... .......... 77% 14.1M 2s 2024-03-29T04:19:22.240 INFO:teuthology.orchestra.run.smithi022.stderr:130250K .......... .......... .......... .......... .......... 77% 13.9M 2s 2024-03-29T04:19:22.243 INFO:teuthology.orchestra.run.smithi022.stderr:130300K .......... .......... .......... .......... .......... 77% 13.2M 2s 2024-03-29T04:19:22.248 INFO:teuthology.orchestra.run.smithi022.stderr:130350K .......... .......... .......... .......... .......... 77% 11.7M 2s 2024-03-29T04:19:22.251 INFO:teuthology.orchestra.run.smithi022.stderr:130400K .......... .......... .......... .......... .......... 77% 13.5M 2s 2024-03-29T04:19:22.255 INFO:teuthology.orchestra.run.smithi022.stderr:130450K .......... .......... .......... .......... .......... 77% 14.4M 2s 2024-03-29T04:19:22.258 INFO:teuthology.orchestra.run.smithi022.stderr:130500K .......... .......... .......... .......... .......... 77% 14.0M 2s 2024-03-29T04:19:22.262 INFO:teuthology.orchestra.run.smithi022.stderr:130550K .......... .......... .......... .......... .......... 77% 11.7M 2s 2024-03-29T04:19:22.266 INFO:teuthology.orchestra.run.smithi022.stderr:130600K .......... .......... .......... .......... .......... 77% 13.8M 2s 2024-03-29T04:19:22.269 INFO:teuthology.orchestra.run.smithi022.stderr:130650K .......... .......... .......... .......... .......... 77% 15.0M 2s 2024-03-29T04:19:22.272 INFO:teuthology.orchestra.run.smithi022.stderr:130700K .......... .......... .......... .......... .......... 77% 15.7M 2s 2024-03-29T04:19:22.276 INFO:teuthology.orchestra.run.smithi022.stderr:130750K .......... .......... .......... .......... .......... 77% 13.0M 2s 2024-03-29T04:19:22.279 INFO:teuthology.orchestra.run.smithi022.stderr:130800K .......... .......... .......... .......... .......... 77% 13.9M 2s 2024-03-29T04:19:22.283 INFO:teuthology.orchestra.run.smithi022.stderr:130850K .......... .......... .......... .......... .......... 77% 13.7M 2s 2024-03-29T04:19:22.286 INFO:teuthology.orchestra.run.smithi022.stderr:130900K .......... .......... .......... .......... .......... 77% 14.2M 2s 2024-03-29T04:19:22.291 INFO:teuthology.orchestra.run.smithi022.stderr:130950K .......... .......... .......... .......... .......... 77% 11.4M 2s 2024-03-29T04:19:22.294 INFO:teuthology.orchestra.run.smithi022.stderr:131000K .......... .......... .......... .......... .......... 77% 13.2M 2s 2024-03-29T04:19:22.298 INFO:teuthology.orchestra.run.smithi022.stderr:131050K .......... .......... .......... .......... .......... 77% 12.5M 2s 2024-03-29T04:19:22.302 INFO:teuthology.orchestra.run.smithi022.stderr:131100K .......... .......... .......... .......... .......... 77% 12.9M 2s 2024-03-29T04:19:22.307 INFO:teuthology.orchestra.run.smithi022.stderr:131150K .......... .......... .......... .......... .......... 77% 10.5M 2s 2024-03-29T04:19:22.309 INFO:teuthology.orchestra.run.smithi022.stderr:131200K .......... .......... .......... .......... .......... 77% 17.1M 2s 2024-03-29T04:19:22.311 INFO:teuthology.orchestra.run.smithi022.stderr:131250K .......... .......... .......... .......... .......... 77% 29.7M 2s 2024-03-29T04:19:22.313 INFO:teuthology.orchestra.run.smithi022.stderr:131300K .......... .......... .......... .......... .......... 77% 28.4M 2s 2024-03-29T04:19:22.316 INFO:teuthology.orchestra.run.smithi022.stderr:131350K .......... .......... .......... .......... .......... 77% 14.9M 2s 2024-03-29T04:19:22.318 INFO:teuthology.orchestra.run.smithi022.stderr:131400K .......... .......... .......... .......... .......... 77% 21.7M 2s 2024-03-29T04:19:22.321 INFO:teuthology.orchestra.run.smithi022.stderr:131450K .......... .......... .......... .......... .......... 78% 23.0M 2s 2024-03-29T04:19:22.323 INFO:teuthology.orchestra.run.smithi022.stderr:131500K .......... .......... .......... .......... .......... 78% 23.4M 2s 2024-03-29T04:19:22.325 INFO:teuthology.orchestra.run.smithi022.stderr:131550K .......... .......... .......... .......... .......... 78% 20.5M 2s 2024-03-29T04:19:22.327 INFO:teuthology.orchestra.run.smithi022.stderr:131600K .......... .......... .......... .......... .......... 78% 23.6M 2s 2024-03-29T04:19:22.329 INFO:teuthology.orchestra.run.smithi022.stderr:131650K .......... .......... .......... .......... .......... 78% 23.2M 2s 2024-03-29T04:19:22.331 INFO:teuthology.orchestra.run.smithi022.stderr:131700K .......... .......... .......... .......... .......... 78% 22.8M 2s 2024-03-29T04:19:22.334 INFO:teuthology.orchestra.run.smithi022.stderr:131750K .......... .......... .......... .......... .......... 78% 19.3M 2s 2024-03-29T04:19:22.336 INFO:teuthology.orchestra.run.smithi022.stderr:131800K .......... .......... .......... .......... .......... 78% 22.3M 2s 2024-03-29T04:19:22.338 INFO:teuthology.orchestra.run.smithi022.stderr:131850K .......... .......... .......... .......... .......... 78% 22.4M 2s 2024-03-29T04:19:22.340 INFO:teuthology.orchestra.run.smithi022.stderr:131900K .......... .......... .......... .......... .......... 78% 22.5M 2s 2024-03-29T04:19:22.343 INFO:teuthology.orchestra.run.smithi022.stderr:131950K .......... .......... .......... .......... .......... 78% 20.0M 2s 2024-03-29T04:19:22.345 INFO:teuthology.orchestra.run.smithi022.stderr:132000K .......... .......... .......... .......... .......... 78% 22.7M 2s 2024-03-29T04:19:22.347 INFO:teuthology.orchestra.run.smithi022.stderr:132050K .......... .......... .......... .......... .......... 78% 21.9M 2s 2024-03-29T04:19:22.350 INFO:teuthology.orchestra.run.smithi022.stderr:132100K .......... .......... .......... .......... .......... 78% 27.7M 2s 2024-03-29T04:19:22.351 INFO:teuthology.orchestra.run.smithi022.stderr:132150K .......... .......... .......... .......... .......... 78% 23.5M 2s 2024-03-29T04:19:22.353 INFO:teuthology.orchestra.run.smithi022.stderr:132200K .......... .......... .......... .......... .......... 78% 24.8M 2s 2024-03-29T04:19:22.355 INFO:teuthology.orchestra.run.smithi022.stderr:132250K .......... .......... .......... .......... .......... 78% 25.6M 2s 2024-03-29T04:19:22.356 INFO:teuthology.orchestra.run.smithi022.stderr:132300K .......... .......... .......... .......... .......... 78% 29.4M 2s 2024-03-29T04:19:22.358 INFO:teuthology.orchestra.run.smithi022.stderr:132350K .......... .......... .......... .......... .......... 78% 26.8M 2s 2024-03-29T04:19:22.360 INFO:teuthology.orchestra.run.smithi022.stderr:132400K .......... .......... .......... .......... .......... 78% 32.0M 2s 2024-03-29T04:19:22.361 INFO:teuthology.orchestra.run.smithi022.stderr:132450K .......... .......... .......... .......... .......... 78% 31.5M 2s 2024-03-29T04:19:22.363 INFO:teuthology.orchestra.run.smithi022.stderr:132500K .......... .......... .......... .......... .......... 78% 31.2M 2s 2024-03-29T04:19:22.365 INFO:teuthology.orchestra.run.smithi022.stderr:132550K .......... .......... .......... .......... .......... 78% 27.0M 2s 2024-03-29T04:19:22.366 INFO:teuthology.orchestra.run.smithi022.stderr:132600K .......... .......... .......... .......... .......... 78% 31.8M 2s 2024-03-29T04:19:22.367 INFO:teuthology.orchestra.run.smithi022.stderr:132650K .......... .......... .......... .......... .......... 78% 31.1M 2s 2024-03-29T04:19:22.369 INFO:teuthology.orchestra.run.smithi022.stderr:132700K .......... .......... .......... .......... .......... 78% 31.7M 2s 2024-03-29T04:19:22.371 INFO:teuthology.orchestra.run.smithi022.stderr:132750K .......... .......... .......... .......... .......... 78% 26.8M 2s 2024-03-29T04:19:22.372 INFO:teuthology.orchestra.run.smithi022.stderr:132800K .......... .......... .......... .......... .......... 78% 30.0M 2s 2024-03-29T04:19:22.374 INFO:teuthology.orchestra.run.smithi022.stderr:132850K .......... .......... .......... .......... .......... 78% 28.3M 2s 2024-03-29T04:19:22.376 INFO:teuthology.orchestra.run.smithi022.stderr:132900K .......... .......... .......... .......... .......... 78% 26.4M 2s 2024-03-29T04:19:22.378 INFO:teuthology.orchestra.run.smithi022.stderr:132950K .......... .......... .......... .......... .......... 78% 25.7M 2s 2024-03-29T04:19:22.380 INFO:teuthology.orchestra.run.smithi022.stderr:133000K .......... .......... .......... .......... .......... 78% 27.8M 2s 2024-03-29T04:19:22.382 INFO:teuthology.orchestra.run.smithi022.stderr:133050K .......... .......... .......... .......... .......... 78% 26.8M 2s 2024-03-29T04:19:22.383 INFO:teuthology.orchestra.run.smithi022.stderr:133100K .......... .......... .......... .......... .......... 78% 27.9M 2s 2024-03-29T04:19:22.385 INFO:teuthology.orchestra.run.smithi022.stderr:133150K .......... .......... .......... .......... .......... 79% 23.8M 2s 2024-03-29T04:19:22.387 INFO:teuthology.orchestra.run.smithi022.stderr:133200K .......... .......... .......... .......... .......... 79% 27.9M 2s 2024-03-29T04:19:22.389 INFO:teuthology.orchestra.run.smithi022.stderr:133250K .......... .......... .......... .......... .......... 79% 28.0M 2s 2024-03-29T04:19:22.391 INFO:teuthology.orchestra.run.smithi022.stderr:133300K .......... .......... .......... .......... .......... 79% 27.9M 2s 2024-03-29T04:19:22.393 INFO:teuthology.orchestra.run.smithi022.stderr:133350K .......... .......... .......... .......... .......... 79% 23.4M 2s 2024-03-29T04:19:22.394 INFO:teuthology.orchestra.run.smithi022.stderr:133400K .......... .......... .......... .......... .......... 79% 27.6M 2s 2024-03-29T04:19:22.396 INFO:teuthology.orchestra.run.smithi022.stderr:133450K .......... .......... .......... .......... .......... 79% 27.1M 2s 2024-03-29T04:19:22.398 INFO:teuthology.orchestra.run.smithi022.stderr:133500K .......... .......... .......... .......... .......... 79% 27.5M 2s 2024-03-29T04:19:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:133550K .......... .......... .......... .......... .......... 79% 23.3M 2s 2024-03-29T04:19:22.402 INFO:teuthology.orchestra.run.smithi022.stderr:133600K .......... .......... .......... .......... .......... 79% 27.1M 2s 2024-03-29T04:19:22.404 INFO:teuthology.orchestra.run.smithi022.stderr:133650K .......... .......... .......... .......... .......... 79% 26.8M 2s 2024-03-29T04:19:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:133700K .......... .......... .......... .......... .......... 79% 26.9M 2s 2024-03-29T04:19:22.408 INFO:teuthology.orchestra.run.smithi022.stderr:133750K .......... .......... .......... .......... .......... 79% 22.9M 2s 2024-03-29T04:19:22.410 INFO:teuthology.orchestra.run.smithi022.stderr:133800K .......... .......... .......... .......... .......... 79% 27.1M 2s 2024-03-29T04:19:22.411 INFO:teuthology.orchestra.run.smithi022.stderr:133850K .......... .......... .......... .......... .......... 79% 26.5M 2s 2024-03-29T04:19:22.413 INFO:teuthology.orchestra.run.smithi022.stderr:133900K .......... .......... .......... .......... .......... 79% 26.2M 2s 2024-03-29T04:19:22.416 INFO:teuthology.orchestra.run.smithi022.stderr:133950K .......... .......... .......... .......... .......... 79% 21.5M 2s 2024-03-29T04:19:22.417 INFO:teuthology.orchestra.run.smithi022.stderr:134000K .......... .......... .......... .......... .......... 79% 25.8M 2s 2024-03-29T04:19:22.419 INFO:teuthology.orchestra.run.smithi022.stderr:134050K .......... .......... .......... .......... .......... 79% 25.4M 2s 2024-03-29T04:19:22.421 INFO:teuthology.orchestra.run.smithi022.stderr:134100K .......... .......... .......... .......... .......... 79% 24.5M 2s 2024-03-29T04:19:22.424 INFO:teuthology.orchestra.run.smithi022.stderr:134150K .......... .......... .......... .......... .......... 79% 19.6M 2s 2024-03-29T04:19:22.426 INFO:teuthology.orchestra.run.smithi022.stderr:134200K .......... .......... .......... .......... .......... 79% 23.3M 2s 2024-03-29T04:19:22.428 INFO:teuthology.orchestra.run.smithi022.stderr:134250K .......... .......... .......... .......... .......... 79% 22.9M 2s 2024-03-29T04:19:22.430 INFO:teuthology.orchestra.run.smithi022.stderr:134300K .......... .......... .......... .......... .......... 79% 22.9M 2s 2024-03-29T04:19:22.433 INFO:teuthology.orchestra.run.smithi022.stderr:134350K .......... .......... .......... .......... .......... 79% 20.8M 2s 2024-03-29T04:19:22.435 INFO:teuthology.orchestra.run.smithi022.stderr:134400K .......... .......... .......... .......... .......... 79% 22.5M 2s 2024-03-29T04:19:22.437 INFO:teuthology.orchestra.run.smithi022.stderr:134450K .......... .......... .......... .......... .......... 79% 23.4M 2s 2024-03-29T04:19:22.439 INFO:teuthology.orchestra.run.smithi022.stderr:134500K .......... .......... .......... .......... .......... 79% 24.0M 2s 2024-03-29T04:19:22.441 INFO:teuthology.orchestra.run.smithi022.stderr:134550K .......... .......... .......... .......... .......... 79% 20.8M 2s 2024-03-29T04:19:22.444 INFO:teuthology.orchestra.run.smithi022.stderr:134600K .......... .......... .......... .......... .......... 79% 22.8M 2s 2024-03-29T04:19:22.446 INFO:teuthology.orchestra.run.smithi022.stderr:134650K .......... .......... .......... .......... .......... 79% 22.3M 2s 2024-03-29T04:19:22.448 INFO:teuthology.orchestra.run.smithi022.stderr:134700K .......... .......... .......... .......... .......... 79% 22.2M 2s 2024-03-29T04:19:22.450 INFO:teuthology.orchestra.run.smithi022.stderr:134750K .......... .......... .......... .......... .......... 79% 19.1M 2s 2024-03-29T04:19:22.453 INFO:teuthology.orchestra.run.smithi022.stderr:134800K .......... .......... .......... .......... .......... 79% 21.5M 2s 2024-03-29T04:19:22.455 INFO:teuthology.orchestra.run.smithi022.stderr:134850K .......... .......... .......... .......... .......... 80% 20.8M 2s 2024-03-29T04:19:22.457 INFO:teuthology.orchestra.run.smithi022.stderr:134900K .......... .......... .......... .......... .......... 80% 20.8M 2s 2024-03-29T04:19:22.460 INFO:teuthology.orchestra.run.smithi022.stderr:134950K .......... .......... .......... .......... .......... 80% 17.8M 2s 2024-03-29T04:19:22.463 INFO:teuthology.orchestra.run.smithi022.stderr:135000K .......... .......... .......... .......... .......... 80% 18.3M 2s 2024-03-29T04:19:22.465 INFO:teuthology.orchestra.run.smithi022.stderr:135050K .......... .......... .......... .......... .......... 80% 19.4M 2s 2024-03-29T04:19:22.468 INFO:teuthology.orchestra.run.smithi022.stderr:135100K .......... .......... .......... .......... .......... 80% 18.6M 2s 2024-03-29T04:19:22.471 INFO:teuthology.orchestra.run.smithi022.stderr:135150K .......... .......... .......... .......... .......... 80% 17.4M 2s 2024-03-29T04:19:22.472 INFO:teuthology.orchestra.run.smithi022.stderr:135200K .......... .......... .......... .......... .......... 80% 32.8M 2s 2024-03-29T04:19:22.474 INFO:teuthology.orchestra.run.smithi022.stderr:135250K .......... .......... .......... .......... .......... 80% 29.8M 2s 2024-03-29T04:19:22.476 INFO:teuthology.orchestra.run.smithi022.stderr:135300K .......... .......... .......... .......... .......... 80% 23.0M 2s 2024-03-29T04:19:22.478 INFO:teuthology.orchestra.run.smithi022.stderr:135350K .......... .......... .......... .......... .......... 80% 22.6M 2s 2024-03-29T04:19:22.480 INFO:teuthology.orchestra.run.smithi022.stderr:135400K .......... .......... .......... .......... .......... 80% 28.3M 2s 2024-03-29T04:19:22.482 INFO:teuthology.orchestra.run.smithi022.stderr:135450K .......... .......... .......... .......... .......... 80% 27.9M 2s 2024-03-29T04:19:22.483 INFO:teuthology.orchestra.run.smithi022.stderr:135500K .......... .......... .......... .......... .......... 80% 29.5M 2s 2024-03-29T04:19:22.485 INFO:teuthology.orchestra.run.smithi022.stderr:135550K .......... .......... .......... .......... .......... 80% 26.3M 2s 2024-03-29T04:19:22.487 INFO:teuthology.orchestra.run.smithi022.stderr:135600K .......... .......... .......... .......... .......... 80% 29.0M 2s 2024-03-29T04:19:22.488 INFO:teuthology.orchestra.run.smithi022.stderr:135650K .......... .......... .......... .......... .......... 80% 30.5M 2s 2024-03-29T04:19:22.490 INFO:teuthology.orchestra.run.smithi022.stderr:135700K .......... .......... .......... .......... .......... 80% 30.9M 2s 2024-03-29T04:19:22.492 INFO:teuthology.orchestra.run.smithi022.stderr:135750K .......... .......... .......... .......... .......... 80% 22.2M 2s 2024-03-29T04:19:22.494 INFO:teuthology.orchestra.run.smithi022.stderr:135800K .......... .......... .......... .......... .......... 80% 30.4M 2s 2024-03-29T04:19:22.495 INFO:teuthology.orchestra.run.smithi022.stderr:135850K .......... .......... .......... .......... .......... 80% 29.4M 2s 2024-03-29T04:19:22.497 INFO:teuthology.orchestra.run.smithi022.stderr:135900K .......... .......... .......... .......... .......... 80% 30.8M 2s 2024-03-29T04:19:22.499 INFO:teuthology.orchestra.run.smithi022.stderr:135950K .......... .......... .......... .......... .......... 80% 26.2M 2s 2024-03-29T04:19:22.500 INFO:teuthology.orchestra.run.smithi022.stderr:136000K .......... .......... .......... .......... .......... 80% 30.7M 2s 2024-03-29T04:19:22.502 INFO:teuthology.orchestra.run.smithi022.stderr:136050K .......... .......... .......... .......... .......... 80% 29.1M 2s 2024-03-29T04:19:22.504 INFO:teuthology.orchestra.run.smithi022.stderr:136100K .......... .......... .......... .......... .......... 80% 29.7M 2s 2024-03-29T04:19:22.506 INFO:teuthology.orchestra.run.smithi022.stderr:136150K .......... .......... .......... .......... .......... 80% 25.9M 2s 2024-03-29T04:19:22.507 INFO:teuthology.orchestra.run.smithi022.stderr:136200K .......... .......... .......... .......... .......... 80% 28.5M 2s 2024-03-29T04:19:22.509 INFO:teuthology.orchestra.run.smithi022.stderr:136250K .......... .......... .......... .......... .......... 80% 28.4M 2s 2024-03-29T04:19:22.511 INFO:teuthology.orchestra.run.smithi022.stderr:136300K .......... .......... .......... .......... .......... 80% 27.6M 2s 2024-03-29T04:19:22.513 INFO:teuthology.orchestra.run.smithi022.stderr:136350K .......... .......... .......... .......... .......... 80% 23.6M 2s 2024-03-29T04:19:22.515 INFO:teuthology.orchestra.run.smithi022.stderr:136400K .......... .......... .......... .......... .......... 80% 26.2M 2s 2024-03-29T04:19:22.517 INFO:teuthology.orchestra.run.smithi022.stderr:136450K .......... .......... .......... .......... .......... 80% 25.4M 2s 2024-03-29T04:19:22.518 INFO:teuthology.orchestra.run.smithi022.stderr:136500K .......... .......... .......... .......... .......... 81% 27.9M 2s 2024-03-29T04:19:22.520 INFO:teuthology.orchestra.run.smithi022.stderr:136550K .......... .......... .......... .......... .......... 81% 25.0M 2s 2024-03-29T04:19:22.522 INFO:teuthology.orchestra.run.smithi022.stderr:136600K .......... .......... .......... .......... .......... 81% 29.8M 2s 2024-03-29T04:19:22.524 INFO:teuthology.orchestra.run.smithi022.stderr:136650K .......... .......... .......... .......... .......... 81% 26.2M 2s 2024-03-29T04:19:22.525 INFO:teuthology.orchestra.run.smithi022.stderr:136700K .......... .......... .......... .......... .......... 81% 29.8M 2s 2024-03-29T04:19:22.527 INFO:teuthology.orchestra.run.smithi022.stderr:136750K .......... .......... .......... .......... .......... 81% 25.7M 2s 2024-03-29T04:19:22.529 INFO:teuthology.orchestra.run.smithi022.stderr:136800K .......... .......... .......... .......... .......... 81% 29.5M 2s 2024-03-29T04:19:22.531 INFO:teuthology.orchestra.run.smithi022.stderr:136850K .......... .......... .......... .......... .......... 81% 29.1M 2s 2024-03-29T04:19:22.532 INFO:teuthology.orchestra.run.smithi022.stderr:136900K .......... .......... .......... .......... .......... 81% 29.6M 2s 2024-03-29T04:19:22.534 INFO:teuthology.orchestra.run.smithi022.stderr:136950K .......... .......... .......... .......... .......... 81% 25.1M 2s 2024-03-29T04:19:22.539 INFO:teuthology.orchestra.run.smithi022.stderr:137000K .......... .......... .......... .......... .......... 81% 28.9M 2s 2024-03-29T04:19:22.539 INFO:teuthology.orchestra.run.smithi022.stderr:137050K .......... .......... .......... .......... .......... 81% 29.5M 2s 2024-03-29T04:19:22.540 INFO:teuthology.orchestra.run.smithi022.stderr:137100K .......... .......... .......... .......... .......... 81% 29.0M 2s 2024-03-29T04:19:22.541 INFO:teuthology.orchestra.run.smithi022.stderr:137150K .......... .......... .......... .......... .......... 81% 24.3M 2s 2024-03-29T04:19:22.543 INFO:teuthology.orchestra.run.smithi022.stderr:137200K .......... .......... .......... .......... .......... 81% 28.8M 2s 2024-03-29T04:19:22.545 INFO:teuthology.orchestra.run.smithi022.stderr:137250K .......... .......... .......... .......... .......... 81% 25.8M 2s 2024-03-29T04:19:22.547 INFO:teuthology.orchestra.run.smithi022.stderr:137300K .......... .......... .......... .......... .......... 81% 24.6M 2s 2024-03-29T04:19:22.549 INFO:teuthology.orchestra.run.smithi022.stderr:137350K .......... .......... .......... .......... .......... 81% 21.3M 2s 2024-03-29T04:19:22.551 INFO:teuthology.orchestra.run.smithi022.stderr:137400K .......... .......... .......... .......... .......... 81% 24.1M 2s 2024-03-29T04:19:22.553 INFO:teuthology.orchestra.run.smithi022.stderr:137450K .......... .......... .......... .......... .......... 81% 25.0M 2s 2024-03-29T04:19:22.555 INFO:teuthology.orchestra.run.smithi022.stderr:137500K .......... .......... .......... .......... .......... 81% 24.5M 2s 2024-03-29T04:19:22.557 INFO:teuthology.orchestra.run.smithi022.stderr:137550K .......... .......... .......... .......... .......... 81% 21.0M 2s 2024-03-29T04:19:22.559 INFO:teuthology.orchestra.run.smithi022.stderr:137600K .......... .......... .......... .......... .......... 81% 24.4M 2s 2024-03-29T04:19:22.562 INFO:teuthology.orchestra.run.smithi022.stderr:137650K .......... .......... .......... .......... .......... 81% 24.1M 2s 2024-03-29T04:19:22.564 INFO:teuthology.orchestra.run.smithi022.stderr:137700K .......... .......... .......... .......... .......... 81% 23.5M 2s 2024-03-29T04:19:22.566 INFO:teuthology.orchestra.run.smithi022.stderr:137750K .......... .......... .......... .......... .......... 81% 20.6M 2s 2024-03-29T04:19:22.568 INFO:teuthology.orchestra.run.smithi022.stderr:137800K .......... .......... .......... .......... .......... 81% 23.8M 2s 2024-03-29T04:19:22.570 INFO:teuthology.orchestra.run.smithi022.stderr:137850K .......... .......... .......... .......... .......... 81% 23.9M 2s 2024-03-29T04:19:22.572 INFO:teuthology.orchestra.run.smithi022.stderr:137900K .......... .......... .......... .......... .......... 81% 23.5M 2s 2024-03-29T04:19:22.575 INFO:teuthology.orchestra.run.smithi022.stderr:137950K .......... .......... .......... .......... .......... 81% 20.3M 2s 2024-03-29T04:19:22.577 INFO:teuthology.orchestra.run.smithi022.stderr:138000K .......... .......... .......... .......... .......... 81% 23.6M 2s 2024-03-29T04:19:22.579 INFO:teuthology.orchestra.run.smithi022.stderr:138050K .......... .......... .......... .......... .......... 81% 23.4M 2s 2024-03-29T04:19:22.581 INFO:teuthology.orchestra.run.smithi022.stderr:138100K .......... .......... .......... .......... .......... 81% 23.0M 2s 2024-03-29T04:19:22.583 INFO:teuthology.orchestra.run.smithi022.stderr:138150K .......... .......... .......... .......... .......... 81% 19.7M 2s 2024-03-29T04:19:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:138200K .......... .......... .......... .......... .......... 82% 23.2M 2s 2024-03-29T04:19:22.588 INFO:teuthology.orchestra.run.smithi022.stderr:138250K .......... .......... .......... .......... .......... 82% 21.8M 2s 2024-03-29T04:19:22.590 INFO:teuthology.orchestra.run.smithi022.stderr:138300K .......... .......... .......... .......... .......... 82% 22.8M 2s 2024-03-29T04:19:22.593 INFO:teuthology.orchestra.run.smithi022.stderr:138350K .......... .......... .......... .......... .......... 82% 17.2M 2s 2024-03-29T04:19:22.596 INFO:teuthology.orchestra.run.smithi022.stderr:138400K .......... .......... .......... .......... .......... 82% 16.3M 2s 2024-03-29T04:19:22.599 INFO:teuthology.orchestra.run.smithi022.stderr:138450K .......... .......... .......... .......... .......... 82% 16.7M 2s 2024-03-29T04:19:22.602 INFO:teuthology.orchestra.run.smithi022.stderr:138500K .......... .......... .......... .......... .......... 82% 16.8M 2s 2024-03-29T04:19:22.605 INFO:teuthology.orchestra.run.smithi022.stderr:138550K .......... .......... .......... .......... .......... 82% 14.4M 2s 2024-03-29T04:19:22.608 INFO:teuthology.orchestra.run.smithi022.stderr:138600K .......... .......... .......... .......... .......... 82% 16.5M 2s 2024-03-29T04:19:22.611 INFO:teuthology.orchestra.run.smithi022.stderr:138650K .......... .......... .......... .......... .......... 82% 17.0M 2s 2024-03-29T04:19:22.613 INFO:teuthology.orchestra.run.smithi022.stderr:138700K .......... .......... .......... .......... .......... 82% 20.8M 2s 2024-03-29T04:19:22.616 INFO:teuthology.orchestra.run.smithi022.stderr:138750K .......... .......... .......... .......... .......... 82% 18.6M 2s 2024-03-29T04:19:22.618 INFO:teuthology.orchestra.run.smithi022.stderr:138800K .......... .......... .......... .......... .......... 82% 21.6M 2s 2024-03-29T04:19:22.620 INFO:teuthology.orchestra.run.smithi022.stderr:138850K .......... .......... .......... .......... .......... 82% 21.3M 2s 2024-03-29T04:19:22.623 INFO:teuthology.orchestra.run.smithi022.stderr:138900K .......... .......... .......... .......... .......... 82% 21.3M 2s 2024-03-29T04:19:22.625 INFO:teuthology.orchestra.run.smithi022.stderr:138950K .......... .......... .......... .......... .......... 82% 18.4M 2s 2024-03-29T04:19:22.628 INFO:teuthology.orchestra.run.smithi022.stderr:139000K .......... .......... .......... .......... .......... 82% 21.0M 2s 2024-03-29T04:19:22.630 INFO:teuthology.orchestra.run.smithi022.stderr:139050K .......... .......... .......... .......... .......... 82% 21.0M 2s 2024-03-29T04:19:22.632 INFO:teuthology.orchestra.run.smithi022.stderr:139100K .......... .......... .......... .......... .......... 82% 21.7M 2s 2024-03-29T04:19:22.635 INFO:teuthology.orchestra.run.smithi022.stderr:139150K .......... .......... .......... .......... .......... 82% 16.4M 2s 2024-03-29T04:19:22.638 INFO:teuthology.orchestra.run.smithi022.stderr:139200K .......... .......... .......... .......... .......... 82% 19.6M 2s 2024-03-29T04:19:22.642 INFO:teuthology.orchestra.run.smithi022.stderr:139250K .......... .......... .......... .......... .......... 82% 13.6M 2s 2024-03-29T04:19:22.647 INFO:teuthology.orchestra.run.smithi022.stderr:139300K .......... .......... .......... .......... .......... 82% 8.62M 2s 2024-03-29T04:19:22.649 INFO:teuthology.orchestra.run.smithi022.stderr:139350K .......... .......... .......... .......... .......... 82% 19.2M 2s 2024-03-29T04:19:22.651 INFO:teuthology.orchestra.run.smithi022.stderr:139400K .......... .......... .......... .......... .......... 82% 28.3M 2s 2024-03-29T04:19:22.653 INFO:teuthology.orchestra.run.smithi022.stderr:139450K .......... .......... .......... .......... .......... 82% 25.9M 2s 2024-03-29T04:19:22.656 INFO:teuthology.orchestra.run.smithi022.stderr:139500K .......... .......... .......... .......... .......... 82% 21.7M 2s 2024-03-29T04:19:22.658 INFO:teuthology.orchestra.run.smithi022.stderr:139550K .......... .......... .......... .......... .......... 82% 20.1M 2s 2024-03-29T04:19:22.660 INFO:teuthology.orchestra.run.smithi022.stderr:139600K .......... .......... .......... .......... .......... 82% 21.5M 2s 2024-03-29T04:19:22.662 INFO:teuthology.orchestra.run.smithi022.stderr:139650K .......... .......... .......... .......... .......... 82% 23.8M 2s 2024-03-29T04:19:22.664 INFO:teuthology.orchestra.run.smithi022.stderr:139700K .......... .......... .......... .......... .......... 82% 28.7M 2s 2024-03-29T04:19:22.666 INFO:teuthology.orchestra.run.smithi022.stderr:139750K .......... .......... .......... .......... .......... 82% 23.3M 2s 2024-03-29T04:19:22.667 INFO:teuthology.orchestra.run.smithi022.stderr:139800K .......... .......... .......... .......... .......... 82% 31.5M 2s 2024-03-29T04:19:22.669 INFO:teuthology.orchestra.run.smithi022.stderr:139850K .......... .......... .......... .......... .......... 82% 32.5M 2s 2024-03-29T04:19:22.671 INFO:teuthology.orchestra.run.smithi022.stderr:139900K .......... .......... .......... .......... .......... 83% 30.9M 2s 2024-03-29T04:19:22.673 INFO:teuthology.orchestra.run.smithi022.stderr:139950K .......... .......... .......... .......... .......... 83% 22.5M 2s 2024-03-29T04:19:22.675 INFO:teuthology.orchestra.run.smithi022.stderr:140000K .......... .......... .......... .......... .......... 83% 25.2M 2s 2024-03-29T04:19:22.677 INFO:teuthology.orchestra.run.smithi022.stderr:140050K .......... .......... .......... .......... .......... 83% 24.4M 2s 2024-03-29T04:19:22.679 INFO:teuthology.orchestra.run.smithi022.stderr:140100K .......... .......... .......... .......... .......... 83% 26.2M 2s 2024-03-29T04:19:22.681 INFO:teuthology.orchestra.run.smithi022.stderr:140150K .......... .......... .......... .......... .......... 83% 21.4M 2s 2024-03-29T04:19:22.683 INFO:teuthology.orchestra.run.smithi022.stderr:140200K .......... .......... .......... .......... .......... 83% 24.4M 2s 2024-03-29T04:19:22.685 INFO:teuthology.orchestra.run.smithi022.stderr:140250K .......... .......... .......... .......... .......... 83% 21.9M 2s 2024-03-29T04:19:22.687 INFO:teuthology.orchestra.run.smithi022.stderr:140300K .......... .......... .......... .......... .......... 83% 24.0M 2s 2024-03-29T04:19:22.689 INFO:teuthology.orchestra.run.smithi022.stderr:140350K .......... .......... .......... .......... .......... 83% 20.4M 2s 2024-03-29T04:19:22.691 INFO:teuthology.orchestra.run.smithi022.stderr:140400K .......... .......... .......... .......... .......... 83% 24.2M 2s 2024-03-29T04:19:22.694 INFO:teuthology.orchestra.run.smithi022.stderr:140450K .......... .......... .......... .......... .......... 83% 23.5M 2s 2024-03-29T04:19:22.696 INFO:teuthology.orchestra.run.smithi022.stderr:140500K .......... .......... .......... .......... .......... 83% 22.2M 2s 2024-03-29T04:19:22.698 INFO:teuthology.orchestra.run.smithi022.stderr:140550K .......... .......... .......... .......... .......... 83% 20.2M 2s 2024-03-29T04:19:22.700 INFO:teuthology.orchestra.run.smithi022.stderr:140600K .......... .......... .......... .......... .......... 83% 23.3M 2s 2024-03-29T04:19:22.702 INFO:teuthology.orchestra.run.smithi022.stderr:140650K .......... .......... .......... .......... .......... 83% 23.8M 2s 2024-03-29T04:19:22.704 INFO:teuthology.orchestra.run.smithi022.stderr:140700K .......... .......... .......... .......... .......... 83% 22.7M 2s 2024-03-29T04:19:22.707 INFO:teuthology.orchestra.run.smithi022.stderr:140750K .......... .......... .......... .......... .......... 83% 19.5M 2s 2024-03-29T04:19:22.709 INFO:teuthology.orchestra.run.smithi022.stderr:140800K .......... .......... .......... .......... .......... 83% 23.0M 2s 2024-03-29T04:19:22.711 INFO:teuthology.orchestra.run.smithi022.stderr:140850K .......... .......... .......... .......... .......... 83% 22.7M 2s 2024-03-29T04:19:22.713 INFO:teuthology.orchestra.run.smithi022.stderr:140900K .......... .......... .......... .......... .......... 83% 25.6M 2s 2024-03-29T04:19:22.715 INFO:teuthology.orchestra.run.smithi022.stderr:140950K .......... .......... .......... .......... .......... 83% 21.2M 2s 2024-03-29T04:19:22.717 INFO:teuthology.orchestra.run.smithi022.stderr:141000K .......... .......... .......... .......... .......... 83% 24.7M 2s 2024-03-29T04:19:22.719 INFO:teuthology.orchestra.run.smithi022.stderr:141050K .......... .......... .......... .......... .......... 83% 26.6M 2s 2024-03-29T04:19:22.721 INFO:teuthology.orchestra.run.smithi022.stderr:141100K .......... .......... .......... .......... .......... 83% 26.3M 2s 2024-03-29T04:19:22.723 INFO:teuthology.orchestra.run.smithi022.stderr:141150K .......... .......... .......... .......... .......... 83% 20.4M 2s 2024-03-29T04:19:22.725 INFO:teuthology.orchestra.run.smithi022.stderr:141200K .......... .......... .......... .......... .......... 83% 24.2M 2s 2024-03-29T04:19:22.728 INFO:teuthology.orchestra.run.smithi022.stderr:141250K .......... .......... .......... .......... .......... 83% 23.5M 2s 2024-03-29T04:19:22.729 INFO:teuthology.orchestra.run.smithi022.stderr:141300K .......... .......... .......... .......... .......... 83% 25.1M 2s 2024-03-29T04:19:22.732 INFO:teuthology.orchestra.run.smithi022.stderr:141350K .......... .......... .......... .......... .......... 83% 19.8M 2s 2024-03-29T04:19:22.734 INFO:teuthology.orchestra.run.smithi022.stderr:141400K .......... .......... .......... .......... .......... 83% 22.2M 2s 2024-03-29T04:19:22.736 INFO:teuthology.orchestra.run.smithi022.stderr:141450K .......... .......... .......... .......... .......... 83% 21.9M 2s 2024-03-29T04:19:22.739 INFO:teuthology.orchestra.run.smithi022.stderr:141500K .......... .......... .......... .......... .......... 83% 22.0M 2s 2024-03-29T04:19:22.741 INFO:teuthology.orchestra.run.smithi022.stderr:141550K .......... .......... .......... .......... .......... 84% 18.7M 1s 2024-03-29T04:19:22.743 INFO:teuthology.orchestra.run.smithi022.stderr:141600K .......... .......... .......... .......... .......... 84% 21.7M 1s 2024-03-29T04:19:22.746 INFO:teuthology.orchestra.run.smithi022.stderr:141650K .......... .......... .......... .......... .......... 84% 21.8M 1s 2024-03-29T04:19:22.748 INFO:teuthology.orchestra.run.smithi022.stderr:141700K .......... .......... .......... .......... .......... 84% 21.5M 1s 2024-03-29T04:19:22.751 INFO:teuthology.orchestra.run.smithi022.stderr:141750K .......... .......... .......... .......... .......... 84% 18.6M 1s 2024-03-29T04:19:22.753 INFO:teuthology.orchestra.run.smithi022.stderr:141800K .......... .......... .......... .......... .......... 84% 21.5M 1s 2024-03-29T04:19:22.755 INFO:teuthology.orchestra.run.smithi022.stderr:141850K .......... .......... .......... .......... .......... 84% 21.6M 1s 2024-03-29T04:19:22.757 INFO:teuthology.orchestra.run.smithi022.stderr:141900K .......... .......... .......... .......... .......... 84% 21.5M 1s 2024-03-29T04:19:22.760 INFO:teuthology.orchestra.run.smithi022.stderr:141950K .......... .......... .......... .......... .......... 84% 18.0M 1s 2024-03-29T04:19:22.762 INFO:teuthology.orchestra.run.smithi022.stderr:142000K .......... .......... .......... .......... .......... 84% 21.9M 1s 2024-03-29T04:19:22.765 INFO:teuthology.orchestra.run.smithi022.stderr:142050K .......... .......... .......... .......... .......... 84% 22.0M 1s 2024-03-29T04:19:22.767 INFO:teuthology.orchestra.run.smithi022.stderr:142100K .......... .......... .......... .......... .......... 84% 22.1M 1s 2024-03-29T04:19:22.769 INFO:teuthology.orchestra.run.smithi022.stderr:142150K .......... .......... .......... .......... .......... 84% 18.8M 1s 2024-03-29T04:19:22.772 INFO:teuthology.orchestra.run.smithi022.stderr:142200K .......... .......... .......... .......... .......... 84% 21.8M 1s 2024-03-29T04:19:22.774 INFO:teuthology.orchestra.run.smithi022.stderr:142250K .......... .......... .......... .......... .......... 84% 21.9M 1s 2024-03-29T04:19:22.776 INFO:teuthology.orchestra.run.smithi022.stderr:142300K .......... .......... .......... .......... .......... 84% 21.4M 1s 2024-03-29T04:19:22.779 INFO:teuthology.orchestra.run.smithi022.stderr:142350K .......... .......... .......... .......... .......... 84% 18.2M 1s 2024-03-29T04:19:22.781 INFO:teuthology.orchestra.run.smithi022.stderr:142400K .......... .......... .......... .......... .......... 84% 21.7M 1s 2024-03-29T04:19:22.783 INFO:teuthology.orchestra.run.smithi022.stderr:142450K .......... .......... .......... .......... .......... 84% 22.8M 1s 2024-03-29T04:19:22.785 INFO:teuthology.orchestra.run.smithi022.stderr:142500K .......... .......... .......... .......... .......... 84% 22.2M 1s 2024-03-29T04:19:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:142550K .......... .......... .......... .......... .......... 84% 19.1M 1s 2024-03-29T04:19:22.790 INFO:teuthology.orchestra.run.smithi022.stderr:142600K .......... .......... .......... .......... .......... 84% 22.8M 1s 2024-03-29T04:19:22.793 INFO:teuthology.orchestra.run.smithi022.stderr:142650K .......... .......... .......... .......... .......... 84% 18.5M 1s 2024-03-29T04:19:22.795 INFO:teuthology.orchestra.run.smithi022.stderr:142700K .......... .......... .......... .......... .......... 84% 20.0M 1s 2024-03-29T04:19:22.798 INFO:teuthology.orchestra.run.smithi022.stderr:142750K .......... .......... .......... .......... .......... 84% 19.8M 1s 2024-03-29T04:19:22.800 INFO:teuthology.orchestra.run.smithi022.stderr:142800K .......... .......... .......... .......... .......... 84% 23.2M 1s 2024-03-29T04:19:22.802 INFO:teuthology.orchestra.run.smithi022.stderr:142850K .......... .......... .......... .......... .......... 84% 23.0M 1s 2024-03-29T04:19:22.804 INFO:teuthology.orchestra.run.smithi022.stderr:142900K .......... .......... .......... .......... .......... 84% 23.9M 1s 2024-03-29T04:19:22.806 INFO:teuthology.orchestra.run.smithi022.stderr:142950K .......... .......... .......... .......... .......... 84% 21.4M 1s 2024-03-29T04:19:22.808 INFO:teuthology.orchestra.run.smithi022.stderr:143000K .......... .......... .......... .......... .......... 84% 23.1M 1s 2024-03-29T04:19:22.810 INFO:teuthology.orchestra.run.smithi022.stderr:143050K .......... .......... .......... .......... .......... 84% 23.2M 1s 2024-03-29T04:19:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:143100K .......... .......... .......... .......... .......... 84% 21.7M 1s 2024-03-29T04:19:22.816 INFO:teuthology.orchestra.run.smithi022.stderr:143150K .......... .......... .......... .......... .......... 84% 14.4M 1s 2024-03-29T04:19:22.820 INFO:teuthology.orchestra.run.smithi022.stderr:143200K .......... .......... .......... .......... .......... 84% 12.4M 1s 2024-03-29T04:19:22.824 INFO:teuthology.orchestra.run.smithi022.stderr:143250K .......... .......... .......... .......... .......... 85% 13.5M 1s 2024-03-29T04:19:22.828 INFO:teuthology.orchestra.run.smithi022.stderr:143300K .......... .......... .......... .......... .......... 85% 13.7M 1s 2024-03-29T04:19:22.831 INFO:teuthology.orchestra.run.smithi022.stderr:143350K .......... .......... .......... .......... .......... 85% 11.6M 1s 2024-03-29T04:19:22.834 INFO:teuthology.orchestra.run.smithi022.stderr:143400K .......... .......... .......... .......... .......... 85% 19.8M 1s 2024-03-29T04:19:22.837 INFO:teuthology.orchestra.run.smithi022.stderr:143450K .......... .......... .......... .......... .......... 85% 16.8M 1s 2024-03-29T04:19:22.840 INFO:teuthology.orchestra.run.smithi022.stderr:143500K .......... .......... .......... .......... .......... 85% 18.9M 1s 2024-03-29T04:19:22.841 INFO:teuthology.orchestra.run.smithi022.stderr:143550K .......... .......... .......... .......... .......... 85% 23.1M 1s 2024-03-29T04:19:22.843 INFO:teuthology.orchestra.run.smithi022.stderr:143600K .......... .......... .......... .......... .......... 85% 26.3M 1s 2024-03-29T04:19:22.845 INFO:teuthology.orchestra.run.smithi022.stderr:143650K .......... .......... .......... .......... .......... 85% 27.7M 1s 2024-03-29T04:19:22.847 INFO:teuthology.orchestra.run.smithi022.stderr:143700K .......... .......... .......... .......... .......... 85% 25.6M 1s 2024-03-29T04:19:22.849 INFO:teuthology.orchestra.run.smithi022.stderr:143750K .......... .......... .......... .......... .......... 85% 22.3M 1s 2024-03-29T04:19:22.851 INFO:teuthology.orchestra.run.smithi022.stderr:143800K .......... .......... .......... .......... .......... 85% 26.5M 1s 2024-03-29T04:19:22.853 INFO:teuthology.orchestra.run.smithi022.stderr:143850K .......... .......... .......... .......... .......... 85% 26.3M 1s 2024-03-29T04:19:22.855 INFO:teuthology.orchestra.run.smithi022.stderr:143900K .......... .......... .......... .......... .......... 85% 24.2M 1s 2024-03-29T04:19:22.857 INFO:teuthology.orchestra.run.smithi022.stderr:143950K .......... .......... .......... .......... .......... 85% 21.2M 1s 2024-03-29T04:19:22.859 INFO:teuthology.orchestra.run.smithi022.stderr:144000K .......... .......... .......... .......... .......... 85% 26.0M 1s 2024-03-29T04:19:22.861 INFO:teuthology.orchestra.run.smithi022.stderr:144050K .......... .......... .......... .......... .......... 85% 25.1M 1s 2024-03-29T04:19:22.863 INFO:teuthology.orchestra.run.smithi022.stderr:144100K .......... .......... .......... .......... .......... 85% 26.6M 1s 2024-03-29T04:19:22.865 INFO:teuthology.orchestra.run.smithi022.stderr:144150K .......... .......... .......... .......... .......... 85% 19.3M 1s 2024-03-29T04:19:22.867 INFO:teuthology.orchestra.run.smithi022.stderr:144200K .......... .......... .......... .......... .......... 85% 25.3M 1s 2024-03-29T04:19:22.870 INFO:teuthology.orchestra.run.smithi022.stderr:144250K .......... .......... .......... .......... .......... 85% 17.4M 1s 2024-03-29T04:19:22.873 INFO:teuthology.orchestra.run.smithi022.stderr:144300K .......... .......... .......... .......... .......... 85% 17.2M 1s 2024-03-29T04:19:22.876 INFO:teuthology.orchestra.run.smithi022.stderr:144350K .......... .......... .......... .......... .......... 85% 14.4M 1s 2024-03-29T04:19:22.879 INFO:teuthology.orchestra.run.smithi022.stderr:144400K .......... .......... .......... .......... .......... 85% 18.2M 1s 2024-03-29T04:19:22.882 INFO:teuthology.orchestra.run.smithi022.stderr:144450K .......... .......... .......... .......... .......... 85% 18.0M 1s 2024-03-29T04:19:22.885 INFO:teuthology.orchestra.run.smithi022.stderr:144500K .......... .......... .......... .......... .......... 85% 17.4M 1s 2024-03-29T04:19:22.888 INFO:teuthology.orchestra.run.smithi022.stderr:144550K .......... .......... .......... .......... .......... 85% 15.3M 1s 2024-03-29T04:19:22.891 INFO:teuthology.orchestra.run.smithi022.stderr:144600K .......... .......... .......... .......... .......... 85% 17.8M 1s 2024-03-29T04:19:22.893 INFO:teuthology.orchestra.run.smithi022.stderr:144650K .......... .......... .......... .......... .......... 85% 18.1M 1s 2024-03-29T04:19:22.896 INFO:teuthology.orchestra.run.smithi022.stderr:144700K .......... .......... .......... .......... .......... 85% 18.0M 1s 2024-03-29T04:19:22.899 INFO:teuthology.orchestra.run.smithi022.stderr:144750K .......... .......... .......... .......... .......... 85% 16.1M 1s 2024-03-29T04:19:22.902 INFO:teuthology.orchestra.run.smithi022.stderr:144800K .......... .......... .......... .......... .......... 85% 16.2M 1s 2024-03-29T04:19:22.905 INFO:teuthology.orchestra.run.smithi022.stderr:144850K .......... .......... .......... .......... .......... 85% 17.4M 1s 2024-03-29T04:19:22.908 INFO:teuthology.orchestra.run.smithi022.stderr:144900K .......... .......... .......... .......... .......... 85% 16.2M 1s 2024-03-29T04:19:22.911 INFO:teuthology.orchestra.run.smithi022.stderr:144950K .......... .......... .......... .......... .......... 86% 14.7M 1s 2024-03-29T04:19:22.914 INFO:teuthology.orchestra.run.smithi022.stderr:145000K .......... .......... .......... .......... .......... 86% 17.7M 1s 2024-03-29T04:19:22.917 INFO:teuthology.orchestra.run.smithi022.stderr:145050K .......... .......... .......... .......... .......... 86% 17.3M 1s 2024-03-29T04:19:22.920 INFO:teuthology.orchestra.run.smithi022.stderr:145100K .......... .......... .......... .......... .......... 86% 17.4M 1s 2024-03-29T04:19:22.923 INFO:teuthology.orchestra.run.smithi022.stderr:145150K .......... .......... .......... .......... .......... 86% 15.0M 1s 2024-03-29T04:19:22.926 INFO:teuthology.orchestra.run.smithi022.stderr:145200K .......... .......... .......... .......... .......... 86% 17.6M 1s 2024-03-29T04:19:22.928 INFO:teuthology.orchestra.run.smithi022.stderr:145250K .......... .......... .......... .......... .......... 86% 17.6M 1s 2024-03-29T04:19:22.931 INFO:teuthology.orchestra.run.smithi022.stderr:145300K .......... .......... .......... .......... .......... 86% 17.3M 1s 2024-03-29T04:19:22.935 INFO:teuthology.orchestra.run.smithi022.stderr:145350K .......... .......... .......... .......... .......... 86% 14.4M 1s 2024-03-29T04:19:22.938 INFO:teuthology.orchestra.run.smithi022.stderr:145400K .......... .......... .......... .......... .......... 86% 17.2M 1s 2024-03-29T04:19:22.940 INFO:teuthology.orchestra.run.smithi022.stderr:145450K .......... .......... .......... .......... .......... 86% 16.8M 1s 2024-03-29T04:19:22.947 INFO:teuthology.orchestra.run.smithi022.stderr:145500K .......... .......... .......... .......... .......... 86% 15.0M 1s 2024-03-29T04:19:22.947 INFO:teuthology.orchestra.run.smithi022.stderr:145550K .......... .......... .......... .......... .......... 86% 13.7M 1s 2024-03-29T04:19:22.951 INFO:teuthology.orchestra.run.smithi022.stderr:145600K .......... .......... .......... .......... .......... 86% 15.8M 1s 2024-03-29T04:19:22.954 INFO:teuthology.orchestra.run.smithi022.stderr:145650K .......... .......... .......... .......... .......... 86% 14.7M 1s 2024-03-29T04:19:22.957 INFO:teuthology.orchestra.run.smithi022.stderr:145700K .......... .......... .......... .......... .......... 86% 15.7M 1s 2024-03-29T04:19:22.960 INFO:teuthology.orchestra.run.smithi022.stderr:145750K .......... .......... .......... .......... .......... 86% 14.2M 1s 2024-03-29T04:19:22.963 INFO:teuthology.orchestra.run.smithi022.stderr:145800K .......... .......... .......... .......... .......... 86% 17.1M 1s 2024-03-29T04:19:22.966 INFO:teuthology.orchestra.run.smithi022.stderr:145850K .......... .......... .......... .......... .......... 86% 16.9M 1s 2024-03-29T04:19:22.970 INFO:teuthology.orchestra.run.smithi022.stderr:145900K .......... .......... .......... .......... .......... 86% 13.4M 1s 2024-03-29T04:19:22.974 INFO:teuthology.orchestra.run.smithi022.stderr:145950K .......... .......... .......... .......... .......... 86% 11.3M 1s 2024-03-29T04:19:22.977 INFO:teuthology.orchestra.run.smithi022.stderr:146000K .......... .......... .......... .......... .......... 86% 15.3M 1s 2024-03-29T04:19:22.981 INFO:teuthology.orchestra.run.smithi022.stderr:146050K .......... .......... .......... .......... .......... 86% 15.0M 1s 2024-03-29T04:19:22.984 INFO:teuthology.orchestra.run.smithi022.stderr:146100K .......... .......... .......... .......... .......... 86% 14.3M 1s 2024-03-29T04:19:22.988 INFO:teuthology.orchestra.run.smithi022.stderr:146150K .......... .......... .......... .......... .......... 86% 12.7M 1s 2024-03-29T04:19:22.991 INFO:teuthology.orchestra.run.smithi022.stderr:146200K .......... .......... .......... .......... .......... 86% 15.1M 1s 2024-03-29T04:19:22.994 INFO:teuthology.orchestra.run.smithi022.stderr:146250K .......... .......... .......... .......... .......... 86% 14.1M 1s 2024-03-29T04:19:22.998 INFO:teuthology.orchestra.run.smithi022.stderr:146300K .......... .......... .......... .......... .......... 86% 13.6M 1s 2024-03-29T04:19:23.002 INFO:teuthology.orchestra.run.smithi022.stderr:146350K .......... .......... .......... .......... .......... 86% 12.8M 1s 2024-03-29T04:19:23.005 INFO:teuthology.orchestra.run.smithi022.stderr:146400K .......... .......... .......... .......... .......... 86% 16.7M 1s 2024-03-29T04:19:23.008 INFO:teuthology.orchestra.run.smithi022.stderr:146450K .......... .......... .......... .......... .......... 86% 17.0M 1s 2024-03-29T04:19:23.010 INFO:teuthology.orchestra.run.smithi022.stderr:146500K .......... .......... .......... .......... .......... 86% 17.2M 1s 2024-03-29T04:19:23.014 INFO:teuthology.orchestra.run.smithi022.stderr:146550K .......... .......... .......... .......... .......... 86% 14.9M 1s 2024-03-29T04:19:23.017 INFO:teuthology.orchestra.run.smithi022.stderr:146600K .......... .......... .......... .......... .......... 86% 17.6M 1s 2024-03-29T04:19:23.019 INFO:teuthology.orchestra.run.smithi022.stderr:146650K .......... .......... .......... .......... .......... 87% 17.4M 1s 2024-03-29T04:19:23.023 INFO:teuthology.orchestra.run.smithi022.stderr:146700K .......... .......... .......... .......... .......... 87% 15.7M 1s 2024-03-29T04:19:23.027 INFO:teuthology.orchestra.run.smithi022.stderr:146750K .......... .......... .......... .......... .......... 87% 10.2M 1s 2024-03-29T04:19:23.031 INFO:teuthology.orchestra.run.smithi022.stderr:146800K .......... .......... .......... .......... .......... 87% 11.8M 1s 2024-03-29T04:19:23.036 INFO:teuthology.orchestra.run.smithi022.stderr:146850K .......... .......... .......... .......... .......... 87% 11.9M 1s 2024-03-29T04:19:23.040 INFO:teuthology.orchestra.run.smithi022.stderr:146900K .......... .......... .......... .......... .......... 87% 12.1M 1s 2024-03-29T04:19:23.044 INFO:teuthology.orchestra.run.smithi022.stderr:146950K .......... .......... .......... .......... .......... 87% 10.6M 1s 2024-03-29T04:19:23.048 INFO:teuthology.orchestra.run.smithi022.stderr:147000K .......... .......... .......... .......... .......... 87% 11.9M 1s 2024-03-29T04:19:23.052 INFO:teuthology.orchestra.run.smithi022.stderr:147050K .......... .......... .......... .......... .......... 87% 14.1M 1s 2024-03-29T04:19:23.055 INFO:teuthology.orchestra.run.smithi022.stderr:147100K .......... .......... .......... .......... .......... 87% 15.8M 1s 2024-03-29T04:19:23.058 INFO:teuthology.orchestra.run.smithi022.stderr:147150K .......... .......... .......... .......... .......... 87% 16.3M 1s 2024-03-29T04:19:23.060 INFO:teuthology.orchestra.run.smithi022.stderr:147200K .......... .......... .......... .......... .......... 87% 19.9M 1s 2024-03-29T04:19:23.063 INFO:teuthology.orchestra.run.smithi022.stderr:147250K .......... .......... .......... .......... .......... 87% 15.2M 1s 2024-03-29T04:19:23.066 INFO:teuthology.orchestra.run.smithi022.stderr:147300K .......... .......... .......... .......... .......... 87% 18.9M 1s 2024-03-29T04:19:23.069 INFO:teuthology.orchestra.run.smithi022.stderr:147350K .......... .......... .......... .......... .......... 87% 16.4M 1s 2024-03-29T04:19:23.071 INFO:teuthology.orchestra.run.smithi022.stderr:147400K .......... .......... .......... .......... .......... 87% 18.6M 1s 2024-03-29T04:19:23.073 INFO:teuthology.orchestra.run.smithi022.stderr:147450K .......... .......... .......... .......... .......... 87% 22.4M 1s 2024-03-29T04:19:23.075 INFO:teuthology.orchestra.run.smithi022.stderr:147500K .......... .......... .......... .......... .......... 87% 28.3M 1s 2024-03-29T04:19:23.077 INFO:teuthology.orchestra.run.smithi022.stderr:147550K .......... .......... .......... .......... .......... 87% 23.0M 1s 2024-03-29T04:19:23.079 INFO:teuthology.orchestra.run.smithi022.stderr:147600K .......... .......... .......... .......... .......... 87% 28.9M 1s 2024-03-29T04:19:23.081 INFO:teuthology.orchestra.run.smithi022.stderr:147650K .......... .......... .......... .......... .......... 87% 28.4M 1s 2024-03-29T04:19:23.082 INFO:teuthology.orchestra.run.smithi022.stderr:147700K .......... .......... .......... .......... .......... 87% 29.6M 1s 2024-03-29T04:19:23.085 INFO:teuthology.orchestra.run.smithi022.stderr:147750K .......... .......... .......... .......... .......... 87% 21.5M 1s 2024-03-29T04:19:23.086 INFO:teuthology.orchestra.run.smithi022.stderr:147800K .......... .......... .......... .......... .......... 87% 25.7M 1s 2024-03-29T04:19:23.088 INFO:teuthology.orchestra.run.smithi022.stderr:147850K .......... .......... .......... .......... .......... 87% 27.6M 1s 2024-03-29T04:19:23.091 INFO:teuthology.orchestra.run.smithi022.stderr:147900K .......... .......... .......... .......... .......... 87% 22.0M 1s 2024-03-29T04:19:23.093 INFO:teuthology.orchestra.run.smithi022.stderr:147950K .......... .......... .......... .......... .......... 87% 19.6M 1s 2024-03-29T04:19:23.096 INFO:teuthology.orchestra.run.smithi022.stderr:148000K .......... .......... .......... .......... .......... 87% 19.4M 1s 2024-03-29T04:19:23.098 INFO:teuthology.orchestra.run.smithi022.stderr:148050K .......... .......... .......... .......... .......... 87% 22.7M 1s 2024-03-29T04:19:23.100 INFO:teuthology.orchestra.run.smithi022.stderr:148100K .......... .......... .......... .......... .......... 87% 26.4M 1s 2024-03-29T04:19:23.102 INFO:teuthology.orchestra.run.smithi022.stderr:148150K .......... .......... .......... .......... .......... 87% 23.2M 1s 2024-03-29T04:19:23.104 INFO:teuthology.orchestra.run.smithi022.stderr:148200K .......... .......... .......... .......... .......... 87% 26.4M 1s 2024-03-29T04:19:23.105 INFO:teuthology.orchestra.run.smithi022.stderr:148250K .......... .......... .......... .......... .......... 87% 26.7M 1s 2024-03-29T04:19:23.107 INFO:teuthology.orchestra.run.smithi022.stderr:148300K .......... .......... .......... .......... .......... 88% 25.8M 1s 2024-03-29T04:19:23.109 INFO:teuthology.orchestra.run.smithi022.stderr:148350K .......... .......... .......... .......... .......... 88% 22.1M 1s 2024-03-29T04:19:23.111 INFO:teuthology.orchestra.run.smithi022.stderr:148400K .......... .......... .......... .......... .......... 88% 24.9M 1s 2024-03-29T04:19:23.113 INFO:teuthology.orchestra.run.smithi022.stderr:148450K .......... .......... .......... .......... .......... 88% 26.2M 1s 2024-03-29T04:19:23.115 INFO:teuthology.orchestra.run.smithi022.stderr:148500K .......... .......... .......... .......... .......... 88% 26.5M 1s 2024-03-29T04:19:23.117 INFO:teuthology.orchestra.run.smithi022.stderr:148550K .......... .......... .......... .......... .......... 88% 22.1M 1s 2024-03-29T04:19:23.119 INFO:teuthology.orchestra.run.smithi022.stderr:148600K .......... .......... .......... .......... .......... 88% 25.8M 1s 2024-03-29T04:19:23.121 INFO:teuthology.orchestra.run.smithi022.stderr:148650K .......... .......... .......... .......... .......... 88% 26.3M 1s 2024-03-29T04:19:23.123 INFO:teuthology.orchestra.run.smithi022.stderr:148700K .......... .......... .......... .......... .......... 88% 25.8M 1s 2024-03-29T04:19:23.125 INFO:teuthology.orchestra.run.smithi022.stderr:148750K .......... .......... .......... .......... .......... 88% 22.4M 1s 2024-03-29T04:19:23.127 INFO:teuthology.orchestra.run.smithi022.stderr:148800K .......... .......... .......... .......... .......... 88% 24.9M 1s 2024-03-29T04:19:23.129 INFO:teuthology.orchestra.run.smithi022.stderr:148850K .......... .......... .......... .......... .......... 88% 25.7M 1s 2024-03-29T04:19:23.131 INFO:teuthology.orchestra.run.smithi022.stderr:148900K .......... .......... .......... .......... .......... 88% 24.7M 1s 2024-03-29T04:19:23.134 INFO:teuthology.orchestra.run.smithi022.stderr:148950K .......... .......... .......... .......... .......... 88% 21.2M 1s 2024-03-29T04:19:23.135 INFO:teuthology.orchestra.run.smithi022.stderr:149000K .......... .......... .......... .......... .......... 88% 25.3M 1s 2024-03-29T04:19:23.137 INFO:teuthology.orchestra.run.smithi022.stderr:149050K .......... .......... .......... .......... .......... 88% 25.1M 1s 2024-03-29T04:19:23.139 INFO:teuthology.orchestra.run.smithi022.stderr:149100K .......... .......... .......... .......... .......... 88% 24.7M 1s 2024-03-29T04:19:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:149150K .......... .......... .......... .......... .......... 88% 21.5M 1s 2024-03-29T04:19:23.144 INFO:teuthology.orchestra.run.smithi022.stderr:149200K .......... .......... .......... .......... .......... 88% 22.0M 1s 2024-03-29T04:19:23.146 INFO:teuthology.orchestra.run.smithi022.stderr:149250K .......... .......... .......... .......... .......... 88% 22.6M 1s 2024-03-29T04:19:23.152 INFO:teuthology.orchestra.run.smithi022.stderr:149300K .......... .......... .......... .......... .......... 88% 22.9M 1s 2024-03-29T04:19:23.152 INFO:teuthology.orchestra.run.smithi022.stderr:149350K .......... .......... .......... .......... .......... 88% 19.7M 1s 2024-03-29T04:19:23.153 INFO:teuthology.orchestra.run.smithi022.stderr:149400K .......... .......... .......... .......... .......... 88% 22.3M 1s 2024-03-29T04:19:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:149450K .......... .......... .......... .......... .......... 88% 16.7M 1s 2024-03-29T04:19:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:149500K .......... .......... .......... .......... .......... 88% 21.8M 1s 2024-03-29T04:19:23.160 INFO:teuthology.orchestra.run.smithi022.stderr:149550K .......... .......... .......... .......... .......... 88% 18.8M 1s 2024-03-29T04:19:23.163 INFO:teuthology.orchestra.run.smithi022.stderr:149600K .......... .......... .......... .......... .......... 88% 21.7M 1s 2024-03-29T04:19:23.165 INFO:teuthology.orchestra.run.smithi022.stderr:149650K .......... .......... .......... .......... .......... 88% 21.9M 1s 2024-03-29T04:19:23.167 INFO:teuthology.orchestra.run.smithi022.stderr:149700K .......... .......... .......... .......... .......... 88% 21.6M 1s 2024-03-29T04:19:23.170 INFO:teuthology.orchestra.run.smithi022.stderr:149750K .......... .......... .......... .......... .......... 88% 15.7M 1s 2024-03-29T04:19:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:149800K .......... .......... .......... .......... .......... 88% 17.3M 1s 2024-03-29T04:19:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:149850K .......... .......... .......... .......... .......... 88% 14.9M 1s 2024-03-29T04:19:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:149900K .......... .......... .......... .......... .......... 88% 14.9M 1s 2024-03-29T04:19:23.183 INFO:teuthology.orchestra.run.smithi022.stderr:149950K .......... .......... .......... .......... .......... 88% 13.5M 1s 2024-03-29T04:19:23.186 INFO:teuthology.orchestra.run.smithi022.stderr:150000K .......... .......... .......... .......... .......... 89% 18.0M 1s 2024-03-29T04:19:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:150050K .......... .......... .......... .......... .......... 89% 12.1M 1s 2024-03-29T04:19:23.194 INFO:teuthology.orchestra.run.smithi022.stderr:150100K .......... .......... .......... .......... .......... 89% 12.8M 1s 2024-03-29T04:19:23.198 INFO:teuthology.orchestra.run.smithi022.stderr:150150K .......... .......... .......... .......... .......... 89% 12.9M 1s 2024-03-29T04:19:23.201 INFO:teuthology.orchestra.run.smithi022.stderr:150200K .......... .......... .......... .......... .......... 89% 15.4M 1s 2024-03-29T04:19:23.205 INFO:teuthology.orchestra.run.smithi022.stderr:150250K .......... .......... .......... .......... .......... 89% 14.0M 1s 2024-03-29T04:19:23.208 INFO:teuthology.orchestra.run.smithi022.stderr:150300K .......... .......... .......... .......... .......... 89% 15.0M 1s 2024-03-29T04:19:23.212 INFO:teuthology.orchestra.run.smithi022.stderr:150350K .......... .......... .......... .......... .......... 89% 12.9M 1s 2024-03-29T04:19:23.215 INFO:teuthology.orchestra.run.smithi022.stderr:150400K .......... .......... .......... .......... .......... 89% 14.5M 1s 2024-03-29T04:19:23.218 INFO:teuthology.orchestra.run.smithi022.stderr:150450K .......... .......... .......... .......... .......... 89% 16.2M 1s 2024-03-29T04:19:23.220 INFO:teuthology.orchestra.run.smithi022.stderr:150500K .......... .......... .......... .......... .......... 89% 18.6M 1s 2024-03-29T04:19:23.224 INFO:teuthology.orchestra.run.smithi022.stderr:150550K .......... .......... .......... .......... .......... 89% 12.9M 1s 2024-03-29T04:19:23.228 INFO:teuthology.orchestra.run.smithi022.stderr:150600K .......... .......... .......... .......... .......... 89% 14.4M 1s 2024-03-29T04:19:23.232 INFO:teuthology.orchestra.run.smithi022.stderr:150650K .......... .......... .......... .......... .......... 89% 13.6M 1s 2024-03-29T04:19:23.235 INFO:teuthology.orchestra.run.smithi022.stderr:150700K .......... .......... .......... .......... .......... 89% 12.6M 1s 2024-03-29T04:19:23.240 INFO:teuthology.orchestra.run.smithi022.stderr:150750K .......... .......... .......... .......... .......... 89% 11.3M 1s 2024-03-29T04:19:23.245 INFO:teuthology.orchestra.run.smithi022.stderr:150800K .......... .......... .......... .......... .......... 89% 9.86M 1s 2024-03-29T04:19:23.250 INFO:teuthology.orchestra.run.smithi022.stderr:150850K .......... .......... .......... .......... .......... 89% 9.26M 1s 2024-03-29T04:19:23.255 INFO:teuthology.orchestra.run.smithi022.stderr:150900K .......... .......... .......... .......... .......... 89% 9.25M 1s 2024-03-29T04:19:23.261 INFO:teuthology.orchestra.run.smithi022.stderr:150950K .......... .......... .......... .......... .......... 89% 8.47M 1s 2024-03-29T04:19:23.266 INFO:teuthology.orchestra.run.smithi022.stderr:151000K .......... .......... .......... .......... .......... 89% 10.4M 1s 2024-03-29T04:19:23.270 INFO:teuthology.orchestra.run.smithi022.stderr:151050K .......... .......... .......... .......... .......... 89% 11.4M 1s 2024-03-29T04:19:23.274 INFO:teuthology.orchestra.run.smithi022.stderr:151100K .......... .......... .......... .......... .......... 89% 12.5M 1s 2024-03-29T04:19:23.279 INFO:teuthology.orchestra.run.smithi022.stderr:151150K .......... .......... .......... .......... .......... 89% 8.97M 1s 2024-03-29T04:19:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:151200K .......... .......... .......... .......... .......... 89% 10.3M 1s 2024-03-29T04:19:23.289 INFO:teuthology.orchestra.run.smithi022.stderr:151250K .......... .......... .......... .......... .......... 89% 10.1M 1s 2024-03-29T04:19:23.294 INFO:teuthology.orchestra.run.smithi022.stderr:151300K .......... .......... .......... .......... .......... 89% 10.0M 1s 2024-03-29T04:19:23.300 INFO:teuthology.orchestra.run.smithi022.stderr:151350K .......... .......... .......... .......... .......... 89% 8.27M 1s 2024-03-29T04:19:23.305 INFO:teuthology.orchestra.run.smithi022.stderr:151400K .......... .......... .......... .......... .......... 89% 8.84M 1s 2024-03-29T04:19:23.310 INFO:teuthology.orchestra.run.smithi022.stderr:151450K .......... .......... .......... .......... .......... 89% 10.8M 1s 2024-03-29T04:19:23.315 INFO:teuthology.orchestra.run.smithi022.stderr:151500K .......... .......... .......... .......... .......... 89% 9.77M 1s 2024-03-29T04:19:23.319 INFO:teuthology.orchestra.run.smithi022.stderr:151550K .......... .......... .......... .......... .......... 89% 10.2M 1s 2024-03-29T04:19:23.322 INFO:teuthology.orchestra.run.smithi022.stderr:151600K .......... .......... .......... .......... .......... 89% 18.0M 1s 2024-03-29T04:19:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:151650K .......... .......... .......... .......... .......... 89% 23.1M 1s 2024-03-29T04:19:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:151700K .......... .......... .......... .......... .......... 90% 16.5M 1s 2024-03-29T04:19:23.330 INFO:teuthology.orchestra.run.smithi022.stderr:151750K .......... .......... .......... .......... .......... 90% 16.2M 1s 2024-03-29T04:19:23.332 INFO:teuthology.orchestra.run.smithi022.stderr:151800K .......... .......... .......... .......... .......... 90% 21.3M 1s 2024-03-29T04:19:23.335 INFO:teuthology.orchestra.run.smithi022.stderr:151850K .......... .......... .......... .......... .......... 90% 21.8M 1s 2024-03-29T04:19:23.336 INFO:teuthology.orchestra.run.smithi022.stderr:151900K .......... .......... .......... .......... .......... 90% 28.7M 1s 2024-03-29T04:19:23.338 INFO:teuthology.orchestra.run.smithi022.stderr:151950K .......... .......... .......... .......... .......... 90% 26.7M 1s 2024-03-29T04:19:23.339 INFO:teuthology.orchestra.run.smithi022.stderr:152000K .......... .......... .......... .......... .......... 90% 29.8M 1s 2024-03-29T04:19:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:152050K .......... .......... .......... .......... .......... 90% 29.3M 1s 2024-03-29T04:19:23.343 INFO:teuthology.orchestra.run.smithi022.stderr:152100K .......... .......... .......... .......... .......... 90% 28.2M 1s 2024-03-29T04:19:23.346 INFO:teuthology.orchestra.run.smithi022.stderr:152150K .......... .......... .......... .......... .......... 90% 20.0M 1s 2024-03-29T04:19:23.347 INFO:teuthology.orchestra.run.smithi022.stderr:152200K .......... .......... .......... .......... .......... 90% 25.0M 1s 2024-03-29T04:19:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:152250K .......... .......... .......... .......... .......... 90% 24.9M 1s 2024-03-29T04:19:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:152300K .......... .......... .......... .......... .......... 90% 25.0M 1s 2024-03-29T04:19:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:152350K .......... .......... .......... .......... .......... 90% 22.3M 1s 2024-03-29T04:19:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:152400K .......... .......... .......... .......... .......... 90% 23.2M 1s 2024-03-29T04:19:23.358 INFO:teuthology.orchestra.run.smithi022.stderr:152450K .......... .......... .......... .......... .......... 90% 21.1M 1s 2024-03-29T04:19:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:152500K .......... .......... .......... .......... .......... 90% 20.9M 1s 2024-03-29T04:19:23.363 INFO:teuthology.orchestra.run.smithi022.stderr:152550K .......... .......... .......... .......... .......... 90% 19.8M 1s 2024-03-29T04:19:23.365 INFO:teuthology.orchestra.run.smithi022.stderr:152600K .......... .......... .......... .......... .......... 90% 24.7M 1s 2024-03-29T04:19:23.367 INFO:teuthology.orchestra.run.smithi022.stderr:152650K .......... .......... .......... .......... .......... 90% 24.8M 1s 2024-03-29T04:19:23.369 INFO:teuthology.orchestra.run.smithi022.stderr:152700K .......... .......... .......... .......... .......... 90% 25.2M 1s 2024-03-29T04:19:23.371 INFO:teuthology.orchestra.run.smithi022.stderr:152750K .......... .......... .......... .......... .......... 90% 20.3M 1s 2024-03-29T04:19:23.373 INFO:teuthology.orchestra.run.smithi022.stderr:152800K .......... .......... .......... .......... .......... 90% 23.5M 1s 2024-03-29T04:19:23.375 INFO:teuthology.orchestra.run.smithi022.stderr:152850K .......... .......... .......... .......... .......... 90% 24.3M 1s 2024-03-29T04:19:23.377 INFO:teuthology.orchestra.run.smithi022.stderr:152900K .......... .......... .......... .......... .......... 90% 24.2M 1s 2024-03-29T04:19:23.379 INFO:teuthology.orchestra.run.smithi022.stderr:152950K .......... .......... .......... .......... .......... 90% 20.3M 1s 2024-03-29T04:19:23.382 INFO:teuthology.orchestra.run.smithi022.stderr:153000K .......... .......... .......... .......... .......... 90% 22.6M 1s 2024-03-29T04:19:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:153050K .......... .......... .......... .......... .......... 90% 21.9M 1s 2024-03-29T04:19:23.386 INFO:teuthology.orchestra.run.smithi022.stderr:153100K .......... .......... .......... .......... .......... 90% 22.6M 1s 2024-03-29T04:19:23.389 INFO:teuthology.orchestra.run.smithi022.stderr:153150K .......... .......... .......... .......... .......... 90% 19.3M 1s 2024-03-29T04:19:23.391 INFO:teuthology.orchestra.run.smithi022.stderr:153200K .......... .......... .......... .......... .......... 90% 22.5M 1s 2024-03-29T04:19:23.393 INFO:teuthology.orchestra.run.smithi022.stderr:153250K .......... .......... .......... .......... .......... 90% 22.8M 1s 2024-03-29T04:19:23.395 INFO:teuthology.orchestra.run.smithi022.stderr:153300K .......... .......... .......... .......... .......... 90% 22.5M 1s 2024-03-29T04:19:23.398 INFO:teuthology.orchestra.run.smithi022.stderr:153350K .......... .......... .......... .......... .......... 91% 19.3M 1s 2024-03-29T04:19:23.400 INFO:teuthology.orchestra.run.smithi022.stderr:153400K .......... .......... .......... .......... .......... 91% 22.3M 1s 2024-03-29T04:19:23.402 INFO:teuthology.orchestra.run.smithi022.stderr:153450K .......... .......... .......... .......... .......... 91% 22.0M 1s 2024-03-29T04:19:23.404 INFO:teuthology.orchestra.run.smithi022.stderr:153500K .......... .......... .......... .......... .......... 91% 21.9M 1s 2024-03-29T04:19:23.407 INFO:teuthology.orchestra.run.smithi022.stderr:153550K .......... .......... .......... .......... .......... 91% 18.8M 1s 2024-03-29T04:19:23.409 INFO:teuthology.orchestra.run.smithi022.stderr:153600K .......... .......... .......... .......... .......... 91% 21.7M 1s 2024-03-29T04:19:23.411 INFO:teuthology.orchestra.run.smithi022.stderr:153650K .......... .......... .......... .......... .......... 91% 21.3M 1s 2024-03-29T04:19:23.414 INFO:teuthology.orchestra.run.smithi022.stderr:153700K .......... .......... .......... .......... .......... 91% 21.1M 1s 2024-03-29T04:19:23.416 INFO:teuthology.orchestra.run.smithi022.stderr:153750K .......... .......... .......... .......... .......... 91% 18.5M 1s 2024-03-29T04:19:23.419 INFO:teuthology.orchestra.run.smithi022.stderr:153800K .......... .......... .......... .......... .......... 91% 21.1M 1s 2024-03-29T04:19:23.421 INFO:teuthology.orchestra.run.smithi022.stderr:153850K .......... .......... .......... .......... .......... 91% 21.0M 1s 2024-03-29T04:19:23.423 INFO:teuthology.orchestra.run.smithi022.stderr:153900K .......... .......... .......... .......... .......... 91% 21.0M 1s 2024-03-29T04:19:23.426 INFO:teuthology.orchestra.run.smithi022.stderr:153950K .......... .......... .......... .......... .......... 91% 17.8M 1s 2024-03-29T04:19:23.428 INFO:teuthology.orchestra.run.smithi022.stderr:154000K .......... .......... .......... .......... .......... 91% 20.4M 1s 2024-03-29T04:19:23.431 INFO:teuthology.orchestra.run.smithi022.stderr:154050K .......... .......... .......... .......... .......... 91% 20.7M 1s 2024-03-29T04:19:23.433 INFO:teuthology.orchestra.run.smithi022.stderr:154100K .......... .......... .......... .......... .......... 91% 20.6M 1s 2024-03-29T04:19:23.436 INFO:teuthology.orchestra.run.smithi022.stderr:154150K .......... .......... .......... .......... .......... 91% 16.1M 1s 2024-03-29T04:19:23.440 INFO:teuthology.orchestra.run.smithi022.stderr:154200K .......... .......... .......... .......... .......... 91% 14.3M 1s 2024-03-29T04:19:23.444 INFO:teuthology.orchestra.run.smithi022.stderr:154250K .......... .......... .......... .......... .......... 91% 12.7M 1s 2024-03-29T04:19:23.448 INFO:teuthology.orchestra.run.smithi022.stderr:154300K .......... .......... .......... .......... .......... 91% 12.7M 1s 2024-03-29T04:19:23.452 INFO:teuthology.orchestra.run.smithi022.stderr:154350K .......... .......... .......... .......... .......... 91% 10.7M 1s 2024-03-29T04:19:23.456 INFO:teuthology.orchestra.run.smithi022.stderr:154400K .......... .......... .......... .......... .......... 91% 12.9M 1s 2024-03-29T04:19:23.460 INFO:teuthology.orchestra.run.smithi022.stderr:154450K .......... .......... .......... .......... .......... 91% 12.7M 1s 2024-03-29T04:19:23.464 INFO:teuthology.orchestra.run.smithi022.stderr:154500K .......... .......... .......... .......... .......... 91% 12.7M 1s 2024-03-29T04:19:23.469 INFO:teuthology.orchestra.run.smithi022.stderr:154550K .......... .......... .......... .......... .......... 91% 9.94M 1s 2024-03-29T04:19:23.473 INFO:teuthology.orchestra.run.smithi022.stderr:154600K .......... .......... .......... .......... .......... 91% 11.3M 1s 2024-03-29T04:19:23.477 INFO:teuthology.orchestra.run.smithi022.stderr:154650K .......... .......... .......... .......... .......... 91% 12.1M 1s 2024-03-29T04:19:23.481 INFO:teuthology.orchestra.run.smithi022.stderr:154700K .......... .......... .......... .......... .......... 91% 12.0M 1s 2024-03-29T04:19:23.486 INFO:teuthology.orchestra.run.smithi022.stderr:154750K .......... .......... .......... .......... .......... 91% 10.0M 1s 2024-03-29T04:19:23.490 INFO:teuthology.orchestra.run.smithi022.stderr:154800K .......... .......... .......... .......... .......... 91% 12.9M 1s 2024-03-29T04:19:23.493 INFO:teuthology.orchestra.run.smithi022.stderr:154850K .......... .......... .......... .......... .......... 91% 12.6M 1s 2024-03-29T04:19:23.497 INFO:teuthology.orchestra.run.smithi022.stderr:154900K .......... .......... .......... .......... .......... 91% 12.8M 1s 2024-03-29T04:19:23.502 INFO:teuthology.orchestra.run.smithi022.stderr:154950K .......... .......... .......... .......... .......... 91% 10.6M 1s 2024-03-29T04:19:23.507 INFO:teuthology.orchestra.run.smithi022.stderr:155000K .......... .......... .......... .......... .......... 91% 10.1M 1s 2024-03-29T04:19:23.512 INFO:teuthology.orchestra.run.smithi022.stderr:155050K .......... .......... .......... .......... .......... 92% 10.1M 1s 2024-03-29T04:19:23.517 INFO:teuthology.orchestra.run.smithi022.stderr:155100K .......... .......... .......... .......... .......... 92% 9.76M 1s 2024-03-29T04:19:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:155150K .......... .......... .......... .......... .......... 92% 8.89M 1s 2024-03-29T04:19:23.527 INFO:teuthology.orchestra.run.smithi022.stderr:155200K .......... .......... .......... .......... .......... 92% 10.4M 1s 2024-03-29T04:19:23.532 INFO:teuthology.orchestra.run.smithi022.stderr:155250K .......... .......... .......... .......... .......... 92% 10.4M 1s 2024-03-29T04:19:23.537 INFO:teuthology.orchestra.run.smithi022.stderr:155300K .......... .......... .......... .......... .......... 92% 10.3M 1s 2024-03-29T04:19:23.541 INFO:teuthology.orchestra.run.smithi022.stderr:155350K .......... .......... .......... .......... .......... 92% 9.43M 1s 2024-03-29T04:19:23.546 INFO:teuthology.orchestra.run.smithi022.stderr:155400K .......... .......... .......... .......... .......... 92% 10.8M 1s 2024-03-29T04:19:23.551 INFO:teuthology.orchestra.run.smithi022.stderr:155450K .......... .......... .......... .......... .......... 92% 10.8M 1s 2024-03-29T04:19:23.559 INFO:teuthology.orchestra.run.smithi022.stderr:155500K .......... .......... .......... .......... .......... 92% 10.5M 1s 2024-03-29T04:19:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:155550K .......... .......... .......... .......... .......... 92% 9.94M 1s 2024-03-29T04:19:23.563 INFO:teuthology.orchestra.run.smithi022.stderr:155600K .......... .......... .......... .......... .......... 92% 15.4M 1s 2024-03-29T04:19:23.566 INFO:teuthology.orchestra.run.smithi022.stderr:155650K .......... .......... .......... .......... .......... 92% 17.7M 1s 2024-03-29T04:19:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:155700K .......... .......... .......... .......... .......... 92% 20.5M 1s 2024-03-29T04:19:23.571 INFO:teuthology.orchestra.run.smithi022.stderr:155750K .......... .......... .......... .......... .......... 92% 17.4M 1s 2024-03-29T04:19:23.575 INFO:teuthology.orchestra.run.smithi022.stderr:155800K .......... .......... .......... .......... .......... 92% 12.4M 1s 2024-03-29T04:19:23.577 INFO:teuthology.orchestra.run.smithi022.stderr:155850K .......... .......... .......... .......... .......... 92% 23.0M 1s 2024-03-29T04:19:23.580 INFO:teuthology.orchestra.run.smithi022.stderr:155900K .......... .......... .......... .......... .......... 92% 18.9M 1s 2024-03-29T04:19:23.583 INFO:teuthology.orchestra.run.smithi022.stderr:155950K .......... .......... .......... .......... .......... 92% 17.0M 1s 2024-03-29T04:19:23.586 INFO:teuthology.orchestra.run.smithi022.stderr:156000K .......... .......... .......... .......... .......... 92% 16.4M 1s 2024-03-29T04:19:23.589 INFO:teuthology.orchestra.run.smithi022.stderr:156050K .......... .......... .......... .......... .......... 92% 16.1M 1s 2024-03-29T04:19:23.591 INFO:teuthology.orchestra.run.smithi022.stderr:156100K .......... .......... .......... .......... .......... 92% 18.0M 1s 2024-03-29T04:19:23.594 INFO:teuthology.orchestra.run.smithi022.stderr:156150K .......... .......... .......... .......... .......... 92% 15.0M 1s 2024-03-29T04:19:23.597 INFO:teuthology.orchestra.run.smithi022.stderr:156200K .......... .......... .......... .......... .......... 92% 19.1M 1s 2024-03-29T04:19:23.599 INFO:teuthology.orchestra.run.smithi022.stderr:156250K .......... .......... .......... .......... .......... 92% 20.6M 1s 2024-03-29T04:19:23.602 INFO:teuthology.orchestra.run.smithi022.stderr:156300K .......... .......... .......... .......... .......... 92% 20.5M 1s 2024-03-29T04:19:23.605 INFO:teuthology.orchestra.run.smithi022.stderr:156350K .......... .......... .......... .......... .......... 92% 15.6M 1s 2024-03-29T04:19:23.608 INFO:teuthology.orchestra.run.smithi022.stderr:156400K .......... .......... .......... .......... .......... 92% 18.1M 1s 2024-03-29T04:19:23.610 INFO:teuthology.orchestra.run.smithi022.stderr:156450K .......... .......... .......... .......... .......... 92% 17.4M 1s 2024-03-29T04:19:23.614 INFO:teuthology.orchestra.run.smithi022.stderr:156500K .......... .......... .......... .......... .......... 92% 15.7M 1s 2024-03-29T04:19:23.617 INFO:teuthology.orchestra.run.smithi022.stderr:156550K .......... .......... .......... .......... .......... 92% 15.1M 1s 2024-03-29T04:19:23.620 INFO:teuthology.orchestra.run.smithi022.stderr:156600K .......... .......... .......... .......... .......... 92% 17.6M 1s 2024-03-29T04:19:23.622 INFO:teuthology.orchestra.run.smithi022.stderr:156650K .......... .......... .......... .......... .......... 92% 18.3M 1s 2024-03-29T04:19:23.625 INFO:teuthology.orchestra.run.smithi022.stderr:156700K .......... .......... .......... .......... .......... 92% 15.7M 1s 2024-03-29T04:19:23.629 INFO:teuthology.orchestra.run.smithi022.stderr:156750K .......... .......... .......... .......... .......... 93% 14.2M 1s 2024-03-29T04:19:23.632 INFO:teuthology.orchestra.run.smithi022.stderr:156800K .......... .......... .......... .......... .......... 93% 17.5M 1s 2024-03-29T04:19:23.635 INFO:teuthology.orchestra.run.smithi022.stderr:156850K .......... .......... .......... .......... .......... 93% 14.6M 1s 2024-03-29T04:19:23.639 INFO:teuthology.orchestra.run.smithi022.stderr:156900K .......... .......... .......... .......... .......... 93% 14.0M 1s 2024-03-29T04:19:23.642 INFO:teuthology.orchestra.run.smithi022.stderr:156950K .......... .......... .......... .......... .......... 93% 12.4M 1s 2024-03-29T04:19:23.646 INFO:teuthology.orchestra.run.smithi022.stderr:157000K .......... .......... .......... .......... .......... 93% 14.7M 1s 2024-03-29T04:19:23.649 INFO:teuthology.orchestra.run.smithi022.stderr:157050K .......... .......... .......... .......... .......... 93% 15.3M 1s 2024-03-29T04:19:23.652 INFO:teuthology.orchestra.run.smithi022.stderr:157100K .......... .......... .......... .......... .......... 93% 14.0M 1s 2024-03-29T04:19:23.656 INFO:teuthology.orchestra.run.smithi022.stderr:157150K .......... .......... .......... .......... .......... 93% 12.0M 1s 2024-03-29T04:19:23.660 INFO:teuthology.orchestra.run.smithi022.stderr:157200K .......... .......... .......... .......... .......... 93% 14.2M 1s 2024-03-29T04:19:23.664 INFO:teuthology.orchestra.run.smithi022.stderr:157250K .......... .......... .......... .......... .......... 93% 13.7M 1s 2024-03-29T04:19:23.668 INFO:teuthology.orchestra.run.smithi022.stderr:157300K .......... .......... .......... .......... .......... 93% 12.5M 1s 2024-03-29T04:19:23.672 INFO:teuthology.orchestra.run.smithi022.stderr:157350K .......... .......... .......... .......... .......... 93% 10.8M 1s 2024-03-29T04:19:23.675 INFO:teuthology.orchestra.run.smithi022.stderr:157400K .......... .......... .......... .......... .......... 93% 13.8M 1s 2024-03-29T04:19:23.679 INFO:teuthology.orchestra.run.smithi022.stderr:157450K .......... .......... .......... .......... .......... 93% 13.0M 1s 2024-03-29T04:19:23.683 INFO:teuthology.orchestra.run.smithi022.stderr:157500K .......... .......... .......... .......... .......... 93% 13.4M 1s 2024-03-29T04:19:23.687 INFO:teuthology.orchestra.run.smithi022.stderr:157550K .......... .......... .......... .......... .......... 93% 12.5M 1s 2024-03-29T04:19:23.690 INFO:teuthology.orchestra.run.smithi022.stderr:157600K .......... .......... .......... .......... .......... 93% 15.1M 1s 2024-03-29T04:19:23.694 INFO:teuthology.orchestra.run.smithi022.stderr:157650K .......... .......... .......... .......... .......... 93% 13.1M 1s 2024-03-29T04:19:23.698 INFO:teuthology.orchestra.run.smithi022.stderr:157700K .......... .......... .......... .......... .......... 93% 13.0M 1s 2024-03-29T04:19:23.702 INFO:teuthology.orchestra.run.smithi022.stderr:157750K .......... .......... .......... .......... .......... 93% 11.1M 1s 2024-03-29T04:19:23.706 INFO:teuthology.orchestra.run.smithi022.stderr:157800K .......... .......... .......... .......... .......... 93% 13.5M 1s 2024-03-29T04:19:23.709 INFO:teuthology.orchestra.run.smithi022.stderr:157850K .......... .......... .......... .......... .......... 93% 14.1M 1s 2024-03-29T04:19:23.712 INFO:teuthology.orchestra.run.smithi022.stderr:157900K .......... .......... .......... .......... .......... 93% 16.4M 1s 2024-03-29T04:19:23.716 INFO:teuthology.orchestra.run.smithi022.stderr:157950K .......... .......... .......... .......... .......... 93% 13.4M 1s 2024-03-29T04:19:23.719 INFO:teuthology.orchestra.run.smithi022.stderr:158000K .......... .......... .......... .......... .......... 93% 15.2M 1s 2024-03-29T04:19:23.722 INFO:teuthology.orchestra.run.smithi022.stderr:158050K .......... .......... .......... .......... .......... 93% 14.8M 1s 2024-03-29T04:19:23.725 INFO:teuthology.orchestra.run.smithi022.stderr:158100K .......... .......... .......... .......... .......... 93% 16.4M 1s 2024-03-29T04:19:23.728 INFO:teuthology.orchestra.run.smithi022.stderr:158150K .......... .......... .......... .......... .......... 93% 14.1M 1s 2024-03-29T04:19:23.731 INFO:teuthology.orchestra.run.smithi022.stderr:158200K .......... .......... .......... .......... .......... 93% 16.3M 1s 2024-03-29T04:19:23.735 INFO:teuthology.orchestra.run.smithi022.stderr:158250K .......... .......... .......... .......... .......... 93% 15.8M 1s 2024-03-29T04:19:23.738 INFO:teuthology.orchestra.run.smithi022.stderr:158300K .......... .......... .......... .......... .......... 93% 15.2M 1s 2024-03-29T04:19:23.741 INFO:teuthology.orchestra.run.smithi022.stderr:158350K .......... .......... .......... .......... .......... 93% 13.5M 1s 2024-03-29T04:19:23.744 INFO:teuthology.orchestra.run.smithi022.stderr:158400K .......... .......... .......... .......... .......... 93% 15.9M 1s 2024-03-29T04:19:23.748 INFO:teuthology.orchestra.run.smithi022.stderr:158450K .......... .......... .......... .......... .......... 94% 15.0M 1s 2024-03-29T04:19:23.751 INFO:teuthology.orchestra.run.smithi022.stderr:158500K .......... .......... .......... .......... .......... 94% 15.5M 1s 2024-03-29T04:19:23.754 INFO:teuthology.orchestra.run.smithi022.stderr:158550K .......... .......... .......... .......... .......... 94% 13.2M 1s 2024-03-29T04:19:23.758 INFO:teuthology.orchestra.run.smithi022.stderr:158600K .......... .......... .......... .......... .......... 94% 15.6M 1s 2024-03-29T04:19:23.763 INFO:teuthology.orchestra.run.smithi022.stderr:158650K .......... .......... .......... .......... .......... 94% 15.8M 1s 2024-03-29T04:19:23.764 INFO:teuthology.orchestra.run.smithi022.stderr:158700K .......... .......... .......... .......... .......... 94% 13.8M 1s 2024-03-29T04:19:23.769 INFO:teuthology.orchestra.run.smithi022.stderr:158750K .......... .......... .......... .......... .......... 94% 10.8M 1s 2024-03-29T04:19:23.773 INFO:teuthology.orchestra.run.smithi022.stderr:158800K .......... .......... .......... .......... .......... 94% 12.9M 1s 2024-03-29T04:19:23.777 INFO:teuthology.orchestra.run.smithi022.stderr:158850K .......... .......... .......... .......... .......... 94% 12.9M 1s 2024-03-29T04:19:23.780 INFO:teuthology.orchestra.run.smithi022.stderr:158900K .......... .......... .......... .......... .......... 94% 12.5M 1s 2024-03-29T04:19:23.785 INFO:teuthology.orchestra.run.smithi022.stderr:158950K .......... .......... .......... .......... .......... 94% 10.5M 1s 2024-03-29T04:19:23.789 INFO:teuthology.orchestra.run.smithi022.stderr:159000K .......... .......... .......... .......... .......... 94% 14.1M 1s 2024-03-29T04:19:23.792 INFO:teuthology.orchestra.run.smithi022.stderr:159050K .......... .......... .......... .......... .......... 94% 14.6M 1s 2024-03-29T04:19:23.795 INFO:teuthology.orchestra.run.smithi022.stderr:159100K .......... .......... .......... .......... .......... 94% 16.5M 1s 2024-03-29T04:19:23.798 INFO:teuthology.orchestra.run.smithi022.stderr:159150K .......... .......... .......... .......... .......... 94% 15.2M 1s 2024-03-29T04:19:23.801 INFO:teuthology.orchestra.run.smithi022.stderr:159200K .......... .......... .......... .......... .......... 94% 17.8M 1s 2024-03-29T04:19:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:159250K .......... .......... .......... .......... .......... 94% 17.6M 1s 2024-03-29T04:19:23.806 INFO:teuthology.orchestra.run.smithi022.stderr:159300K .......... .......... .......... .......... .......... 94% 17.8M 1s 2024-03-29T04:19:23.809 INFO:teuthology.orchestra.run.smithi022.stderr:159350K .......... .......... .......... .......... .......... 94% 15.2M 1s 2024-03-29T04:19:23.812 INFO:teuthology.orchestra.run.smithi022.stderr:159400K .......... .......... .......... .......... .......... 94% 17.7M 1s 2024-03-29T04:19:23.815 INFO:teuthology.orchestra.run.smithi022.stderr:159450K .......... .......... .......... .......... .......... 94% 15.1M 1s 2024-03-29T04:19:23.818 INFO:teuthology.orchestra.run.smithi022.stderr:159500K .......... .......... .......... .......... .......... 94% 21.1M 1s 2024-03-29T04:19:23.820 INFO:teuthology.orchestra.run.smithi022.stderr:159550K .......... .......... .......... .......... .......... 94% 19.8M 1s 2024-03-29T04:19:23.822 INFO:teuthology.orchestra.run.smithi022.stderr:159600K .......... .......... .......... .......... .......... 94% 22.6M 0s 2024-03-29T04:19:23.824 INFO:teuthology.orchestra.run.smithi022.stderr:159650K .......... .......... .......... .......... .......... 94% 22.6M 0s 2024-03-29T04:19:23.826 INFO:teuthology.orchestra.run.smithi022.stderr:159700K .......... .......... .......... .......... .......... 94% 22.6M 0s 2024-03-29T04:19:23.828 INFO:teuthology.orchestra.run.smithi022.stderr:159750K .......... .......... .......... .......... .......... 94% 24.7M 0s 2024-03-29T04:19:23.830 INFO:teuthology.orchestra.run.smithi022.stderr:159800K .......... .......... .......... .......... .......... 94% 35.4M 0s 2024-03-29T04:19:23.831 INFO:teuthology.orchestra.run.smithi022.stderr:159850K .......... .......... .......... .......... .......... 94% 32.4M 0s 2024-03-29T04:19:23.833 INFO:teuthology.orchestra.run.smithi022.stderr:159900K .......... .......... .......... .......... .......... 94% 33.4M 0s 2024-03-29T04:19:23.834 INFO:teuthology.orchestra.run.smithi022.stderr:159950K .......... .......... .......... .......... .......... 94% 28.2M 0s 2024-03-29T04:19:23.836 INFO:teuthology.orchestra.run.smithi022.stderr:160000K .......... .......... .......... .......... .......... 94% 34.7M 0s 2024-03-29T04:19:23.837 INFO:teuthology.orchestra.run.smithi022.stderr:160050K .......... .......... .......... .......... .......... 94% 34.2M 0s 2024-03-29T04:19:23.839 INFO:teuthology.orchestra.run.smithi022.stderr:160100K .......... .......... .......... .......... .......... 95% 26.8M 0s 2024-03-29T04:19:23.842 INFO:teuthology.orchestra.run.smithi022.stderr:160150K .......... .......... .......... .......... .......... 95% 22.6M 0s 2024-03-29T04:19:23.844 INFO:teuthology.orchestra.run.smithi022.stderr:160200K .......... .......... .......... .......... .......... 95% 26.5M 0s 2024-03-29T04:19:23.845 INFO:teuthology.orchestra.run.smithi022.stderr:160250K .......... .......... .......... .......... .......... 95% 29.8M 0s 2024-03-29T04:19:23.847 INFO:teuthology.orchestra.run.smithi022.stderr:160300K .......... .......... .......... .......... .......... 95% 22.0M 0s 2024-03-29T04:19:23.850 INFO:teuthology.orchestra.run.smithi022.stderr:160350K .......... .......... .......... .......... .......... 95% 15.6M 0s 2024-03-29T04:19:23.853 INFO:teuthology.orchestra.run.smithi022.stderr:160400K .......... .......... .......... .......... .......... 95% 19.2M 0s 2024-03-29T04:19:23.856 INFO:teuthology.orchestra.run.smithi022.stderr:160450K .......... .......... .......... .......... .......... 95% 18.1M 0s 2024-03-29T04:19:23.858 INFO:teuthology.orchestra.run.smithi022.stderr:160500K .......... .......... .......... .......... .......... 95% 22.2M 0s 2024-03-29T04:19:23.860 INFO:teuthology.orchestra.run.smithi022.stderr:160550K .......... .......... .......... .......... .......... 95% 20.2M 0s 2024-03-29T04:19:23.862 INFO:teuthology.orchestra.run.smithi022.stderr:160600K .......... .......... .......... .......... .......... 95% 25.2M 0s 2024-03-29T04:19:23.864 INFO:teuthology.orchestra.run.smithi022.stderr:160650K .......... .......... .......... .......... .......... 95% 23.7M 0s 2024-03-29T04:19:23.866 INFO:teuthology.orchestra.run.smithi022.stderr:160700K .......... .......... .......... .......... .......... 95% 26.2M 0s 2024-03-29T04:19:23.868 INFO:teuthology.orchestra.run.smithi022.stderr:160750K .......... .......... .......... .......... .......... 95% 22.0M 0s 2024-03-29T04:19:23.870 INFO:teuthology.orchestra.run.smithi022.stderr:160800K .......... .......... .......... .......... .......... 95% 26.4M 0s 2024-03-29T04:19:23.872 INFO:teuthology.orchestra.run.smithi022.stderr:160850K .......... .......... .......... .......... .......... 95% 25.3M 0s 2024-03-29T04:19:23.874 INFO:teuthology.orchestra.run.smithi022.stderr:160900K .......... .......... .......... .......... .......... 95% 23.3M 0s 2024-03-29T04:19:23.877 INFO:teuthology.orchestra.run.smithi022.stderr:160950K .......... .......... .......... .......... .......... 95% 17.7M 0s 2024-03-29T04:19:23.879 INFO:teuthology.orchestra.run.smithi022.stderr:161000K .......... .......... .......... .......... .......... 95% 20.1M 0s 2024-03-29T04:19:23.882 INFO:teuthology.orchestra.run.smithi022.stderr:161050K .......... .......... .......... .......... .......... 95% 20.7M 0s 2024-03-29T04:19:23.884 INFO:teuthology.orchestra.run.smithi022.stderr:161100K .......... .......... .......... .......... .......... 95% 19.9M 0s 2024-03-29T04:19:23.887 INFO:teuthology.orchestra.run.smithi022.stderr:161150K .......... .......... .......... .......... .......... 95% 17.4M 0s 2024-03-29T04:19:23.889 INFO:teuthology.orchestra.run.smithi022.stderr:161200K .......... .......... .......... .......... .......... 95% 20.8M 0s 2024-03-29T04:19:23.892 INFO:teuthology.orchestra.run.smithi022.stderr:161250K .......... .......... .......... .......... .......... 95% 19.7M 0s 2024-03-29T04:19:23.894 INFO:teuthology.orchestra.run.smithi022.stderr:161300K .......... .......... .......... .......... .......... 95% 20.1M 0s 2024-03-29T04:19:23.897 INFO:teuthology.orchestra.run.smithi022.stderr:161350K .......... .......... .......... .......... .......... 95% 16.6M 0s 2024-03-29T04:19:23.900 INFO:teuthology.orchestra.run.smithi022.stderr:161400K .......... .......... .......... .......... .......... 95% 20.0M 0s 2024-03-29T04:19:23.902 INFO:teuthology.orchestra.run.smithi022.stderr:161450K .......... .......... .......... .......... .......... 95% 20.1M 0s 2024-03-29T04:19:23.905 INFO:teuthology.orchestra.run.smithi022.stderr:161500K .......... .......... .......... .......... .......... 95% 19.8M 0s 2024-03-29T04:19:23.907 INFO:teuthology.orchestra.run.smithi022.stderr:161550K .......... .......... .......... .......... .......... 95% 16.1M 0s 2024-03-29T04:19:23.910 INFO:teuthology.orchestra.run.smithi022.stderr:161600K .......... .......... .......... .......... .......... 95% 19.4M 0s 2024-03-29T04:19:23.913 INFO:teuthology.orchestra.run.smithi022.stderr:161650K .......... .......... .......... .......... .......... 95% 19.1M 0s 2024-03-29T04:19:23.915 INFO:teuthology.orchestra.run.smithi022.stderr:161700K .......... .......... .......... .......... .......... 95% 20.4M 0s 2024-03-29T04:19:23.917 INFO:teuthology.orchestra.run.smithi022.stderr:161750K .......... .......... .......... .......... .......... 95% 18.7M 0s 2024-03-29T04:19:23.920 INFO:teuthology.orchestra.run.smithi022.stderr:161800K .......... .......... .......... .......... .......... 96% 18.5M 0s 2024-03-29T04:19:23.923 INFO:teuthology.orchestra.run.smithi022.stderr:161850K .......... .......... .......... .......... .......... 96% 17.6M 0s 2024-03-29T04:19:23.926 INFO:teuthology.orchestra.run.smithi022.stderr:161900K .......... .......... .......... .......... .......... 96% 17.0M 0s 2024-03-29T04:19:23.929 INFO:teuthology.orchestra.run.smithi022.stderr:161950K .......... .......... .......... .......... .......... 96% 14.5M 0s 2024-03-29T04:19:23.932 INFO:teuthology.orchestra.run.smithi022.stderr:162000K .......... .......... .......... .......... .......... 96% 16.8M 0s 2024-03-29T04:19:23.935 INFO:teuthology.orchestra.run.smithi022.stderr:162050K .......... .......... .......... .......... .......... 96% 17.2M 0s 2024-03-29T04:19:23.938 INFO:teuthology.orchestra.run.smithi022.stderr:162100K .......... .......... .......... .......... .......... 96% 17.0M 0s 2024-03-29T04:19:23.941 INFO:teuthology.orchestra.run.smithi022.stderr:162150K .......... .......... .......... .......... .......... 96% 14.3M 0s 2024-03-29T04:19:23.945 INFO:teuthology.orchestra.run.smithi022.stderr:162200K .......... .......... .......... .......... .......... 96% 13.5M 0s 2024-03-29T04:19:23.949 INFO:teuthology.orchestra.run.smithi022.stderr:162250K .......... .......... .......... .......... .......... 96% 13.8M 0s 2024-03-29T04:19:23.952 INFO:teuthology.orchestra.run.smithi022.stderr:162300K .......... .......... .......... .......... .......... 96% 14.1M 0s 2024-03-29T04:19:23.956 INFO:teuthology.orchestra.run.smithi022.stderr:162350K .......... .......... .......... .......... .......... 96% 11.7M 0s 2024-03-29T04:19:23.960 INFO:teuthology.orchestra.run.smithi022.stderr:162400K .......... .......... .......... .......... .......... 96% 13.7M 0s 2024-03-29T04:19:23.964 INFO:teuthology.orchestra.run.smithi022.stderr:162450K .......... .......... .......... .......... .......... 96% 12.9M 0s 2024-03-29T04:19:23.967 INFO:teuthology.orchestra.run.smithi022.stderr:162500K .......... .......... .......... .......... .......... 96% 15.5M 0s 2024-03-29T04:19:23.970 INFO:teuthology.orchestra.run.smithi022.stderr:162550K .......... .......... .......... .......... .......... 96% 14.5M 0s 2024-03-29T04:19:23.973 INFO:teuthology.orchestra.run.smithi022.stderr:162600K .......... .......... .......... .......... .......... 96% 16.6M 0s 2024-03-29T04:19:23.976 INFO:teuthology.orchestra.run.smithi022.stderr:162650K .......... .......... .......... .......... .......... 96% 16.6M 0s 2024-03-29T04:19:23.979 INFO:teuthology.orchestra.run.smithi022.stderr:162700K .......... .......... .......... .......... .......... 96% 15.4M 0s 2024-03-29T04:19:23.982 INFO:teuthology.orchestra.run.smithi022.stderr:162750K .......... .......... .......... .......... .......... 96% 14.6M 0s 2024-03-29T04:19:23.985 INFO:teuthology.orchestra.run.smithi022.stderr:162800K .......... .......... .......... .......... .......... 96% 16.7M 0s 2024-03-29T04:19:23.988 INFO:teuthology.orchestra.run.smithi022.stderr:162850K .......... .......... .......... .......... .......... 96% 16.3M 0s 2024-03-29T04:19:23.991 INFO:teuthology.orchestra.run.smithi022.stderr:162900K .......... .......... .......... .......... .......... 96% 16.8M 0s 2024-03-29T04:19:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:162950K .......... .......... .......... .......... .......... 96% 14.0M 0s 2024-03-29T04:19:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:163000K .......... .......... .......... .......... .......... 96% 16.6M 0s 2024-03-29T04:19:24.001 INFO:teuthology.orchestra.run.smithi022.stderr:163050K .......... .......... .......... .......... .......... 96% 15.7M 0s 2024-03-29T04:19:24.005 INFO:teuthology.orchestra.run.smithi022.stderr:163100K .......... .......... .......... .......... .......... 96% 11.2M 0s 2024-03-29T04:19:24.011 INFO:teuthology.orchestra.run.smithi022.stderr:163150K .......... .......... .......... .......... .......... 96% 9.13M 0s 2024-03-29T04:19:24.015 INFO:teuthology.orchestra.run.smithi022.stderr:163200K .......... .......... .......... .......... .......... 96% 10.7M 0s 2024-03-29T04:19:24.020 INFO:teuthology.orchestra.run.smithi022.stderr:163250K .......... .......... .......... .......... .......... 96% 10.8M 0s 2024-03-29T04:19:24.024 INFO:teuthology.orchestra.run.smithi022.stderr:163300K .......... .......... .......... .......... .......... 96% 10.8M 0s 2024-03-29T04:19:24.029 INFO:teuthology.orchestra.run.smithi022.stderr:163350K .......... .......... .......... .......... .......... 96% 9.27M 0s 2024-03-29T04:19:24.033 INFO:teuthology.orchestra.run.smithi022.stderr:163400K .......... .......... .......... .......... .......... 96% 13.3M 0s 2024-03-29T04:19:24.036 INFO:teuthology.orchestra.run.smithi022.stderr:163450K .......... .......... .......... .......... .......... 96% 14.5M 0s 2024-03-29T04:19:24.040 INFO:teuthology.orchestra.run.smithi022.stderr:163500K .......... .......... .......... .......... .......... 97% 15.2M 0s 2024-03-29T04:19:24.044 INFO:teuthology.orchestra.run.smithi022.stderr:163550K .......... .......... .......... .......... .......... 97% 12.9M 0s 2024-03-29T04:19:24.047 INFO:teuthology.orchestra.run.smithi022.stderr:163600K .......... .......... .......... .......... .......... 97% 14.5M 0s 2024-03-29T04:19:24.051 INFO:teuthology.orchestra.run.smithi022.stderr:163650K .......... .......... .......... .......... .......... 97% 12.9M 0s 2024-03-29T04:19:24.055 INFO:teuthology.orchestra.run.smithi022.stderr:163700K .......... .......... .......... .......... .......... 97% 11.3M 0s 2024-03-29T04:19:24.060 INFO:teuthology.orchestra.run.smithi022.stderr:163750K .......... .......... .......... .......... .......... 97% 9.89M 0s 2024-03-29T04:19:24.064 INFO:teuthology.orchestra.run.smithi022.stderr:163800K .......... .......... .......... .......... .......... 97% 11.7M 0s 2024-03-29T04:19:24.069 INFO:teuthology.orchestra.run.smithi022.stderr:163850K .......... .......... .......... .......... .......... 97% 11.1M 0s 2024-03-29T04:19:24.072 INFO:teuthology.orchestra.run.smithi022.stderr:163900K .......... .......... .......... .......... .......... 97% 13.6M 0s 2024-03-29T04:19:24.076 INFO:teuthology.orchestra.run.smithi022.stderr:163950K .......... .......... .......... .......... .......... 97% 12.3M 0s 2024-03-29T04:19:24.081 INFO:teuthology.orchestra.run.smithi022.stderr:164000K .......... .......... .......... .......... .......... 97% 9.11M 0s 2024-03-29T04:19:24.084 INFO:teuthology.orchestra.run.smithi022.stderr:164050K .......... .......... .......... .......... .......... 97% 18.9M 0s 2024-03-29T04:19:24.085 INFO:teuthology.orchestra.run.smithi022.stderr:164100K .......... .......... .......... .......... .......... 97% 27.2M 0s 2024-03-29T04:19:24.087 INFO:teuthology.orchestra.run.smithi022.stderr:164150K .......... .......... .......... .......... .......... 97% 25.5M 0s 2024-03-29T04:19:24.089 INFO:teuthology.orchestra.run.smithi022.stderr:164200K .......... .......... .......... .......... .......... 97% 29.5M 0s 2024-03-29T04:19:24.091 INFO:teuthology.orchestra.run.smithi022.stderr:164250K .......... .......... .......... .......... .......... 97% 24.6M 0s 2024-03-29T04:19:24.093 INFO:teuthology.orchestra.run.smithi022.stderr:164300K .......... .......... .......... .......... .......... 97% 24.0M 0s 2024-03-29T04:19:24.096 INFO:teuthology.orchestra.run.smithi022.stderr:164350K .......... .......... .......... .......... .......... 97% 18.7M 0s 2024-03-29T04:19:24.098 INFO:teuthology.orchestra.run.smithi022.stderr:164400K .......... .......... .......... .......... .......... 97% 20.9M 0s 2024-03-29T04:19:24.100 INFO:teuthology.orchestra.run.smithi022.stderr:164450K .......... .......... .......... .......... .......... 97% 21.0M 0s 2024-03-29T04:19:24.103 INFO:teuthology.orchestra.run.smithi022.stderr:164500K .......... .......... .......... .......... .......... 97% 20.6M 0s 2024-03-29T04:19:24.106 INFO:teuthology.orchestra.run.smithi022.stderr:164550K .......... .......... .......... .......... .......... 97% 17.2M 0s 2024-03-29T04:19:24.108 INFO:teuthology.orchestra.run.smithi022.stderr:164600K .......... .......... .......... .......... .......... 97% 19.2M 0s 2024-03-29T04:19:24.111 INFO:teuthology.orchestra.run.smithi022.stderr:164650K .......... .......... .......... .......... .......... 97% 20.4M 0s 2024-03-29T04:19:24.113 INFO:teuthology.orchestra.run.smithi022.stderr:164700K .......... .......... .......... .......... .......... 97% 19.6M 0s 2024-03-29T04:19:24.116 INFO:teuthology.orchestra.run.smithi022.stderr:164750K .......... .......... .......... .......... .......... 97% 16.1M 0s 2024-03-29T04:19:24.119 INFO:teuthology.orchestra.run.smithi022.stderr:164800K .......... .......... .......... .......... .......... 97% 18.3M 0s 2024-03-29T04:19:24.121 INFO:teuthology.orchestra.run.smithi022.stderr:164850K .......... .......... .......... .......... .......... 97% 18.1M 0s 2024-03-29T04:19:24.124 INFO:teuthology.orchestra.run.smithi022.stderr:164900K .......... .......... .......... .......... .......... 97% 18.1M 0s 2024-03-29T04:19:24.127 INFO:teuthology.orchestra.run.smithi022.stderr:164950K .......... .......... .......... .......... .......... 97% 15.4M 0s 2024-03-29T04:19:24.130 INFO:teuthology.orchestra.run.smithi022.stderr:165000K .......... .......... .......... .......... .......... 97% 17.8M 0s 2024-03-29T04:19:24.133 INFO:teuthology.orchestra.run.smithi022.stderr:165050K .......... .......... .......... .......... .......... 97% 16.7M 0s 2024-03-29T04:19:24.135 INFO:teuthology.orchestra.run.smithi022.stderr:165100K .......... .......... .......... .......... .......... 97% 19.8M 0s 2024-03-29T04:19:24.138 INFO:teuthology.orchestra.run.smithi022.stderr:165150K .......... .......... .......... .......... .......... 98% 18.2M 0s 2024-03-29T04:19:24.141 INFO:teuthology.orchestra.run.smithi022.stderr:165200K .......... .......... .......... .......... .......... 98% 19.6M 0s 2024-03-29T04:19:24.143 INFO:teuthology.orchestra.run.smithi022.stderr:165250K .......... .......... .......... .......... .......... 98% 19.5M 0s 2024-03-29T04:19:24.146 INFO:teuthology.orchestra.run.smithi022.stderr:165300K .......... .......... .......... .......... .......... 98% 19.5M 0s 2024-03-29T04:19:24.149 INFO:teuthology.orchestra.run.smithi022.stderr:165350K .......... .......... .......... .......... .......... 98% 15.9M 0s 2024-03-29T04:19:24.151 INFO:teuthology.orchestra.run.smithi022.stderr:165400K .......... .......... .......... .......... .......... 98% 18.0M 0s 2024-03-29T04:19:24.154 INFO:teuthology.orchestra.run.smithi022.stderr:165450K .......... .......... .......... .......... .......... 98% 19.5M 0s 2024-03-29T04:19:24.156 INFO:teuthology.orchestra.run.smithi022.stderr:165500K .......... .......... .......... .......... .......... 98% 28.9M 0s 2024-03-29T04:19:24.157 INFO:teuthology.orchestra.run.smithi022.stderr:165550K .......... .......... .......... .......... .......... 98% 25.3M 0s 2024-03-29T04:19:24.159 INFO:teuthology.orchestra.run.smithi022.stderr:165600K .......... .......... .......... .......... .......... 98% 28.7M 0s 2024-03-29T04:19:24.161 INFO:teuthology.orchestra.run.smithi022.stderr:165650K .......... .......... .......... .......... .......... 98% 29.4M 0s 2024-03-29T04:19:24.162 INFO:teuthology.orchestra.run.smithi022.stderr:165700K .......... .......... .......... .......... .......... 98% 29.0M 0s 2024-03-29T04:19:24.165 INFO:teuthology.orchestra.run.smithi022.stderr:165750K .......... .......... .......... .......... .......... 98% 21.6M 0s 2024-03-29T04:19:24.172 INFO:teuthology.orchestra.run.smithi022.stderr:165800K .......... .......... .......... .......... .......... 98% 25.6M 0s 2024-03-29T04:19:24.172 INFO:teuthology.orchestra.run.smithi022.stderr:165850K .......... .......... .......... .......... .......... 98% 25.4M 0s 2024-03-29T04:19:24.173 INFO:teuthology.orchestra.run.smithi022.stderr:165900K .......... .......... .......... .......... .......... 98% 25.8M 0s 2024-03-29T04:19:24.173 INFO:teuthology.orchestra.run.smithi022.stderr:165950K .......... .......... .......... .......... .......... 98% 21.7M 0s 2024-03-29T04:19:24.175 INFO:teuthology.orchestra.run.smithi022.stderr:166000K .......... .......... .......... .......... .......... 98% 25.3M 0s 2024-03-29T04:19:24.177 INFO:teuthology.orchestra.run.smithi022.stderr:166050K .......... .......... .......... .......... .......... 98% 25.1M 0s 2024-03-29T04:19:24.178 INFO:teuthology.orchestra.run.smithi022.stderr:166100K .......... .......... .......... .......... .......... 98% 25.2M 0s 2024-03-29T04:19:24.181 INFO:teuthology.orchestra.run.smithi022.stderr:166150K .......... .......... .......... .......... .......... 98% 21.1M 0s 2024-03-29T04:19:24.183 INFO:teuthology.orchestra.run.smithi022.stderr:166200K .......... .......... .......... .......... .......... 98% 24.3M 0s 2024-03-29T04:19:24.185 INFO:teuthology.orchestra.run.smithi022.stderr:166250K .......... .......... .......... .......... .......... 98% 23.6M 0s 2024-03-29T04:19:24.187 INFO:teuthology.orchestra.run.smithi022.stderr:166300K .......... .......... .......... .......... .......... 98% 21.2M 0s 2024-03-29T04:19:24.190 INFO:teuthology.orchestra.run.smithi022.stderr:166350K .......... .......... .......... .......... .......... 98% 19.0M 0s 2024-03-29T04:19:24.192 INFO:teuthology.orchestra.run.smithi022.stderr:166400K .......... .......... .......... .......... .......... 98% 21.7M 0s 2024-03-29T04:19:24.194 INFO:teuthology.orchestra.run.smithi022.stderr:166450K .......... .......... .......... .......... .......... 98% 21.5M 0s 2024-03-29T04:19:24.197 INFO:teuthology.orchestra.run.smithi022.stderr:166500K .......... .......... .......... .......... .......... 98% 22.1M 0s 2024-03-29T04:19:24.199 INFO:teuthology.orchestra.run.smithi022.stderr:166550K .......... .......... .......... .......... .......... 98% 17.7M 0s 2024-03-29T04:19:24.202 INFO:teuthology.orchestra.run.smithi022.stderr:166600K .......... .......... .......... .......... .......... 98% 21.0M 0s 2024-03-29T04:19:24.204 INFO:teuthology.orchestra.run.smithi022.stderr:166650K .......... .......... .......... .......... .......... 98% 20.9M 0s 2024-03-29T04:19:24.206 INFO:teuthology.orchestra.run.smithi022.stderr:166700K .......... .......... .......... .......... .......... 98% 20.7M 0s 2024-03-29T04:19:24.209 INFO:teuthology.orchestra.run.smithi022.stderr:166750K .......... .......... .......... .......... .......... 98% 16.8M 0s 2024-03-29T04:19:24.212 INFO:teuthology.orchestra.run.smithi022.stderr:166800K .......... .......... .......... .......... .......... 98% 19.3M 0s 2024-03-29T04:19:24.215 INFO:teuthology.orchestra.run.smithi022.stderr:166850K .......... .......... .......... .......... .......... 99% 20.5M 0s 2024-03-29T04:19:24.217 INFO:teuthology.orchestra.run.smithi022.stderr:166900K .......... .......... .......... .......... .......... 99% 19.4M 0s 2024-03-29T04:19:24.220 INFO:teuthology.orchestra.run.smithi022.stderr:166950K .......... .......... .......... .......... .......... 99% 17.3M 0s 2024-03-29T04:19:24.222 INFO:teuthology.orchestra.run.smithi022.stderr:167000K .......... .......... .......... .......... .......... 99% 20.4M 0s 2024-03-29T04:19:24.225 INFO:teuthology.orchestra.run.smithi022.stderr:167050K .......... .......... .......... .......... .......... 99% 19.4M 0s 2024-03-29T04:19:24.227 INFO:teuthology.orchestra.run.smithi022.stderr:167100K .......... .......... .......... .......... .......... 99% 19.5M 0s 2024-03-29T04:19:24.230 INFO:teuthology.orchestra.run.smithi022.stderr:167150K .......... .......... .......... .......... .......... 99% 16.8M 0s 2024-03-29T04:19:24.232 INFO:teuthology.orchestra.run.smithi022.stderr:167200K .......... .......... .......... .......... .......... 99% 19.8M 0s 2024-03-29T04:19:24.235 INFO:teuthology.orchestra.run.smithi022.stderr:167250K .......... .......... .......... .......... .......... 99% 20.0M 0s 2024-03-29T04:19:24.237 INFO:teuthology.orchestra.run.smithi022.stderr:167300K .......... .......... .......... .......... .......... 99% 19.0M 0s 2024-03-29T04:19:24.241 INFO:teuthology.orchestra.run.smithi022.stderr:167350K .......... .......... .......... .......... .......... 99% 15.2M 0s 2024-03-29T04:19:24.243 INFO:teuthology.orchestra.run.smithi022.stderr:167400K .......... .......... .......... .......... .......... 99% 18.6M 0s 2024-03-29T04:19:24.246 INFO:teuthology.orchestra.run.smithi022.stderr:167450K .......... .......... .......... .......... .......... 99% 18.3M 0s 2024-03-29T04:19:24.249 INFO:teuthology.orchestra.run.smithi022.stderr:167500K .......... .......... .......... .......... .......... 99% 16.2M 0s 2024-03-29T04:19:24.252 INFO:teuthology.orchestra.run.smithi022.stderr:167550K .......... .......... .......... .......... .......... 99% 13.6M 0s 2024-03-29T04:19:24.255 INFO:teuthology.orchestra.run.smithi022.stderr:167600K .......... .......... .......... .......... .......... 99% 18.2M 0s 2024-03-29T04:19:24.258 INFO:teuthology.orchestra.run.smithi022.stderr:167650K .......... .......... .......... .......... .......... 99% 17.4M 0s 2024-03-29T04:19:24.261 INFO:teuthology.orchestra.run.smithi022.stderr:167700K .......... .......... .......... .......... .......... 99% 16.5M 0s 2024-03-29T04:19:24.265 INFO:teuthology.orchestra.run.smithi022.stderr:167750K .......... .......... .......... .......... .......... 99% 13.0M 0s 2024-03-29T04:19:24.269 INFO:teuthology.orchestra.run.smithi022.stderr:167800K .......... .......... .......... .......... .......... 99% 11.4M 0s 2024-03-29T04:19:24.273 INFO:teuthology.orchestra.run.smithi022.stderr:167850K .......... .......... .......... .......... .......... 99% 11.4M 0s 2024-03-29T04:19:24.278 INFO:teuthology.orchestra.run.smithi022.stderr:167900K .......... .......... .......... .......... .......... 99% 12.0M 0s 2024-03-29T04:19:24.281 INFO:teuthology.orchestra.run.smithi022.stderr:167950K .......... .......... .......... .......... .......... 99% 13.1M 0s 2024-03-29T04:19:24.284 INFO:teuthology.orchestra.run.smithi022.stderr:168000K .......... .......... .......... .......... .......... 99% 16.9M 0s 2024-03-29T04:19:24.287 INFO:teuthology.orchestra.run.smithi022.stderr:168050K .......... .......... .......... .......... .......... 99% 17.9M 0s 2024-03-29T04:19:24.290 INFO:teuthology.orchestra.run.smithi022.stderr:168100K .......... .......... .......... .......... .......... 99% 17.9M 0s 2024-03-29T04:19:24.293 INFO:teuthology.orchestra.run.smithi022.stderr:168150K .......... .......... .......... .......... .......... 99% 14.3M 0s 2024-03-29T04:19:24.295 INFO:teuthology.orchestra.run.smithi022.stderr:168200K .......... .......... .......... .......... .......... 99% 25.5M 0s 2024-03-29T04:19:24.297 INFO:teuthology.orchestra.run.smithi022.stderr:168250K .......... .......... .......... .......... .......... 99% 30.2M 0s 2024-03-29T04:19:24.298 INFO:teuthology.orchestra.run.smithi022.stderr:168300K .......... .......... .......... .......... .......... 99% 27.1M 0s 2024-03-29T04:19:24.300 INFO:teuthology.orchestra.run.smithi022.stderr:168350K .......... .......... .......... .......... .......... 99% 22.4M 0s 2024-03-29T04:19:24.302 INFO:teuthology.orchestra.run.smithi022.stderr:168400K .......... .......... .......... .......... .......... 99% 29.4M 0s 2024-03-29T04:19:24.303 INFO:teuthology.orchestra.run.smithi022.stderr:168450K .......... .......... .......... .......... .......... 99% 29.4M 0s 2024-03-29T04:19:24.305 INFO:teuthology.orchestra.run.smithi022.stderr:168500K .......... .......... .......... .......... .......... 99% 29.8M 0s 2024-03-29T04:19:24.306 INFO:teuthology.orchestra.run.smithi022.stderr:168550K .......... .......... . 100% 24.9M=9.4s 2024-03-29T04:19:24.306 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-03-29T04:19:24.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29 04:19:24 (17.5 MB/s) - 'linux-5.4.tar.gz' saved [172616875/172616875] 2024-03-29T04:19:24.307 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-03-29T04:19:24.308 INFO:tasks.cephfs.test_uninlining:Running Data Uninlining test with: without clearing cache, without unmount client, without mount.open_background, without multimds, without snapshots 2024-03-29T04:19:24.308 DEBUG:tasks.cephfs.test_uninlining:setting max_mds:1 2024-03-29T04:19:24.309 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-03-29T04:19:24.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2ca398640 1 Processor -- start 2024-03-29T04:19:24.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2ca398640 1 -- start start 2024-03-29T04:19:24.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2ca398640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c415bd70 0x7ff2c406eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:24.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2ca398640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c406dc00 con 0x7ff2c40afb90 2024-03-29T04:19:24.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2ca398640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c406dee0 con 0x7ff2c406f610 2024-03-29T04:19:24.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2ca398640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c406e1c0 con 0x7ff2c415bd70 2024-03-29T04:19:24.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c37fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff2c40afb90 0x7ff2c415c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:45008/0 (socket says 172.21.15.22:45008) 2024-03-29T04:19:24.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c37fe640 1 -- 172.21.15.22:0/1905254505 learned_addr learned my addr 172.21.15.22:0/1905254505 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:19:24.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c3fff640 1 --2- 172.21.15.22:0/1905254505 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c415bd70 0x7ff2c406eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:24.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c2ffd640 1 -- 172.21.15.22:0/1905254505 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2744528865 0 0) 0x7ff2c406dc00 con 0x7ff2c40afb90 2024-03-29T04:19:24.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c2ffd640 1 -- 172.21.15.22:0/1905254505 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2ac003530 con 0x7ff2c40afb90 2024-03-29T04:19:24.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c2ffd640 1 -- 172.21.15.22:0/1905254505 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3529941976 0 0) 0x7ff2ac003530 con 0x7ff2c40afb90 2024-03-29T04:19:24.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c2ffd640 1 -- 172.21.15.22:0/1905254505 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c415bd70 msgr2=0x7ff2c406eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:24.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c2ffd640 1 --2- 172.21.15.22:0/1905254505 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c415bd70 0x7ff2c406eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c2ffd640 1 -- 172.21.15.22:0/1905254505 >> v1:172.21.15.151:6789/0 conn(0x7ff2c406f610 legacy=0x7ff2c406faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c2ffd640 1 -- 172.21.15.22:0/1905254505 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c406e4a0 con 0x7ff2c40afb90 2024-03-29T04:19:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c2ffd640 1 -- 172.21.15.22:0/1905254505 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7ff2b4001680 con 0x7ff2c40afb90 2024-03-29T04:19:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c2ffd640 1 -- 172.21.15.22:0/1905254505 >> v1:172.21.15.22:6789/0 conn(0x7ff2c40afb90 legacy=0x7ff2c415c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c2ffd640 1 --2- 172.21.15.22:0/1905254505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2ac0039f0 0x7ff2ac003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c2ffd640 1 --2- 172.21.15.22:0/1905254505 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c415bd70 0x7ff2ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c2ffd640 1 --2- 172.21.15.22:0/1905254505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ac004d50 0x7ff2ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c2ffd640 1 -- 172.21.15.22:0/1905254505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c406e1c0 con 0x7ff2ac004d50 2024-03-29T04:19:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c2ffd640 1 -- 172.21.15.22:0/1905254505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ac00dc90 con 0x7ff2ac0039f0 2024-03-29T04:19:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c2ffd640 1 -- 172.21.15.22:0/1905254505 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff2ac00de90 con 0x7ff2c415bd70 2024-03-29T04:19:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c8935640 1 --2- 172.21.15.22:0/1905254505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ac004d50 0x7ff2ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c37fe640 1 --2- 172.21.15.22:0/1905254505 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c415bd70 0x7ff2ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c3fff640 1 --2- 172.21.15.22:0/1905254505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2ac0039f0 0x7ff2ac003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c8935640 1 -- 172.21.15.22:0/1905254505 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c415bd70 msgr2=0x7ff2ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:24.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c8935640 1 --2- 172.21.15.22:0/1905254505 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c415bd70 0x7ff2ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:24.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c8935640 1 -- 172.21.15.22:0/1905254505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2ac0039f0 msgr2=0x7ff2ac003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:24.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c8935640 1 --2- 172.21.15.22:0/1905254505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2ac0039f0 0x7ff2ac003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:24.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c8935640 1 -- 172.21.15.22:0/1905254505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ac003530 con 0x7ff2ac004d50 2024-03-29T04:19:24.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.491+0000 7ff2c3fff640 1 --2- 172.21.15.22:0/1905254505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2ac0039f0 0x7ff2ac003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:19:24.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.495+0000 7ff2c8935640 1 --2- 172.21.15.22:0/1905254505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ac004d50 0x7ff2ac0051d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc001e40 tx=0x7ff2bc01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ab063c1fa1dbb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:24.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.495+0000 7ff2c2ffd640 1 -- 172.21.15.22:0/1905254505 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc001a90 con 0x7ff2ac004d50 2024-03-29T04:19:24.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.495+0000 7ff2c2ffd640 1 -- 172.21.15.22:0/1905254505 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2bc003b80 con 0x7ff2ac004d50 2024-03-29T04:19:24.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.495+0000 7ff2c2ffd640 1 -- 172.21.15.22:0/1905254505 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc005080 con 0x7ff2ac004d50 2024-03-29T04:19:24.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.495+0000 7ff2ca398640 1 -- 172.21.15.22:0/1905254505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ac004d50 msgr2=0x7ff2ac0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:24.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.495+0000 7ff2ca398640 1 --2- 172.21.15.22:0/1905254505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ac004d50 0x7ff2ac0051d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc001e40 tx=0x7ff2bc01c760 comp rx=0 tx=0).stop 2024-03-29T04:19:24.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.495+0000 7ff2c8935640 1 -- 172.21.15.22:0/1905254505 reap_dead start 2024-03-29T04:19:24.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.495+0000 7ff2ca398640 1 -- 172.21.15.22:0/1905254505 shutdown_connections 2024-03-29T04:19:24.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.495+0000 7ff2ca398640 1 -- 172.21.15.22:0/1905254505 >> 172.21.15.22:0/1905254505 conn(0x7ff2c40d0c90 msgr2=0x7ff2c40d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:24.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.495+0000 7ff2ca398640 1 -- 172.21.15.22:0/1905254505 shutdown_connections 2024-03-29T04:19:24.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.495+0000 7ff2ca398640 1 -- 172.21.15.22:0/1905254505 wait complete. 2024-03-29T04:19:24.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.495+0000 7ff2ca398640 1 Processor -- start 2024-03-29T04:19:24.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.495+0000 7ff2ca398640 1 -- start start 2024-03-29T04:19:24.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2ca398640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2ac004d50 0x7ff2c41533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:24.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2ca398640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c40aad90 0x7ff2c40ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:24.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2ca398640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2c414b4e0 0x7ff2c406b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:24.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2ca398640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c406ee20 con 0x7ff2c414b4e0 2024-03-29T04:19:24.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2ca398640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c406eca0 con 0x7ff2c40aad90 2024-03-29T04:19:24.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2ca398640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c406efa0 con 0x7ff2ac004d50 2024-03-29T04:19:24.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2c8935640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2c414b4e0 0x7ff2c406b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:24.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2c8935640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2c414b4e0 0x7ff2c406b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47488/0 (socket says 172.21.15.22:47488) 2024-03-29T04:19:24.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2c8935640 1 -- 172.21.15.22:0/3345801693 learned_addr learned my addr 172.21.15.22:0/3345801693 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:24.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2c3fff640 1 --2- 172.21.15.22:0/3345801693 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2ac004d50 0x7ff2c41533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:24.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2c37fe640 1 --2- 172.21.15.22:0/3345801693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c40aad90 0x7ff2c40ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:24.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2c8935640 1 -- 172.21.15.22:0/3345801693 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2ac004d50 msgr2=0x7ff2c41533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:24.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2c8935640 1 --2- 172.21.15.22:0/3345801693 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2ac004d50 0x7ff2c41533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:24.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2c8935640 1 -- 172.21.15.22:0/3345801693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c40aad90 msgr2=0x7ff2c40ac160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:24.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2c8935640 1 --2- 172.21.15.22:0/3345801693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c40aad90 0x7ff2c40ac160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:24.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2c8935640 1 -- 172.21.15.22:0/3345801693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c40a9cf0 con 0x7ff2c414b4e0 2024-03-29T04:19:24.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2c37fe640 1 --2- 172.21.15.22:0/3345801693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c40aad90 0x7ff2c40ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:19:24.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2c8935640 1 --2- 172.21.15.22:0/3345801693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2c414b4e0 0x7ff2c406b660 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc098170 tx=0x7ff2ac005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:24.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2c0ff9640 1 -- 172.21.15.22:0/3345801693 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc0ba070 con 0x7ff2c414b4e0 2024-03-29T04:19:24.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2c0ff9640 1 -- 172.21.15.22:0/3345801693 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2bc0012e0 con 0x7ff2c414b4e0 2024-03-29T04:19:24.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2c0ff9640 1 -- 172.21.15.22:0/3345801693 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc007dc0 con 0x7ff2c414b4e0 2024-03-29T04:19:24.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2ca398640 1 -- 172.21.15.22:0/3345801693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c40a9fd0 con 0x7ff2c414b4e0 2024-03-29T04:19:24.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2ca398640 1 -- 172.21.15.22:0/3345801693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c40aa590 con 0x7ff2c414b4e0 2024-03-29T04:19:24.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2ca398640 1 -- 172.21.15.22:0/3345801693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c4068900 con 0x7ff2c414b4e0 2024-03-29T04:19:24.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2c0ff9640 1 -- 172.21.15.22:0/3345801693 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff2bc007030 con 0x7ff2c414b4e0 2024-03-29T04:19:24.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.499+0000 7ff2c0ff9640 1 --2- 172.21.15.22:0/3345801693 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff2b0067ff0 0x7ff2b006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:24.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.503+0000 7ff2c0ff9640 1 -- 172.21.15.22:0/3345801693 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7ff2bc11fb30 con 0x7ff2c414b4e0 2024-03-29T04:19:24.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.503+0000 7ff2c3fff640 1 --2- 172.21.15.22:0/3345801693 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff2b0067ff0 0x7ff2b006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:24.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.503+0000 7ff2c3fff640 1 --2- 172.21.15.22:0/3345801693 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff2b0067ff0 0x7ff2b006a4b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff2c415b300 tx=0x7ff2b8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:24.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.507+0000 7ff2c0ff9640 1 -- 172.21.15.22:0/3345801693 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff2bc0ac020 con 0x7ff2c414b4e0 2024-03-29T04:19:24.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.795+0000 7ff2ca398640 1 -- 172.21.15.22:0/3345801693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) -- 0x7ff2c40af8d0 con 0x7ff2c414b4e0 2024-03-29T04:19:24.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.847+0000 7ff2c0ff9640 1 -- 172.21.15.22:0/3345801693 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v20) ==== 105+0+0 (secure 0 0 0) 0x7ff2bc016470 con 0x7ff2c414b4e0 2024-03-29T04:19:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.847+0000 7ff2ca398640 1 -- 172.21.15.22:0/3345801693 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff2b0067ff0 msgr2=0x7ff2b006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.847+0000 7ff2ca398640 1 --2- 172.21.15.22:0/3345801693 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff2b0067ff0 0x7ff2b006a4b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff2c415b300 tx=0x7ff2b8008040 comp rx=0 tx=0).stop 2024-03-29T04:19:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.847+0000 7ff2ca398640 1 -- 172.21.15.22:0/3345801693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2c414b4e0 msgr2=0x7ff2c406b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.847+0000 7ff2ca398640 1 --2- 172.21.15.22:0/3345801693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2c414b4e0 0x7ff2c406b660 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc098170 tx=0x7ff2ac005840 comp rx=0 tx=0).stop 2024-03-29T04:19:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.847+0000 7ff2ca398640 1 -- 172.21.15.22:0/3345801693 shutdown_connections 2024-03-29T04:19:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.847+0000 7ff2ca398640 1 --2- 172.21.15.22:0/3345801693 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff2b0067ff0 0x7ff2b006a4b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.847+0000 7ff2ca398640 1 --2- 172.21.15.22:0/3345801693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2c414b4e0 0x7ff2c406b660 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.847+0000 7ff2ca398640 1 --2- 172.21.15.22:0/3345801693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c40aad90 0x7ff2c40ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.847+0000 7ff2ca398640 1 --2- 172.21.15.22:0/3345801693 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2ac004d50 0x7ff2c41533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.847+0000 7ff2ca398640 1 -- 172.21.15.22:0/3345801693 >> 172.21.15.22:0/3345801693 conn(0x7ff2c40d0c90 msgr2=0x7ff2c4155420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.847+0000 7ff2ca398640 1 -- 172.21.15.22:0/3345801693 shutdown_connections 2024-03-29T04:19:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:24.847+0000 7ff2ca398640 1 -- 172.21.15.22:0/3345801693 wait complete. 2024-03-29T04:19:24.847 DEBUG:tasks.cephfs.test_uninlining:getting client_max_inline_size 2024-03-29T04:19:24.848 DEBUG:tasks.cephfs.test_uninlining:got client_max_inline_size:4096 2024-03-29T04:19:24.848 DEBUG:tasks.cephfs.test_uninlining:extracting tarball 2024-03-29T04:19:24.848 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && tar -x -z -f linux-5.4.tar.gz linux-5.4/fs/ceph linux-5.4/fs/orangefs linux-5.4/fs/ext2 2024-03-29T04:19:31.594 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:19:31.598 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:19:32.802 DEBUG:tasks.cephfs.test_uninlining:fetching dir list 2024-03-29T04:19:32.802 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && find linux-5.4/ -mindepth 2 -maxdepth 2 -type d 2024-03-29T04:19:32.823 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/ceph 2024-03-29T04:19:32.823 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/ext2 2024-03-29T04:19:32.823 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/orangefs 2024-03-29T04:19:32.824 DEBUG:tasks.cephfs.test_uninlining:distributing dir pins 2024-03-29T04:19:32.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:32.824 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/ceph) 2024-03-29T04:19:32.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:32.878 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/ext2) 2024-03-29T04:19:32.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:32.931 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/orangefs) 2024-03-29T04:19:32.983 DEBUG:tasks.cephfs.test_uninlining:listing files with size <= 4096 2024-03-29T04:19:32.983 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && find linux-5.4/ -type f -size -4097c 2024-03-29T04:19:33.111 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/ceph/Kconfig 2024-03-29T04:19:33.111 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/ceph/Makefile 2024-03-29T04:19:33.111 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/ceph/ceph_frag.c 2024-03-29T04:19:33.111 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/ceph/io.h 2024-03-29T04:19:33.111 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/ceph/ioctl.h 2024-03-29T04:19:33.112 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/ext2/Kconfig 2024-03-29T04:19:33.112 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/ext2/xattr_user.c 2024-03-29T04:19:33.112 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/ext2/xattr_security.c 2024-03-29T04:19:33.112 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/ext2/Makefile 2024-03-29T04:19:33.112 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/ext2/symlink.c 2024-03-29T04:19:33.112 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/ext2/acl.h 2024-03-29T04:19:33.112 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/ext2/xattr_trusted.c 2024-03-29T04:19:33.112 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/ext2/xattr.h 2024-03-29T04:19:33.112 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/orangefs/Kconfig 2024-03-29T04:19:33.112 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-29T04:19:33.112 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-29T04:19:33.112 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/orangefs/dcache.c 2024-03-29T04:19:33.112 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-29T04:19:33.112 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-29T04:19:33.112 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-29T04:19:33.113 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/orangefs/Makefile 2024-03-29T04:19:33.113 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/orangefs/symlink.c 2024-03-29T04:19:33.113 INFO:teuthology.orchestra.run.smithi022.stdout:linux-5.4/fs/orangefs/downcall.h 2024-03-29T04:19:33.113 INFO:tasks.cephfs.test_uninlining:Found 23 inlined files 2024-03-29T04:19:33.113 DEBUG:tasks.cephfs.test_uninlining:starting scrub 2024-03-29T04:19:33.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:33.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive 2024-03-29T04:19:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.259+0000 7fa9d0520640 1 Processor -- start 2024-03-29T04:19:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.259+0000 7fa9d0520640 1 -- start start 2024-03-29T04:19:33.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.259+0000 7fa9d0520640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa9c815bd50 0x7fa9c806e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.259+0000 7fa9d0520640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fa9c80cefa0 0x7fa9c80b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.259+0000 7fa9d0520640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9c80b3680 con 0x7fa9c815c130 2024-03-29T04:19:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.259+0000 7fa9d0520640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fa9c80cfd70 con 0x7fa9c815bd50 2024-03-29T04:19:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.259+0000 7fa9d0520640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fa9c80d0050 con 0x7fa9c80cefa0 2024-03-29T04:19:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.259+0000 7fa9cdabb640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fa9c815c130 0x7fa9c80ce890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50322/0 (socket says 172.21.15.22:50322) 2024-03-29T04:19:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.259+0000 7fa9cdabb640 1 -- 172.21.15.22:0/2873314788 learned_addr learned my addr 172.21.15.22:0/2873314788 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:19:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.259+0000 7fa9ce2bc640 1 --2- 172.21.15.22:0/2873314788 >> v2:172.21.15.22:3300/0 conn(0x7fa9c815bd50 0x7fa9c806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.259+0000 7fa9ceabd640 1 --2- 172.21.15.22:0/2873314788 >> v2:172.21.15.151:3300/0 conn(0x7fa9c80cefa0 0x7fa9c80b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.263+0000 7fa9cd2ba640 1 -- 172.21.15.22:0/2873314788 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3405653814 0 0) 0x7fa9c80b3680 con 0x7fa9c815c130 2024-03-29T04:19:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.263+0000 7fa9cd2ba640 1 -- 172.21.15.22:0/2873314788 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9b4003530 con 0x7fa9c815c130 2024-03-29T04:19:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.263+0000 7fa9ce2bc640 1 -- 172.21.15.22:0/2873314788 >> v2:172.21.15.151:3300/0 conn(0x7fa9c80cefa0 msgr2=0x7fa9c80b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.263+0000 7fa9ce2bc640 1 --2- 172.21.15.22:0/2873314788 >> v2:172.21.15.151:3300/0 conn(0x7fa9c80cefa0 0x7fa9c80b1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.263+0000 7fa9ce2bc640 1 -- 172.21.15.22:0/2873314788 >> v1:172.21.15.22:6789/0 conn(0x7fa9c815c130 legacy=0x7fa9c80ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:33.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.263+0000 7fa9ce2bc640 1 -- 172.21.15.22:0/2873314788 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c80b3960 con 0x7fa9c815bd50 2024-03-29T04:19:33.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.267+0000 7fa9ce2bc640 1 --2- 172.21.15.22:0/2873314788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9c815bd50 0x7fa9c806e410 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80b2fb0 tx=0x7fa9b801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cfae90ec5ca6ab7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:33.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.267+0000 7fa9cd2ba640 1 -- 172.21.15.22:0/2873314788 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b80ab610 con 0x7fa9c815bd50 2024-03-29T04:19:33.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.267+0000 7fa9cd2ba640 1 -- 172.21.15.22:0/2873314788 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9b80b0070 con 0x7fa9c815bd50 2024-03-29T04:19:33.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.267+0000 7fa9cd2ba640 1 -- 172.21.15.22:0/2873314788 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b80ab9d0 con 0x7fa9c815bd50 2024-03-29T04:19:33.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.267+0000 7fa9d0520640 1 -- 172.21.15.22:0/2873314788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9c815bd50 msgr2=0x7fa9c806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:33.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.267+0000 7fa9d0520640 1 --2- 172.21.15.22:0/2873314788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9c815bd50 0x7fa9c806e410 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80b2fb0 tx=0x7fa9b801c3f0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.267+0000 7fa9d0520640 1 -- 172.21.15.22:0/2873314788 shutdown_connections 2024-03-29T04:19:33.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.267+0000 7fa9d0520640 1 --2- 172.21.15.22:0/2873314788 >> v2:172.21.15.151:3300/0 conn(0x7fa9c80cefa0 0x7fa9c80b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.267+0000 7fa9d0520640 1 --2- 172.21.15.22:0/2873314788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9c815bd50 0x7fa9c806e410 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.267+0000 7fa9d0520640 1 -- 172.21.15.22:0/2873314788 >> 172.21.15.22:0/2873314788 conn(0x7fa9c8071b80 msgr2=0x7fa9c80b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:33.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.267+0000 7fa9d0520640 1 -- 172.21.15.22:0/2873314788 shutdown_connections 2024-03-29T04:19:33.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.267+0000 7fa9d0520640 1 -- 172.21.15.22:0/2873314788 wait complete. 2024-03-29T04:19:33.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.267+0000 7fa9d0520640 1 Processor -- start 2024-03-29T04:19:33.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.267+0000 7fa9d0520640 1 -- start start 2024-03-29T04:19:33.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9d0520640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa9c80cefa0 0x7fa9c8157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9d0520640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9c815bd50 0x7fa9c80c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9d0520640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c815c130 0x7fa9c80c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9d0520640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c80ce340 con 0x7fa9c815bd50 2024-03-29T04:19:33.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9d0520640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c80ce1c0 con 0x7fa9c815c130 2024-03-29T04:19:33.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9d0520640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa9c80ce4c0 con 0x7fa9c80cefa0 2024-03-29T04:19:33.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9cdabb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9c815bd50 0x7fa9c80c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9cdabb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9c815bd50 0x7fa9c80c79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37784/0 (socket says 172.21.15.22:37784) 2024-03-29T04:19:33.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9cdabb640 1 -- 172.21.15.22:0/2619910640 learned_addr learned my addr 172.21.15.22:0/2619910640 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:33.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9ceabd640 1 --2- 172.21.15.22:0/2619910640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c815c130 0x7fa9c80c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9ce2bc640 1 --2- 172.21.15.22:0/2619910640 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa9c80cefa0 0x7fa9c8157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9cdabb640 1 -- 172.21.15.22:0/2619910640 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa9c80cefa0 msgr2=0x7fa9c8157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:33.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9cdabb640 1 --2- 172.21.15.22:0/2619910640 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa9c80cefa0 0x7fa9c8157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9cdabb640 1 -- 172.21.15.22:0/2619910640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c815c130 msgr2=0x7fa9c80c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:33.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9cdabb640 1 --2- 172.21.15.22:0/2619910640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c815c130 0x7fa9c80c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9cdabb640 1 -- 172.21.15.22:0/2619910640 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c815a770 con 0x7fa9c815bd50 2024-03-29T04:19:33.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9ceabd640 1 --2- 172.21.15.22:0/2619910640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c815c130 0x7fa9c80c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:19:33.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9cdabb640 1 --2- 172.21.15.22:0/2619910640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9c815bd50 0x7fa9c80c79a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400c420 tx=0x7fa9c400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:33.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9beffd640 1 -- 172.21.15.22:0/2619910640 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4017070 con 0x7fa9c815bd50 2024-03-29T04:19:33.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9beffd640 1 -- 172.21.15.22:0/2619910640 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c4005d10 con 0x7fa9c815bd50 2024-03-29T04:19:33.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9d0520640 1 -- 172.21.15.22:0/2619910640 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c815aa50 con 0x7fa9c815bd50 2024-03-29T04:19:33.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9beffd640 1 -- 172.21.15.22:0/2619910640 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4007310 con 0x7fa9c815bd50 2024-03-29T04:19:33.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9d0520640 1 -- 172.21.15.22:0/2619910640 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c815b2f0 con 0x7fa9c815bd50 2024-03-29T04:19:33.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9beffd640 1 -- 172.21.15.22:0/2619910640 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fa9c401a070 con 0x7fa9c815bd50 2024-03-29T04:19:33.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9beffd640 1 --2- 172.21.15.22:0/2619910640 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa9ac067fc0 0x7fa9ac06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9beffd640 1 -- 172.21.15.22:0/2619910640 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fa9c4085ba0 con 0x7fa9c815bd50 2024-03-29T04:19:33.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9ce2bc640 1 --2- 172.21.15.22:0/2619910640 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa9ac067fc0 0x7fa9ac06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.271+0000 7fa9d36a2000 1 Processor -- start 2024-03-29T04:19:33.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9d36a2000 1 -- start start 2024-03-29T04:19:33.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9ce2bc640 1 --2- 172.21.15.22:0/2619910640 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa9ac067fc0 0x7fa9ac06a480 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80af000 tx=0x7fa9b8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:33.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9d36a2000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44e39000 0x555c44e393e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9d36a2000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39920 0x555c44def310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9d36a2000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44def850 0x555c44defc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9d36a2000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555c44c3dc60 con 0x555c44e39000 2024-03-29T04:19:33.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9d36a2000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555c44bdab80 con 0x555c44e39920 2024-03-29T04:19:33.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9d36a2000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x555c44c39c10 con 0x555c44def850 2024-03-29T04:19:33.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9ce2bc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44e39000 0x555c44e393e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9ce2bc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44e39000 0x555c44e393e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37786/0 (socket says 172.21.15.22:37786) 2024-03-29T04:19:33.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9ce2bc640 1 -- 172.21.15.22:0/2175664731 learned_addr learned my addr 172.21.15.22:0/2175664731 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:33.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9ceabd640 1 --2- 172.21.15.22:0/2175664731 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44def850 0x555c44defc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9cdabb640 1 --2- 172.21.15.22:0/2175664731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39920 0x555c44def310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9ce2bc640 1 -- 172.21.15.22:0/2175664731 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44def850 msgr2=0x555c44defc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:33.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9ce2bc640 1 --2- 172.21.15.22:0/2175664731 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44def850 0x555c44defc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9ce2bc640 1 -- 172.21.15.22:0/2175664731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39920 msgr2=0x555c44def310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:33.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9ce2bc640 1 --2- 172.21.15.22:0/2175664731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39920 0x555c44def310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9ce2bc640 1 -- 172.21.15.22:0/2175664731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c44ce4f10 con 0x555c44e39000 2024-03-29T04:19:33.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9ceabd640 1 --2- 172.21.15.22:0/2175664731 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44def850 0x555c44defc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:19:33.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9cdabb640 1 --2- 172.21.15.22:0/2175664731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39920 0x555c44def310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:19:33.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9ce2bc640 1 --2- 172.21.15.22:0/2175664731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44e39000 0x555c44e393e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80ad040 tx=0x7fa9b8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd32b75d2480a8ca server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:33.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9abfff640 1 -- 172.21.15.22:0/2175664731 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b80b5740 con 0x555c44e39000 2024-03-29T04:19:33.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9abfff640 1 -- 172.21.15.22:0/2175664731 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9b80c5070 con 0x555c44e39000 2024-03-29T04:19:33.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9abfff640 1 -- 172.21.15.22:0/2175664731 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b80b5ae0 con 0x555c44e39000 2024-03-29T04:19:33.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9d36a2000 1 -- 172.21.15.22:0/2175664731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44e39000 msgr2=0x555c44e393e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:33.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9d36a2000 1 --2- 172.21.15.22:0/2175664731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44e39000 0x555c44e393e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80ad040 tx=0x7fa9b8004740 comp rx=0 tx=0).stop 2024-03-29T04:19:33.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9d36a2000 1 -- 172.21.15.22:0/2175664731 shutdown_connections 2024-03-29T04:19:33.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9d36a2000 1 --2- 172.21.15.22:0/2175664731 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44def850 0x555c44defc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9d36a2000 1 --2- 172.21.15.22:0/2175664731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39920 0x555c44def310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9d36a2000 1 --2- 172.21.15.22:0/2175664731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44e39000 0x555c44e393e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9d36a2000 1 -- 172.21.15.22:0/2175664731 >> 172.21.15.22:0/2175664731 conn(0x555c44beabd0 msgr2=0x555c44dedbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:33.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9d36a2000 1 -- 172.21.15.22:0/2175664731 shutdown_connections 2024-03-29T04:19:33.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9d36a2000 1 -- 172.21.15.22:0/2175664731 wait complete. 2024-03-29T04:19:33.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.275+0000 7fa9d36a2000 1 -- start start 2024-03-29T04:19:33.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9d36a2000 1 Processor -- start 2024-03-29T04:19:33.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9d36a2000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44def850 0x555c44df3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9d36a2000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39000 0x555c44e1d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9d36a2000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44e39920 0x555c44e1d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9d36a2000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555c44bfb2a0 con 0x555c44def850 2024-03-29T04:19:33.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9d36a2000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555c44bfc760 con 0x555c44e39000 2024-03-29T04:19:33.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9d36a2000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x555c44bfeee0 con 0x555c44e39920 2024-03-29T04:19:33.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9d36a2000 10 client.0 _ll_register_callbacks cb 0x555c44ddaaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:19:33.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9ce2bc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44def850 0x555c44df3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9ce2bc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44def850 0x555c44df3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37790/0 (socket says 172.21.15.22:37790) 2024-03-29T04:19:33.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9ce2bc640 1 -- 172.21.15.22:0/1693955137 learned_addr learned my addr 172.21.15.22:0/1693955137 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:33.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9ceabd640 1 --2- 172.21.15.22:0/1693955137 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44e39920 0x555c44e1d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9cdabb640 1 --2- 172.21.15.22:0/1693955137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39000 0x555c44e1d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9ce2bc640 1 -- 172.21.15.22:0/1693955137 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44e39920 msgr2=0x555c44e1d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:33.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9ce2bc640 1 --2- 172.21.15.22:0/1693955137 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44e39920 0x555c44e1d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9ce2bc640 1 -- 172.21.15.22:0/1693955137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39000 msgr2=0x555c44e1d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:33.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9ce2bc640 1 --2- 172.21.15.22:0/1693955137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39000 0x555c44e1d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9ce2bc640 1 -- 172.21.15.22:0/1693955137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c44c178d0 con 0x555c44def850 2024-03-29T04:19:33.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9cdabb640 1 --2- 172.21.15.22:0/1693955137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39000 0x555c44e1d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:19:33.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9ceabd640 1 --2- 172.21.15.22:0/1693955137 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44e39920 0x555c44e1d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:19:33.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9ce2bc640 1 --2- 172.21.15.22:0/1693955137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44def850 0x555c44df3e10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa9b801c430 tx=0x7fa9b801ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6c7eee1c53dcf9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:33.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-29T04:19:33.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9a97fa640 1 -- 172.21.15.22:0/1693955137 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b80ce070 con 0x555c44def850 2024-03-29T04:19:33.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9a97fa640 1 -- 172.21.15.22:0/1693955137 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fa9b8003de0 con 0x555c44def850 2024-03-29T04:19:33.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9d36a2000 1 -- 172.21.15.22:0/1693955137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9b80c5070 con 0x555c44def850 2024-03-29T04:19:33.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9a97fa640 1 -- 172.21.15.22:0/1693955137 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa9b80d9050 con 0x555c44def850 2024-03-29T04:19:33.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9d36a2000 10 client.5784 fetch_fsmap learned FSMap version 20 2024-03-29T04:19:33.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9d36a2000 1 -- 172.21.15.22:0/1693955137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x555c44c830d0 con 0x555c44def850 2024-03-29T04:19:33.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9a97fa640 1 -- 172.21.15.22:0/1693955137 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9b80c5070 con 0x555c44def850 2024-03-29T04:19:33.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9a97fa640 1 -- 172.21.15.22:0/1693955137 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fa9b80dc020 con 0x555c44def850 2024-03-29T04:19:33.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9d36a2000 10 client.5784 fetch_fsmap finished waiting for FSMap version 20 2024-03-29T04:19:33.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9d36a2000 10 client.5784 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-29T04:19:33.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9d36a2000 20 client.5784 populate_metadata read hostname 'smithi022' 2024-03-29T04:19:33.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9d36a2000 1 --2- 172.21.15.22:0/1693955137 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x555c44e3f640 0x555c44e3fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9d36a2000 4 client.5784 mds_command: new command op to 5529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:19:33.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9d36a2000 1 -- 172.21.15.22:0/1693955137 --> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c44bd8f70 con 0x555c44e3f640 2024-03-29T04:19:33.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9cdabb640 1 --2- 172.21.15.22:0/1693955137 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x555c44e3f640 0x555c44e3fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9a97fa640 10 client.5784 ms_handle_connect on v2:172.21.15.22:6834/2966923505 2024-03-29T04:19:33.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9cdabb640 1 --2- 172.21.15.22:0/1693955137 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x555c44e3f640 0x555c44e3fa50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13d1a98c045dfc73 server_cookie=26d0b61d0586267f in_seq=0 out_seq=0 2024-03-29T04:19:33.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9a97fa640 1 -- 172.21.15.22:0/1693955137 <== mds.0 v2:172.21.15.22:6834/2966923505 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x555c44bd8f70 con 0x555c44e3f640 2024-03-29T04:19:33.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9a97fa640 10 client.5784 handle_command_reply: tid=0 2024-03-29T04:19:33.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.279+0000 7fa9d36a2000 1 client.5784 shutdown 2024-03-29T04:19:33.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.283+0000 7fa9d36a2000 1 -- 172.21.15.22:0/1693955137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44def850 msgr2=0x555c44df3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:33.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.283+0000 7fa9d36a2000 1 --2- 172.21.15.22:0/1693955137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44def850 0x555c44df3e10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa9b801c430 tx=0x7fa9b801ac30 comp rx=0 tx=0).stop 2024-03-29T04:19:33.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.283+0000 7fa9d36a2000 1 -- 172.21.15.22:0/1693955137 shutdown_connections 2024-03-29T04:19:33.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.283+0000 7fa9d36a2000 1 --2- 172.21.15.22:0/1693955137 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x555c44e3f640 0x555c44e3fa50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.283+0000 7fa9d36a2000 1 --2- 172.21.15.22:0/1693955137 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44e39920 0x555c44e1d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.283+0000 7fa9d36a2000 1 --2- 172.21.15.22:0/1693955137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39000 0x555c44e1d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.283+0000 7fa9d36a2000 1 --2- 172.21.15.22:0/1693955137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44def850 0x555c44df3e10 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.283+0000 7fa9a97fa640 0 client.5784 ms_handle_reset on v2:172.21.15.22:6834/2966923505 2024-03-29T04:19:33.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.283+0000 7fa9d36a2000 1 -- 172.21.15.22:0/1693955137 >> 172.21.15.22:0/1693955137 conn(0x555c44beabd0 msgr2=0x555c44dedbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:33.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.283+0000 7fa9d36a2000 1 -- 172.21.15.22:0/1693955137 shutdown_connections 2024-03-29T04:19:33.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.283+0000 7fa9d36a2000 1 -- 172.21.15.22:0/1693955137 wait complete. 2024-03-29T04:19:33.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.283+0000 7fa9d36a2000 20 client.5784 trim_cache size 0 max 16384 2024-03-29T04:19:33.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.299+0000 7fa9d36a2000 1 Processor -- start 2024-03-29T04:19:33.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.299+0000 7fa9d36a2000 1 -- start start 2024-03-29T04:19:33.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.299+0000 7fa9d36a2000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44def850 0x555c44e45c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.299+0000 7fa9d36a2000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44e39000 0x555c44e4f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.299+0000 7fa9d36a2000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39920 0x555c44e4fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.299+0000 7fa9d36a2000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c8203dc0 con 0x555c44e39000 2024-03-29T04:19:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.299+0000 7fa9d36a2000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c8202e00 con 0x555c44e39920 2024-03-29T04:19:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.299+0000 7fa9d36a2000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fa9c80b63a0 con 0x555c44def850 2024-03-29T04:19:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.299+0000 7fa9cdabb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44e39000 0x555c44e4f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.299+0000 7fa9cdabb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44e39000 0x555c44e4f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37806/0 (socket says 172.21.15.22:37806) 2024-03-29T04:19:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.299+0000 7fa9cdabb640 1 -- 172.21.15.22:0/3362113308 learned_addr learned my addr 172.21.15.22:0/3362113308 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.299+0000 7fa9cdabb640 1 -- 172.21.15.22:0/3362113308 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44def850 msgr2=0x555c44e45c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.299+0000 7fa9ce2bc640 1 --2- 172.21.15.22:0/3362113308 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44def850 0x555c44e45c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.299+0000 7fa9ceabd640 1 --2- 172.21.15.22:0/3362113308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39920 0x555c44e4fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.299+0000 7fa9cdabb640 1 --2- 172.21.15.22:0/3362113308 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44def850 0x555c44e45c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.299+0000 7fa9cdabb640 1 -- 172.21.15.22:0/3362113308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39920 msgr2=0x555c44e4fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.299+0000 7fa9cdabb640 1 --2- 172.21.15.22:0/3362113308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39920 0x555c44e4fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.299+0000 7fa9cdabb640 1 -- 172.21.15.22:0/3362113308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c44c0b410 con 0x555c44e39000 2024-03-29T04:19:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.299+0000 7fa9cdabb640 1 --2- 172.21.15.22:0/3362113308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44e39000 0x555c44e4f700 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40557e0 tx=0x7fa9c4074040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d0078dba003d62 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:33.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.299+0000 7fa9ab7fe640 1 -- 172.21.15.22:0/3362113308 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4053de0 con 0x555c44e39000 2024-03-29T04:19:33.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.299+0000 7fa9ab7fe640 1 -- 172.21.15.22:0/3362113308 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c4052150 con 0x555c44e39000 2024-03-29T04:19:33.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9ab7fe640 1 -- 172.21.15.22:0/3362113308 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c40076d0 con 0x555c44e39000 2024-03-29T04:19:33.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 1 -- 172.21.15.22:0/3362113308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44e39000 msgr2=0x555c44e4f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:33.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 1 --2- 172.21.15.22:0/3362113308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44e39000 0x555c44e4f700 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40557e0 tx=0x7fa9c4074040 comp rx=0 tx=0).stop 2024-03-29T04:19:33.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 1 -- 172.21.15.22:0/3362113308 shutdown_connections 2024-03-29T04:19:33.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 1 --2- 172.21.15.22:0/3362113308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39920 0x555c44e4fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 1 --2- 172.21.15.22:0/3362113308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44e39000 0x555c44e4f700 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 1 --2- 172.21.15.22:0/3362113308 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44def850 0x555c44e45c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 1 -- 172.21.15.22:0/3362113308 >> 172.21.15.22:0/3362113308 conn(0x555c44beabd0 msgr2=0x555c44e3dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:33.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 1 -- 172.21.15.22:0/3362113308 shutdown_connections 2024-03-29T04:19:33.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 1 -- 172.21.15.22:0/3362113308 wait complete. 2024-03-29T04:19:33.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 1 -- start start 2024-03-29T04:19:33.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 1 Processor -- start 2024-03-29T04:19:33.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44def850 0x555c44dee3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39000 0x555c44dee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44e39920 0x555c44e41a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555c44cf95f0 con 0x555c44e39920 2024-03-29T04:19:33.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c80b69c0 con 0x555c44e39000 2024-03-29T04:19:33.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x555c44bc52c0 con 0x555c44def850 2024-03-29T04:19:33.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 10 client.0 _ll_register_callbacks cb 0x7fa9b8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:19:33.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9ceabd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44e39920 0x555c44e41a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9ceabd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44e39920 0x555c44e41a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37818/0 (socket says 172.21.15.22:37818) 2024-03-29T04:19:33.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9ceabd640 1 -- 172.21.15.22:0/2224557791 learned_addr learned my addr 172.21.15.22:0/2224557791 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:33.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9cdabb640 1 --2- 172.21.15.22:0/2224557791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39000 0x555c44dee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9ce2bc640 1 --2- 172.21.15.22:0/2224557791 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44def850 0x555c44dee3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9ceabd640 1 -- 172.21.15.22:0/2224557791 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44def850 msgr2=0x555c44dee3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:33.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9ceabd640 1 --2- 172.21.15.22:0/2224557791 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44def850 0x555c44dee3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9ceabd640 1 -- 172.21.15.22:0/2224557791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39000 msgr2=0x555c44dee8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:33.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9ceabd640 1 --2- 172.21.15.22:0/2224557791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39000 0x555c44dee8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9ceabd640 1 -- 172.21.15.22:0/2224557791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c44cbf1e0 con 0x555c44e39920 2024-03-29T04:19:33.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9ceabd640 1 --2- 172.21.15.22:0/2224557791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44e39920 0x555c44e41a30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0007720 tx=0x7fa9c0007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d941192c1cb67bb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:33.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-29T04:19:33.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9a97fa640 1 -- 172.21.15.22:0/2224557791 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c001c070 con 0x555c44e39920 2024-03-29T04:19:33.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9a97fa640 1 -- 172.21.15.22:0/2224557791 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fa9c0014c50 con 0x555c44e39920 2024-03-29T04:19:33.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 1 -- 172.21.15.22:0/2224557791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c4052150 con 0x555c44e39920 2024-03-29T04:19:33.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9a97fa640 1 -- 172.21.15.22:0/2224557791 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa9c0027050 con 0x555c44e39920 2024-03-29T04:19:33.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 10 client.5790 fetch_fsmap learned FSMap version 20 2024-03-29T04:19:33.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 1 -- 172.21.15.22:0/2224557791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x555c448f93b0 con 0x555c44e39920 2024-03-29T04:19:33.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9a97fa640 1 -- 172.21.15.22:0/2224557791 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c4052150 con 0x555c44e39920 2024-03-29T04:19:33.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9a97fa640 1 -- 172.21.15.22:0/2224557791 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fa9c002a020 con 0x555c44e39920 2024-03-29T04:19:33.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 10 client.5790 fetch_fsmap finished waiting for FSMap version 20 2024-03-29T04:19:33.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 10 client.5790 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-29T04:19:33.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 20 client.5790 populate_metadata read hostname 'smithi022' 2024-03-29T04:19:33.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 1 --2- 172.21.15.22:0/2224557791 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x555c44e41f70 0x555c44e42380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 4 client.5790 mds_command: new command op to 5529 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-03-29T04:19:33.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9d36a2000 1 -- 172.21.15.22:0/2224557791 --> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x555c44c4b1a0 con 0x555c44e41f70 2024-03-29T04:19:33.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.303+0000 7fa9ce2bc640 1 --2- 172.21.15.22:0/2224557791 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x555c44e41f70 0x555c44e42380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9a97fa640 10 client.5790 ms_handle_connect on v2:172.21.15.22:6834/2966923505 2024-03-29T04:19:33.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9ce2bc640 1 --2- 172.21.15.22:0/2224557791 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x555c44e41f70 0x555c44e42380 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e76e9654fce8b319 server_cookie=1097e187044f98b2 in_seq=0 out_seq=0 2024-03-29T04:19:33.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9a97fa640 1 -- 172.21.15.22:0/2224557791 <== mds.0 v2:172.21.15.22:6834/2966923505 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555c44c4b1a0 con 0x555c44e41f70 2024-03-29T04:19:33.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9a97fa640 10 client.5790 handle_command_reply: tid=0 2024-03-29T04:19:33.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d36a2000 1 client.5790 shutdown 2024-03-29T04:19:33.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d36a2000 1 -- 172.21.15.22:0/2224557791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44e39920 msgr2=0x555c44e41a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:33.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d36a2000 1 --2- 172.21.15.22:0/2224557791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44e39920 0x555c44e41a30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0007720 tx=0x7fa9c0007be0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d36a2000 1 -- 172.21.15.22:0/2224557791 shutdown_connections 2024-03-29T04:19:33.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d36a2000 1 --2- 172.21.15.22:0/2224557791 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x555c44e41f70 0x555c44e42380 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d36a2000 1 --2- 172.21.15.22:0/2224557791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c44e39920 0x555c44e41a30 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d36a2000 1 --2- 172.21.15.22:0/2224557791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c44e39000 0x555c44dee8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d36a2000 1 --2- 172.21.15.22:0/2224557791 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x555c44def850 0x555c44dee3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9a97fa640 0 client.5790 ms_handle_reset on v2:172.21.15.22:6834/2966923505 2024-03-29T04:19:33.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d36a2000 1 -- 172.21.15.22:0/2224557791 >> 172.21.15.22:0/2224557791 conn(0x555c44beabd0 msgr2=0x555c44e3e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:33.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d36a2000 1 -- 172.21.15.22:0/2224557791 shutdown_connections 2024-03-29T04:19:33.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d36a2000 1 -- 172.21.15.22:0/2224557791 wait complete. 2024-03-29T04:19:33.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d36a2000 20 client.5790 trim_cache size 0 max 16384 2024-03-29T04:19:33.293 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-29T04:19:33.293 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-29T04:19:33.293 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "ad732467-bc36-471e-8646-b2f190b4f928", 2024-03-29T04:19:33.293 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-29T04:19:33.293 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-29T04:19:33.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d0520640 1 -- 172.21.15.22:0/2619910640 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa9ac067fc0 msgr2=0x7fa9ac06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:33.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d0520640 1 --2- 172.21.15.22:0/2619910640 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa9ac067fc0 0x7fa9ac06a480 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80af000 tx=0x7fa9b8008040 comp rx=0 tx=0).stop 2024-03-29T04:19:33.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d0520640 1 -- 172.21.15.22:0/2619910640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9c815bd50 msgr2=0x7fa9c80c79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:33.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d0520640 1 --2- 172.21.15.22:0/2619910640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9c815bd50 0x7fa9c80c79a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400c420 tx=0x7fa9c400c8e0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d0520640 1 -- 172.21.15.22:0/2619910640 shutdown_connections 2024-03-29T04:19:33.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d0520640 1 --2- 172.21.15.22:0/2619910640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c815c130 0x7fa9c80c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d0520640 1 --2- 172.21.15.22:0/2619910640 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fa9ac067fc0 0x7fa9ac06a480 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d0520640 1 --2- 172.21.15.22:0/2619910640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9c815bd50 0x7fa9c80c79a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d0520640 1 --2- 172.21.15.22:0/2619910640 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fa9c80cefa0 0x7fa9c8157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d0520640 1 -- 172.21.15.22:0/2619910640 >> 172.21.15.22:0/2619910640 conn(0x7fa9c8071b80 msgr2=0x7fa9c80d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:33.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d0520640 1 -- 172.21.15.22:0/2619910640 shutdown_connections 2024-03-29T04:19:33.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.307+0000 7fa9d0520640 1 -- 172.21.15.22:0/2619910640 wait complete. 2024-03-29T04:19:33.312 DEBUG:tasks.cephfs.test_uninlining:scrub start response: {'return_code': 0, 'scrub_tag': 'ad732467-bc36-471e-8646-b2f190b4f928', 'mode': 'asynchronous'} 2024-03-29T04:19:33.313 DEBUG:tasks.cephfs.test_uninlining:waiting for scrub to complete 2024-03-29T04:19:33.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:19:33.313 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-03-29T04:19:33.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.487+0000 7f6533746640 1 Processor -- start 2024-03-29T04:19:33.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.487+0000 7f6533746640 1 -- start start 2024-03-29T04:19:33.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.487+0000 7f6533746640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f652c06f9e0 0x7f652c06b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.487+0000 7f6533746640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f652c0d0bc0 0x7f652c15c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.487+0000 7f6533746640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f652c15d190 con 0x7f652c06cfe0 2024-03-29T04:19:33.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.487+0000 7f6533746640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f652c06cb50 con 0x7f652c0d0bc0 2024-03-29T04:19:33.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.487+0000 7f6533746640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f652c06bf40 con 0x7f652c06f9e0 2024-03-29T04:19:33.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.491+0000 7f6531ce3640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f652c06cfe0 0x7f652c06fdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:50334/0 (socket says 172.21.15.22:50334) 2024-03-29T04:19:33.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.491+0000 7f6531ce3640 1 -- 172.21.15.22:0/3472535712 learned_addr learned my addr 172.21.15.22:0/3472535712 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:19:33.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.491+0000 7f6530ce1640 1 --2- 172.21.15.22:0/3472535712 >> v2:172.21.15.151:3300/0 conn(0x7f652c0d0bc0 0x7f652c15c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.491+0000 7f65314e2640 1 --2- 172.21.15.22:0/3472535712 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f652c06f9e0 0x7f652c06b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.491+0000 7f652bfff640 1 -- 172.21.15.22:0/3472535712 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2975879033 0 0) 0x7f652c15d190 con 0x7f652c06cfe0 2024-03-29T04:19:33.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.491+0000 7f652bfff640 1 -- 172.21.15.22:0/3472535712 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6514003530 con 0x7f652c06cfe0 2024-03-29T04:19:33.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.491+0000 7f6530ce1640 1 -- 172.21.15.22:0/3472535712 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f652c06f9e0 msgr2=0x7f652c06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:33.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.491+0000 7f6530ce1640 1 --2- 172.21.15.22:0/3472535712 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f652c06f9e0 0x7f652c06b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.491+0000 7f6530ce1640 1 -- 172.21.15.22:0/3472535712 >> v1:172.21.15.22:6789/0 conn(0x7f652c06cfe0 legacy=0x7f652c06fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:33.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.491+0000 7f6530ce1640 1 -- 172.21.15.22:0/3472535712 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652c06c220 con 0x7f652c0d0bc0 2024-03-29T04:19:33.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.491+0000 7f65314e2640 1 --2- 172.21.15.22:0/3472535712 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f652c06f9e0 0x7f652c06b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:19:33.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.491+0000 7f6530ce1640 1 --2- 172.21.15.22:0/3472535712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f652c0d0bc0 0x7f652c15c860 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f65200097f0 tx=0x7f652001c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ccdfc57bbae65d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:33.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.495+0000 7f652bfff640 1 -- 172.21.15.22:0/3472535712 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65200b1070 con 0x7f652c0d0bc0 2024-03-29T04:19:33.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.495+0000 7f652bfff640 1 -- 172.21.15.22:0/3472535712 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65200ac600 con 0x7f652c0d0bc0 2024-03-29T04:19:33.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.495+0000 7f652bfff640 1 -- 172.21.15.22:0/3472535712 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65200aca10 con 0x7f652c0d0bc0 2024-03-29T04:19:33.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.495+0000 7f6533746640 1 -- 172.21.15.22:0/3472535712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f652c0d0bc0 msgr2=0x7f652c15c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.495+0000 7f6533746640 1 --2- 172.21.15.22:0/3472535712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f652c0d0bc0 0x7f652c15c860 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f65200097f0 tx=0x7f652001c490 comp rx=0 tx=0).stop 2024-03-29T04:19:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.495+0000 7f6533746640 1 -- 172.21.15.22:0/3472535712 shutdown_connections 2024-03-29T04:19:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.495+0000 7f6533746640 1 --2- 172.21.15.22:0/3472535712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f652c0d0bc0 0x7f652c15c860 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.495+0000 7f6533746640 1 --2- 172.21.15.22:0/3472535712 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f652c06f9e0 0x7f652c06b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.495+0000 7f6533746640 1 -- 172.21.15.22:0/3472535712 >> 172.21.15.22:0/3472535712 conn(0x7f652c0d1070 msgr2=0x7f652c0d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.495+0000 7f6533746640 1 -- 172.21.15.22:0/3472535712 shutdown_connections 2024-03-29T04:19:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.495+0000 7f6533746640 1 -- 172.21.15.22:0/3472535712 wait complete. 2024-03-29T04:19:33.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.495+0000 7f6533746640 1 Processor -- start 2024-03-29T04:19:33.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.495+0000 7f6533746640 1 -- start start 2024-03-29T04:19:33.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f6533746640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f652c06cfe0 0x7f652c0afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f6533746640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f652c06f9e0 0x7f652c1537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f6533746640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f652c0d0bc0 0x7f652c072520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f6533746640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f652c15cf20 con 0x7f652c06cfe0 2024-03-29T04:19:33.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f6533746640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f652c15cda0 con 0x7f652c0d0bc0 2024-03-29T04:19:33.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f6533746640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f652c1696b0 con 0x7f652c06f9e0 2024-03-29T04:19:33.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f65314e2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f652c06cfe0 0x7f652c0afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f65314e2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f652c06cfe0 0x7f652c0afdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37826/0 (socket says 172.21.15.22:37826) 2024-03-29T04:19:33.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f65314e2640 1 -- 172.21.15.22:0/1071676752 learned_addr learned my addr 172.21.15.22:0/1071676752 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:33.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f6530ce1640 1 --2- 172.21.15.22:0/1071676752 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f652c06f9e0 0x7f652c1537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f6531ce3640 1 --2- 172.21.15.22:0/1071676752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f652c0d0bc0 0x7f652c072520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f65314e2640 1 -- 172.21.15.22:0/1071676752 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f652c06f9e0 msgr2=0x7f652c1537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:33.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f65314e2640 1 --2- 172.21.15.22:0/1071676752 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f652c06f9e0 0x7f652c1537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f65314e2640 1 -- 172.21.15.22:0/1071676752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f652c0d0bc0 msgr2=0x7f652c072520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:33.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f65314e2640 1 --2- 172.21.15.22:0/1071676752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f652c0d0bc0 0x7f652c072520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f65314e2640 1 -- 172.21.15.22:0/1071676752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652c13bd00 con 0x7f652c06cfe0 2024-03-29T04:19:33.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f6531ce3640 1 --2- 172.21.15.22:0/1071676752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f652c0d0bc0 0x7f652c072520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:19:33.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f65314e2640 1 --2- 172.21.15.22:0/1071676752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f652c06cfe0 0x7f652c0afdc0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f651c00ec10 tx=0x7f651c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:33.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f6529ffb640 1 -- 172.21.15.22:0/1071676752 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c017070 con 0x7f652c06cfe0 2024-03-29T04:19:33.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f6529ffb640 1 -- 172.21.15.22:0/1071676752 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f651c00cc30 con 0x7f652c06cfe0 2024-03-29T04:19:33.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f6533746640 1 -- 172.21.15.22:0/1071676752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f652c13bfe0 con 0x7f652c06cfe0 2024-03-29T04:19:33.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f6529ffb640 1 -- 172.21.15.22:0/1071676752 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c013470 con 0x7f652c06cfe0 2024-03-29T04:19:33.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.499+0000 7f6533746640 1 -- 172.21.15.22:0/1071676752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f652c13c5a0 con 0x7f652c06cfe0 2024-03-29T04:19:33.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f6529ffb640 1 -- 172.21.15.22:0/1071676752 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f651c01a070 con 0x7f652c06cfe0 2024-03-29T04:19:33.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f6529ffb640 1 --2- 172.21.15.22:0/1071676752 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6518067ff0 0x7f651806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f6529ffb640 1 -- 172.21.15.22:0/1071676752 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f651c086930 con 0x7f652c06cfe0 2024-03-29T04:19:33.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f6530ce1640 1 --2- 172.21.15.22:0/1071676752 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6518067ff0 0x7f651806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f65368c8000 1 Processor -- start 2024-03-29T04:19:33.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f65368c8000 1 -- start start 2024-03-29T04:19:33.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f6530ce1640 1 --2- 172.21.15.22:0/1071676752 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6518067ff0 0x7f651806a4b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f65200ae000 tx=0x7f65200a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:33.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f65368c8000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5556011dca60 0x5556011dce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f65368c8000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556011dd380 0x55560118fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f65368c8000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011900f0 0x5556011904d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f65368c8000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555601074e20 con 0x5556011900f0 2024-03-29T04:19:33.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f65368c8000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555600fddb00 con 0x5556011dd380 2024-03-29T04:19:33.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f65368c8000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x555600f7a940 con 0x5556011dca60 2024-03-29T04:19:33.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f6531ce3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011900f0 0x5556011904d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f6531ce3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011900f0 0x5556011904d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37830/0 (socket says 172.21.15.22:37830) 2024-03-29T04:19:33.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f6531ce3640 1 -- 172.21.15.22:0/2926209002 learned_addr learned my addr 172.21.15.22:0/2926209002 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:33.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f6530ce1640 1 --2- 172.21.15.22:0/2926209002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556011dd380 0x55560118fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f65314e2640 1 --2- 172.21.15.22:0/2926209002 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5556011dca60 0x5556011dce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f6531ce3640 1 -- 172.21.15.22:0/2926209002 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5556011dca60 msgr2=0x5556011dce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:33.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f6531ce3640 1 --2- 172.21.15.22:0/2926209002 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5556011dca60 0x5556011dce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f6531ce3640 1 -- 172.21.15.22:0/2926209002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556011dd380 msgr2=0x55560118fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:33.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f6531ce3640 1 --2- 172.21.15.22:0/2926209002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556011dd380 0x55560118fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f6531ce3640 1 -- 172.21.15.22:0/2926209002 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555600f63380 con 0x5556011900f0 2024-03-29T04:19:33.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f6530ce1640 1 --2- 172.21.15.22:0/2926209002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556011dd380 0x55560118fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:19:33.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f6531ce3640 1 --2- 172.21.15.22:0/2926209002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011900f0 0x5556011904d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6524002070 tx=0x7f6524000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f5fb276ea23442 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:33.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f6512ffd640 1 -- 172.21.15.22:0/2926209002 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6524019070 con 0x5556011900f0 2024-03-29T04:19:33.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f6512ffd640 1 -- 172.21.15.22:0/2926209002 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6524014480 con 0x5556011900f0 2024-03-29T04:19:33.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.503+0000 7f6512ffd640 1 -- 172.21.15.22:0/2926209002 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6524014830 con 0x5556011900f0 2024-03-29T04:19:33.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f65368c8000 1 -- 172.21.15.22:0/2926209002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011900f0 msgr2=0x5556011904d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:33.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f65368c8000 1 --2- 172.21.15.22:0/2926209002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011900f0 0x5556011904d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6524002070 tx=0x7f6524000f50 comp rx=0 tx=0).stop 2024-03-29T04:19:33.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f65368c8000 1 -- 172.21.15.22:0/2926209002 shutdown_connections 2024-03-29T04:19:33.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f65368c8000 1 --2- 172.21.15.22:0/2926209002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011900f0 0x5556011904d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f65368c8000 1 --2- 172.21.15.22:0/2926209002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556011dd380 0x55560118fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f65368c8000 1 --2- 172.21.15.22:0/2926209002 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5556011dca60 0x5556011dce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f65368c8000 1 -- 172.21.15.22:0/2926209002 >> 172.21.15.22:0/2926209002 conn(0x5556011b93a0 msgr2=0x5556011bb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:33.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f65368c8000 1 -- 172.21.15.22:0/2926209002 shutdown_connections 2024-03-29T04:19:33.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f65368c8000 1 -- 172.21.15.22:0/2926209002 wait complete. 2024-03-29T04:19:33.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f65368c8000 1 -- start start 2024-03-29T04:19:33.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f65368c8000 1 Processor -- start 2024-03-29T04:19:33.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f65368c8000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556011900f0 0x5556011e3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f65368c8000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5556011dca60 0x5556011e40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f65368c8000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011dd380 0x555601199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f65368c8000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555600f9b1e0 con 0x5556011dd380 2024-03-29T04:19:33.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f65368c8000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555600fd9ab0 con 0x5556011900f0 2024-03-29T04:19:33.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f65368c8000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x555600f78940 con 0x5556011dca60 2024-03-29T04:19:33.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f65368c8000 10 client.0 _ll_register_callbacks cb 0x5556011796a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:19:33.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f6531ce3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011dd380 0x555601199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f6531ce3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011dd380 0x555601199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37842/0 (socket says 172.21.15.22:37842) 2024-03-29T04:19:33.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f6531ce3640 1 -- 172.21.15.22:0/1072454758 learned_addr learned my addr 172.21.15.22:0/1072454758 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:33.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f65314e2640 1 --2- 172.21.15.22:0/1072454758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556011900f0 0x5556011e3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.507+0000 7f6530ce1640 1 --2- 172.21.15.22:0/1072454758 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5556011dca60 0x5556011e40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f6531ce3640 1 -- 172.21.15.22:0/1072454758 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5556011dca60 msgr2=0x5556011e40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:33.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f6531ce3640 1 --2- 172.21.15.22:0/1072454758 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5556011dca60 0x5556011e40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f6531ce3640 1 -- 172.21.15.22:0/1072454758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556011900f0 msgr2=0x5556011e3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:33.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f6531ce3640 1 --2- 172.21.15.22:0/1072454758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556011900f0 0x5556011e3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f6531ce3640 1 -- 172.21.15.22:0/1072454758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555600f99b70 con 0x5556011dd380 2024-03-29T04:19:33.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f6531ce3640 1 --2- 172.21.15.22:0/1072454758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011dd380 0x555601199bc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6524000f90 tx=0x7f6524001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faea0ac64b3091a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:33.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f650bfff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-29T04:19:33.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f650bfff640 1 -- 172.21.15.22:0/1072454758 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6524019070 con 0x5556011dd380 2024-03-29T04:19:33.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f650bfff640 1 -- 172.21.15.22:0/1072454758 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f6524004d80 con 0x5556011dd380 2024-03-29T04:19:33.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f65368c8000 1 -- 172.21.15.22:0/1072454758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6524014480 con 0x5556011dd380 2024-03-29T04:19:33.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f650bfff640 1 -- 172.21.15.22:0/1072454758 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f652402c050 con 0x5556011dd380 2024-03-29T04:19:33.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f65368c8000 10 client.5802 fetch_fsmap learned FSMap version 20 2024-03-29T04:19:33.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f65368c8000 1 -- 172.21.15.22:0/1072454758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x555600f7ab90 con 0x5556011dd380 2024-03-29T04:19:33.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f650bfff640 1 -- 172.21.15.22:0/1072454758 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6524014480 con 0x5556011dd380 2024-03-29T04:19:33.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f650bfff640 1 -- 172.21.15.22:0/1072454758 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f652402f020 con 0x5556011dd380 2024-03-29T04:19:33.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f65368c8000 10 client.5802 fetch_fsmap finished waiting for FSMap version 20 2024-03-29T04:19:33.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f65368c8000 10 client.5802 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-29T04:19:33.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f65368c8000 20 client.5802 populate_metadata read hostname 'smithi022' 2024-03-29T04:19:33.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f65368c8000 1 --2- 172.21.15.22:0/1072454758 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x55560119a900 0x55560119ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f65368c8000 4 client.5802 mds_command: new command op to 5529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:19:33.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f65368c8000 1 -- 172.21.15.22:0/1072454758 --> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555600f78dd0 con 0x55560119a900 2024-03-29T04:19:33.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f65314e2640 1 --2- 172.21.15.22:0/1072454758 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x55560119a900 0x55560119ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.511+0000 7f65314e2640 1 --2- 172.21.15.22:0/1072454758 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x55560119a900 0x55560119ace0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7026e60f52fa2a44 server_cookie=30ab6e8c4b208f56 in_seq=0 out_seq=0 2024-03-29T04:19:33.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.515+0000 7f650bfff640 10 client.5802 ms_handle_connect on v2:172.21.15.22:6834/2966923505 2024-03-29T04:19:33.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.515+0000 7f650bfff640 1 -- 172.21.15.22:0/1072454758 <== mds.0 v2:172.21.15.22:6834/2966923505 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x555600f78dd0 con 0x55560119a900 2024-03-29T04:19:33.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.515+0000 7f650bfff640 10 client.5802 handle_command_reply: tid=0 2024-03-29T04:19:33.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.515+0000 7f65368c8000 1 client.5802 shutdown 2024-03-29T04:19:33.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.515+0000 7f65368c8000 1 -- 172.21.15.22:0/1072454758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011dd380 msgr2=0x555601199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:33.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.515+0000 7f65368c8000 1 --2- 172.21.15.22:0/1072454758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011dd380 0x555601199bc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6524000f90 tx=0x7f6524001260 comp rx=0 tx=0).stop 2024-03-29T04:19:33.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.515+0000 7f65368c8000 1 -- 172.21.15.22:0/1072454758 shutdown_connections 2024-03-29T04:19:33.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.515+0000 7f65368c8000 1 --2- 172.21.15.22:0/1072454758 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x55560119a900 0x55560119ace0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.515+0000 7f65368c8000 1 --2- 172.21.15.22:0/1072454758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011dd380 0x555601199bc0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.515+0000 7f65368c8000 1 --2- 172.21.15.22:0/1072454758 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5556011dca60 0x5556011e40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.515+0000 7f65368c8000 1 --2- 172.21.15.22:0/1072454758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556011900f0 0x5556011e3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.515+0000 7f650bfff640 0 client.5802 ms_handle_reset on v2:172.21.15.22:6834/2966923505 2024-03-29T04:19:33.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.515+0000 7f65368c8000 1 -- 172.21.15.22:0/1072454758 >> 172.21.15.22:0/1072454758 conn(0x5556011b93a0 msgr2=0x55560118aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:33.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.515+0000 7f65368c8000 1 -- 172.21.15.22:0/1072454758 shutdown_connections 2024-03-29T04:19:33.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.515+0000 7f65368c8000 1 -- 172.21.15.22:0/1072454758 wait complete. 2024-03-29T04:19:33.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.515+0000 7f65368c8000 20 client.5802 trim_cache size 0 max 16384 2024-03-29T04:19:33.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f65368c8000 1 Processor -- start 2024-03-29T04:19:33.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f65368c8000 1 -- start start 2024-03-29T04:19:33.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f65368c8000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556011900f0 0x55560118a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f65368c8000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011dca60 0x55560118ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f65368c8000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5556011dd380 0x5556011df770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f65368c8000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f652c0b5990 con 0x5556011dca60 2024-03-29T04:19:33.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f65368c8000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f652c14e750 con 0x5556011900f0 2024-03-29T04:19:33.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f65368c8000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f652c0b5340 con 0x5556011dd380 2024-03-29T04:19:33.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f6530ce1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011dca60 0x55560118ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f6530ce1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011dca60 0x55560118ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37854/0 (socket says 172.21.15.22:37854) 2024-03-29T04:19:33.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f6530ce1640 1 -- 172.21.15.22:0/959873106 learned_addr learned my addr 172.21.15.22:0/959873106 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:33.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f65314e2640 1 --2- 172.21.15.22:0/959873106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556011900f0 0x55560118a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f6531ce3640 1 --2- 172.21.15.22:0/959873106 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5556011dd380 0x5556011df770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f6530ce1640 1 -- 172.21.15.22:0/959873106 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5556011dd380 msgr2=0x5556011df770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:33.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f6530ce1640 1 --2- 172.21.15.22:0/959873106 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5556011dd380 0x5556011df770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f6530ce1640 1 -- 172.21.15.22:0/959873106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556011900f0 msgr2=0x55560118a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:33.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f6530ce1640 1 --2- 172.21.15.22:0/959873106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556011900f0 0x55560118a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f6530ce1640 1 -- 172.21.15.22:0/959873106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556011dd1b0 con 0x5556011dca60 2024-03-29T04:19:33.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f6530ce1640 1 --2- 172.21.15.22:0/959873106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011dca60 0x55560118ab50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f652001c4d0 tx=0x7f6520009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a3b33f1a0e7ae4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:33.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f65127fc640 1 -- 172.21.15.22:0/959873106 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65200b1040 con 0x5556011dca60 2024-03-29T04:19:33.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f65127fc640 1 -- 172.21.15.22:0/959873106 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65200acac0 con 0x5556011dca60 2024-03-29T04:19:33.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f65127fc640 1 -- 172.21.15.22:0/959873106 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65200b6440 con 0x5556011dca60 2024-03-29T04:19:33.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f65368c8000 1 -- 172.21.15.22:0/959873106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011dca60 msgr2=0x55560118ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:33.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f65368c8000 1 --2- 172.21.15.22:0/959873106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011dca60 0x55560118ab50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f652001c4d0 tx=0x7f6520009190 comp rx=0 tx=0).stop 2024-03-29T04:19:33.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f65368c8000 1 -- 172.21.15.22:0/959873106 shutdown_connections 2024-03-29T04:19:33.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f65368c8000 1 --2- 172.21.15.22:0/959873106 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5556011dd380 0x5556011df770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f65368c8000 1 --2- 172.21.15.22:0/959873106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011dca60 0x55560118ab50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f65368c8000 1 --2- 172.21.15.22:0/959873106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556011900f0 0x55560118a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f65368c8000 1 -- 172.21.15.22:0/959873106 >> 172.21.15.22:0/959873106 conn(0x5556011b93a0 msgr2=0x555601189f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:33.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f65368c8000 1 -- 172.21.15.22:0/959873106 shutdown_connections 2024-03-29T04:19:33.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.539+0000 7f65368c8000 1 -- 172.21.15.22:0/959873106 wait complete. 2024-03-29T04:19:33.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f65368c8000 1 -- start start 2024-03-29T04:19:33.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f65368c8000 1 Processor -- start 2024-03-29T04:19:33.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f65368c8000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5556011900f0 0x55560124eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f65368c8000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556011dca60 0x55560124f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f65368c8000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011dd380 0x5556011e1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f65368c8000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555600fa9150 con 0x5556011dd380 2024-03-29T04:19:33.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f65368c8000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5556010ac750 con 0x5556011dca60 2024-03-29T04:19:33.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f65368c8000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x555600f94750 con 0x5556011900f0 2024-03-29T04:19:33.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f65368c8000 10 client.0 _ll_register_callbacks cb 0x7f6524017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:19:33.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f6531ce3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011dd380 0x5556011e1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f6531ce3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011dd380 0x5556011e1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37870/0 (socket says 172.21.15.22:37870) 2024-03-29T04:19:33.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f6531ce3640 1 -- 172.21.15.22:0/3752621751 learned_addr learned my addr 172.21.15.22:0/3752621751 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:19:33.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f6530ce1640 1 --2- 172.21.15.22:0/3752621751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556011dca60 0x55560124f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f65314e2640 1 --2- 172.21.15.22:0/3752621751 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5556011900f0 0x55560124eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f6531ce3640 1 -- 172.21.15.22:0/3752621751 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5556011900f0 msgr2=0x55560124eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:33.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f6531ce3640 1 --2- 172.21.15.22:0/3752621751 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5556011900f0 0x55560124eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f6531ce3640 1 -- 172.21.15.22:0/3752621751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556011dca60 msgr2=0x55560124f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:19:33.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f6531ce3640 1 --2- 172.21.15.22:0/3752621751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556011dca60 0x55560124f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f6531ce3640 1 -- 172.21.15.22:0/3752621751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55560105e470 con 0x5556011dd380 2024-03-29T04:19:33.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f6531ce3640 1 --2- 172.21.15.22:0/3752621751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011dd380 0x5556011e1a90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6524007400 tx=0x7f6524009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164fe43e54e44e1b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:19:33.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f650bfff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-29T04:19:33.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f650bfff640 1 -- 172.21.15.22:0/3752621751 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6524034070 con 0x5556011dd380 2024-03-29T04:19:33.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f650bfff640 1 -- 172.21.15.22:0/3752621751 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f6524004d80 con 0x5556011dd380 2024-03-29T04:19:33.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f65368c8000 1 -- 172.21.15.22:0/3752621751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65200acac0 con 0x5556011dd380 2024-03-29T04:19:33.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f650bfff640 1 -- 172.21.15.22:0/3752621751 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f652403f050 con 0x5556011dd380 2024-03-29T04:19:33.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f65368c8000 10 client.5808 fetch_fsmap learned FSMap version 20 2024-03-29T04:19:33.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f65368c8000 1 -- 172.21.15.22:0/3752621751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x555600f6c1e0 con 0x5556011dd380 2024-03-29T04:19:33.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f650bfff640 1 -- 172.21.15.22:0/3752621751 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65200acac0 con 0x5556011dd380 2024-03-29T04:19:33.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f650bfff640 1 -- 172.21.15.22:0/3752621751 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f6524042020 con 0x5556011dd380 2024-03-29T04:19:33.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f65368c8000 10 client.5808 fetch_fsmap finished waiting for FSMap version 20 2024-03-29T04:19:33.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f65368c8000 10 client.5808 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-29T04:19:33.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f65368c8000 20 client.5808 populate_metadata read hostname 'smithi022' 2024-03-29T04:19:33.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f65368c8000 1 --2- 172.21.15.22:0/3752621751 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x5556011e1fd0 0x5556011e23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:19:33.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f65368c8000 4 client.5808 mds_command: new command op to 5529 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T04:19:33.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f65368c8000 1 -- 172.21.15.22:0/3752621751 --> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555600feb040 con 0x5556011e1fd0 2024-03-29T04:19:33.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f65314e2640 1 --2- 172.21.15.22:0/3752621751 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x5556011e1fd0 0x5556011e23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:19:33.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f650bfff640 10 client.5808 ms_handle_connect on v2:172.21.15.22:6834/2966923505 2024-03-29T04:19:33.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.543+0000 7f65314e2640 1 --2- 172.21.15.22:0/3752621751 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x5556011e1fd0 0x5556011e23e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff2938d7da04ce4e server_cookie=eb84b1a57b46ce0f in_seq=0 out_seq=0 2024-03-29T04:19:33.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f650bfff640 1 -- 172.21.15.22:0/3752621751 <== mds.0 v2:172.21.15.22:6834/2966923505 1 ==== command_reply(tid 0: 0 ) ==== 8+0+564 (crc 0 0 0) 0x555600feb040 con 0x5556011e1fd0 2024-03-29T04:19:33.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f650bfff640 10 client.5808 handle_command_reply: tid=0 2024-03-29T04:19:33.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f65368c8000 1 client.5808 shutdown 2024-03-29T04:19:33.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f65368c8000 1 -- 172.21.15.22:0/3752621751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011dd380 msgr2=0x5556011e1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:33.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f65368c8000 1 --2- 172.21.15.22:0/3752621751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011dd380 0x5556011e1a90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6524007400 tx=0x7f6524009040 comp rx=0 tx=0).stop 2024-03-29T04:19:33.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f65368c8000 1 -- 172.21.15.22:0/3752621751 shutdown_connections 2024-03-29T04:19:33.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f65368c8000 1 --2- 172.21.15.22:0/3752621751 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x5556011e1fd0 0x5556011e23e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f65368c8000 1 --2- 172.21.15.22:0/3752621751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556011dd380 0x5556011e1a90 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f65368c8000 1 --2- 172.21.15.22:0/3752621751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556011dca60 0x55560124f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f650bfff640 0 client.5808 ms_handle_reset on v2:172.21.15.22:6834/2966923505 2024-03-29T04:19:33.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f65368c8000 1 --2- 172.21.15.22:0/3752621751 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5556011900f0 0x55560124eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f65368c8000 1 -- 172.21.15.22:0/3752621751 >> 172.21.15.22:0/3752621751 conn(0x5556011b93a0 msgr2=0x555601199b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:33.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f65368c8000 1 -- 172.21.15.22:0/3752621751 shutdown_connections 2024-03-29T04:19:33.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f65368c8000 1 -- 172.21.15.22:0/3752621751 wait complete. 2024-03-29T04:19:33.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f65368c8000 20 client.5808 trim_cache size 0 max 16384 2024-03-29T04:19:33.533 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-29T04:19:33.533 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-29T04:19:33.533 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-29T04:19:33.533 INFO:teuthology.orchestra.run.smithi022.stdout: "ad732467-bc36-471e-8646-b2f190b4f928": { 2024-03-29T04:19:33.533 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-29T04:19:33.533 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "ad732467-bc36-471e-8646-b2f190b4f928", 2024-03-29T04:19:33.534 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive" 2024-03-29T04:19:33.534 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:19:33.534 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-29T04:19:33.534 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_stats": { 2024-03-29T04:19:33.534 INFO:teuthology.orchestra.run.smithi022.stdout: "ad732467-bc36-471e-8646-b2f190b4f928": { 2024-03-29T04:19:33.534 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-03-29T04:19:33.307684+0000", 2024-03-29T04:19:33.534 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-29T04:19:33.534 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-03-29T04:19:33.534 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-03-29T04:19:33.534 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-03-29T04:19:33.534 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-03-29T04:19:33.534 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:19:33.534 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:19:33.534 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-29T04:19:33.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f6533746640 1 -- 172.21.15.22:0/1071676752 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6518067ff0 msgr2=0x7f651806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:33.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f6533746640 1 --2- 172.21.15.22:0/1071676752 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6518067ff0 0x7f651806a4b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f65200ae000 tx=0x7f65200a7000 comp rx=0 tx=0).stop 2024-03-29T04:19:33.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f6533746640 1 -- 172.21.15.22:0/1071676752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f652c06cfe0 msgr2=0x7f652c0afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:19:33.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f6533746640 1 --2- 172.21.15.22:0/1071676752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f652c06cfe0 0x7f652c0afdc0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f651c00ec10 tx=0x7f651c00c520 comp rx=0 tx=0).stop 2024-03-29T04:19:33.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f6533746640 1 -- 172.21.15.22:0/1071676752 shutdown_connections 2024-03-29T04:19:33.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f6533746640 1 --2- 172.21.15.22:0/1071676752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f652c0d0bc0 0x7f652c072520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f6533746640 1 --2- 172.21.15.22:0/1071676752 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f652c06f9e0 0x7f652c1537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f6533746640 1 --2- 172.21.15.22:0/1071676752 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6518067ff0 0x7f651806a4b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f6533746640 1 --2- 172.21.15.22:0/1071676752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f652c06cfe0 0x7f652c0afdc0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:19:33.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f6533746640 1 -- 172.21.15.22:0/1071676752 >> 172.21.15.22:0/1071676752 conn(0x7f652c0d1070 msgr2=0x7f652c06fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:19:33.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f6533746640 1 -- 172.21.15.22:0/1071676752 shutdown_connections 2024-03-29T04:19:33.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:19:33.547+0000 7f6533746640 1 -- 172.21.15.22:0/1071676752 wait complete. 2024-03-29T04:19:33.550 INFO:tasks.cephfs.filesystem:scrub status for tag:ad732467-bc36-471e-8646-b2f190b4f928 - {'path': '/', 'tag': 'ad732467-bc36-471e-8646-b2f190b4f928', 'options': 'recursive'} 2024-03-29T04:20:01.642 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:20:01.645 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:20:03.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:20:03.551 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-03-29T04:20:03.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.707+0000 7fe2ee9ef640 1 Processor -- start 2024-03-29T04:20:03.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.707+0000 7fe2ee9ef640 1 -- start start 2024-03-29T04:20:03.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.711+0000 7fe2ee9ef640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe2e815afc0 0x7fe2e80b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:03.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.711+0000 7fe2ee9ef640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe2e80b4520 0x7fe2e806f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:03.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.711+0000 7fe2ee9ef640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fe2e806ff20 0x7fe2e806c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:03.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.711+0000 7fe2ee9ef640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fe2e8070c30 con 0x7fe2e815afc0 2024-03-29T04:20:03.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.711+0000 7fe2ee9ef640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fe2e80b4960 con 0x7fe2e806ff20 2024-03-29T04:20:03.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.711+0000 7fe2ee9ef640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe2e80b4ba0 con 0x7fe2e80b4520 2024-03-29T04:20:03.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.711+0000 7fe2e7fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe2e815afc0 0x7fe2e80b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:03.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.711+0000 7fe2e7fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe2e815afc0 0x7fe2e80b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52888/0 (socket says 172.21.15.22:52888) 2024-03-29T04:20:03.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.711+0000 7fe2e7fff640 1 -- 172.21.15.22:0/93490477 learned_addr learned my addr 172.21.15.22:0/93490477 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:03.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.711+0000 7fe2ecf8c640 1 --2- 172.21.15.22:0/93490477 >> v2:172.21.15.151:3300/0 conn(0x7fe2e806ff20 0x7fe2e806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:03.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.711+0000 7fe2e77fe640 1 --2- 172.21.15.22:0/93490477 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe2e80b4520 0x7fe2e806f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:03.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.711+0000 7fe2e7fff640 1 -- 172.21.15.22:0/93490477 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe2e80b4520 msgr2=0x7fe2e806f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:03.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.711+0000 7fe2e7fff640 1 --2- 172.21.15.22:0/93490477 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe2e80b4520 0x7fe2e806f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.711+0000 7fe2e7fff640 1 -- 172.21.15.22:0/93490477 >> v2:172.21.15.151:3300/0 conn(0x7fe2e806ff20 msgr2=0x7fe2e806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:03.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.711+0000 7fe2e7fff640 1 --2- 172.21.15.22:0/93490477 >> v2:172.21.15.151:3300/0 conn(0x7fe2e806ff20 0x7fe2e806c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.711+0000 7fe2e7fff640 1 -- 172.21.15.22:0/93490477 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e815ec60 con 0x7fe2e815afc0 2024-03-29T04:20:03.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.711+0000 7fe2e77fe640 1 --2- 172.21.15.22:0/93490477 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe2e80b4520 0x7fe2e806f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:20:03.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.711+0000 7fe2e7fff640 1 --2- 172.21.15.22:0/93490477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e815afc0 0x7fe2e80b3fe0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80b5da0 tx=0x7fe2e001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3306cd0a3ae45c2d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:03.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.711+0000 7fe2e6ffd640 1 -- 172.21.15.22:0/93490477 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e00b0070 con 0x7fe2e815afc0 2024-03-29T04:20:03.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.711+0000 7fe2e6ffd640 1 -- 172.21.15.22:0/93490477 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e00ab600 con 0x7fe2e815afc0 2024-03-29T04:20:03.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2e6ffd640 1 -- 172.21.15.22:0/93490477 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e00ab9d0 con 0x7fe2e815afc0 2024-03-29T04:20:03.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2ee9ef640 1 -- 172.21.15.22:0/93490477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e815afc0 msgr2=0x7fe2e80b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:03.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2ee9ef640 1 --2- 172.21.15.22:0/93490477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e815afc0 0x7fe2e80b3fe0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80b5da0 tx=0x7fe2e001c260 comp rx=0 tx=0).stop 2024-03-29T04:20:03.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2ee9ef640 1 -- 172.21.15.22:0/93490477 shutdown_connections 2024-03-29T04:20:03.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2ee9ef640 1 --2- 172.21.15.22:0/93490477 >> v2:172.21.15.151:3300/0 conn(0x7fe2e806ff20 0x7fe2e806c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2ee9ef640 1 --2- 172.21.15.22:0/93490477 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe2e80b4520 0x7fe2e806f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2ee9ef640 1 --2- 172.21.15.22:0/93490477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e815afc0 0x7fe2e80b3fe0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2ee9ef640 1 -- 172.21.15.22:0/93490477 >> 172.21.15.22:0/93490477 conn(0x7fe2e80d1070 msgr2=0x7fe2e80d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:03.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2ee9ef640 1 -- 172.21.15.22:0/93490477 shutdown_connections 2024-03-29T04:20:03.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2ee9ef640 1 -- 172.21.15.22:0/93490477 wait complete. 2024-03-29T04:20:03.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2ee9ef640 1 Processor -- start 2024-03-29T04:20:03.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2ee9ef640 1 -- start start 2024-03-29T04:20:03.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2ee9ef640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe2e806ff20 0x7fe2e8143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:03.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2ee9ef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e80b4520 0x7fe2e81629d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:03.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2ee9ef640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2e815afc0 0x7fe2e8162f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:03.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2ee9ef640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e815fa20 con 0x7fe2e80b4520 2024-03-29T04:20:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2ee9ef640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e815b3a0 con 0x7fe2e815afc0 2024-03-29T04:20:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2ee9ef640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe2e8144270 con 0x7fe2e806ff20 2024-03-29T04:20:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2e77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e80b4520 0x7fe2e81629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2e77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e80b4520 0x7fe2e81629d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52900/0 (socket says 172.21.15.22:52900) 2024-03-29T04:20:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2e77fe640 1 -- 172.21.15.22:0/1040545389 learned_addr learned my addr 172.21.15.22:0/1040545389 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2ecf8c640 1 --2- 172.21.15.22:0/1040545389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2e815afc0 0x7fe2e8162f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2e7fff640 1 --2- 172.21.15.22:0/1040545389 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe2e806ff20 0x7fe2e8143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2e77fe640 1 -- 172.21.15.22:0/1040545389 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe2e806ff20 msgr2=0x7fe2e8143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2e77fe640 1 --2- 172.21.15.22:0/1040545389 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe2e806ff20 0x7fe2e8143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2e77fe640 1 -- 172.21.15.22:0/1040545389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2e815afc0 msgr2=0x7fe2e8162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2e77fe640 1 --2- 172.21.15.22:0/1040545389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2e815afc0 0x7fe2e8162f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2e77fe640 1 -- 172.21.15.22:0/1040545389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e80d01b0 con 0x7fe2e80b4520 2024-03-29T04:20:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2e77fe640 1 --2- 172.21.15.22:0/1040545389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e80b4520 0x7fe2e81629d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00d830 tx=0x7fe2dc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2e4ff9640 1 -- 172.21.15.22:0/1040545389 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc018020 con 0x7fe2e80b4520 2024-03-29T04:20:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2e4ff9640 1 -- 172.21.15.22:0/1040545389 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2dc0096c0 con 0x7fe2e80b4520 2024-03-29T04:20:03.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2ee9ef640 1 -- 172.21.15.22:0/1040545389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2e80cfed0 con 0x7fe2e80b4520 2024-03-29T04:20:03.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2e4ff9640 1 -- 172.21.15.22:0/1040545389 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc009ab0 con 0x7fe2e80b4520 2024-03-29T04:20:03.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.715+0000 7fe2ee9ef640 1 -- 172.21.15.22:0/1040545389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2e8160780 con 0x7fe2e80b4520 2024-03-29T04:20:03.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.719+0000 7fe2e4ff9640 1 -- 172.21.15.22:0/1040545389 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe2dc01b020 con 0x7fe2e80b4520 2024-03-29T04:20:03.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.719+0000 7fe2e4ff9640 1 --2- 172.21.15.22:0/1040545389 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe2d4067ff0 0x7fe2d406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:03.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.719+0000 7fe2e4ff9640 1 -- 172.21.15.22:0/1040545389 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fe2dc086dc0 con 0x7fe2e80b4520 2024-03-29T04:20:03.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.719+0000 7fe2e7fff640 1 --2- 172.21.15.22:0/1040545389 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe2d4067ff0 0x7fe2d406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:03.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.719+0000 7fe2f1b71000 1 Processor -- start 2024-03-29T04:20:03.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.719+0000 7fe2f1b71000 1 -- start start 2024-03-29T04:20:03.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.719+0000 7fe2e7fff640 1 --2- 172.21.15.22:0/1040545389 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe2d4067ff0 0x7fe2d406a4b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe2e00ad000 tx=0x7fe2e0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:03.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.719+0000 7fe2f1b71000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577f100da60 0x5577f100de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:03.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.719+0000 7fe2f1b71000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100e380 0x5577f0fc0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:03.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.719+0000 7fe2f1b71000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5577f0fc10f0 0x5577f0fc14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:03.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.719+0000 7fe2f1b71000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5577f0ea5e20 con 0x5577f100e380 2024-03-29T04:20:03.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.719+0000 7fe2f1b71000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5577f0e0eb00 con 0x5577f100da60 2024-03-29T04:20:03.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.719+0000 7fe2f1b71000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x5577f0dab940 con 0x5577f0fc10f0 2024-03-29T04:20:03.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.719+0000 7fe2e77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100e380 0x5577f0fc0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:03.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.719+0000 7fe2e77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100e380 0x5577f0fc0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52914/0 (socket says 172.21.15.22:52914) 2024-03-29T04:20:03.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.719+0000 7fe2e77fe640 1 -- 172.21.15.22:0/2113331529 learned_addr learned my addr 172.21.15.22:0/2113331529 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:03.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.719+0000 7fe2e7fff640 1 --2- 172.21.15.22:0/2113331529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577f100da60 0x5577f100de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:03.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2ecf8c640 1 --2- 172.21.15.22:0/2113331529 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5577f0fc10f0 0x5577f0fc14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:03.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2e77fe640 1 -- 172.21.15.22:0/2113331529 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5577f0fc10f0 msgr2=0x5577f0fc14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:03.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2e77fe640 1 --2- 172.21.15.22:0/2113331529 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5577f0fc10f0 0x5577f0fc14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2e77fe640 1 -- 172.21.15.22:0/2113331529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577f100da60 msgr2=0x5577f100de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:03.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2e77fe640 1 --2- 172.21.15.22:0/2113331529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577f100da60 0x5577f100de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2e77fe640 1 -- 172.21.15.22:0/2113331529 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577f0d94380 con 0x5577f100e380 2024-03-29T04:20:03.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2ecf8c640 1 --2- 172.21.15.22:0/2113331529 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5577f0fc10f0 0x5577f0fc14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:03.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2e77fe640 1 --2- 172.21.15.22:0/2113331529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100e380 0x5577f0fc0bb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc085170 tx=0x7fe2dc0569c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d312cf8fa4efc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:03.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2cdffb640 1 -- 172.21.15.22:0/2113331529 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc083070 con 0x5577f100e380 2024-03-29T04:20:03.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2cdffb640 1 -- 172.21.15.22:0/2113331529 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2dc05b100 con 0x5577f100e380 2024-03-29T04:20:03.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2cdffb640 1 -- 172.21.15.22:0/2113331529 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc06f070 con 0x5577f100e380 2024-03-29T04:20:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2f1b71000 1 -- 172.21.15.22:0/2113331529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100e380 msgr2=0x5577f0fc0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2f1b71000 1 --2- 172.21.15.22:0/2113331529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100e380 0x5577f0fc0bb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc085170 tx=0x7fe2dc0569c0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2f1b71000 1 -- 172.21.15.22:0/2113331529 shutdown_connections 2024-03-29T04:20:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2f1b71000 1 --2- 172.21.15.22:0/2113331529 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5577f0fc10f0 0x5577f0fc14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2f1b71000 1 --2- 172.21.15.22:0/2113331529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100e380 0x5577f0fc0bb0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2f1b71000 1 --2- 172.21.15.22:0/2113331529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577f100da60 0x5577f100de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2f1b71000 1 -- 172.21.15.22:0/2113331529 >> 172.21.15.22:0/2113331529 conn(0x5577f0fea3a0 msgr2=0x5577f0fec790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2f1b71000 1 -- 172.21.15.22:0/2113331529 shutdown_connections 2024-03-29T04:20:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2f1b71000 1 -- 172.21.15.22:0/2113331529 wait complete. 2024-03-29T04:20:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2f1b71000 1 -- start start 2024-03-29T04:20:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.723+0000 7fe2f1b71000 1 Processor -- start 2024-03-29T04:20:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577f0fc10f0 0x5577f1014bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100da60 0x5577f10150f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5577f100e380 0x5577f0fcabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5577f0dcc1e0 con 0x5577f100da60 2024-03-29T04:20:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5577f0e0aab0 con 0x5577f0fc10f0 2024-03-29T04:20:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x5577f0da9940 con 0x5577f100e380 2024-03-29T04:20:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 10 client.0 _ll_register_callbacks cb 0x5577f0faa6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:20:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2e77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100da60 0x5577f10150f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2e77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100da60 0x5577f10150f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52930/0 (socket says 172.21.15.22:52930) 2024-03-29T04:20:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2e77fe640 1 -- 172.21.15.22:0/3453478497 learned_addr learned my addr 172.21.15.22:0/3453478497 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2e7fff640 1 --2- 172.21.15.22:0/3453478497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577f0fc10f0 0x5577f1014bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2ecf8c640 1 --2- 172.21.15.22:0/3453478497 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5577f100e380 0x5577f0fcabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2e77fe640 1 -- 172.21.15.22:0/3453478497 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5577f100e380 msgr2=0x5577f0fcabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2e77fe640 1 --2- 172.21.15.22:0/3453478497 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5577f100e380 0x5577f0fcabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2e77fe640 1 -- 172.21.15.22:0/3453478497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577f0fc10f0 msgr2=0x5577f1014bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2e77fe640 1 --2- 172.21.15.22:0/3453478497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577f0fc10f0 0x5577f1014bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2e77fe640 1 -- 172.21.15.22:0/3453478497 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577f0dcab70 con 0x5577f100da60 2024-03-29T04:20:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2e77fe640 1 --2- 172.21.15.22:0/3453478497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100da60 0x5577f10150f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc085220 tx=0x7fe2dc00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=accbde0c459111e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-29T04:20:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2c6ffd640 1 -- 172.21.15.22:0/3453478497 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc083070 con 0x5577f100da60 2024-03-29T04:20:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2c6ffd640 1 -- 172.21.15.22:0/3453478497 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fe2dc072ce0 con 0x5577f100da60 2024-03-29T04:20:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 1 -- 172.21.15.22:0/3453478497 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2dc05b100 con 0x5577f100da60 2024-03-29T04:20:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2c6ffd640 1 -- 172.21.15.22:0/3453478497 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe2dc07a050 con 0x5577f100da60 2024-03-29T04:20:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 10 client.5820 fetch_fsmap learned FSMap version 20 2024-03-29T04:20:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 1 -- 172.21.15.22:0/3453478497 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x5577f0dabb90 con 0x5577f100da60 2024-03-29T04:20:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2c6ffd640 1 -- 172.21.15.22:0/3453478497 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2dc05b100 con 0x5577f100da60 2024-03-29T04:20:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2c6ffd640 1 -- 172.21.15.22:0/3453478497 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fe2dc07d020 con 0x5577f100da60 2024-03-29T04:20:03.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 10 client.5820 fetch_fsmap finished waiting for FSMap version 20 2024-03-29T04:20:03.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 10 client.5820 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-29T04:20:03.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 20 client.5820 populate_metadata read hostname 'smithi022' 2024-03-29T04:20:03.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 1 --2- 172.21.15.22:0/3453478497 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x5577f0fcb930 0x5577f0fcbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:03.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 4 client.5820 mds_command: new command op to 5529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:20:03.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 1 -- 172.21.15.22:0/3453478497 --> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577f0da9dd0 con 0x5577f0fcb930 2024-03-29T04:20:03.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2e7fff640 1 --2- 172.21.15.22:0/3453478497 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x5577f0fcb930 0x5577f0fcbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:03.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2e7fff640 1 --2- 172.21.15.22:0/3453478497 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x5577f0fcb930 0x5577f0fcbd10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae6a4701cd114cad server_cookie=9e0bf8ef4fdaf164 in_seq=0 out_seq=0 2024-03-29T04:20:03.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2c6ffd640 10 client.5820 ms_handle_connect on v2:172.21.15.22:6834/2966923505 2024-03-29T04:20:03.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2c6ffd640 1 -- 172.21.15.22:0/3453478497 <== mds.0 v2:172.21.15.22:6834/2966923505 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x5577f0da9dd0 con 0x5577f0fcb930 2024-03-29T04:20:03.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2c6ffd640 10 client.5820 handle_command_reply: tid=0 2024-03-29T04:20:03.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 1 client.5820 shutdown 2024-03-29T04:20:03.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 1 -- 172.21.15.22:0/3453478497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100da60 msgr2=0x5577f10150f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:03.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 1 --2- 172.21.15.22:0/3453478497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100da60 0x5577f10150f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc085220 tx=0x7fe2dc00bfd0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 1 -- 172.21.15.22:0/3453478497 shutdown_connections 2024-03-29T04:20:03.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 1 --2- 172.21.15.22:0/3453478497 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x5577f0fcb930 0x5577f0fcbd10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 1 --2- 172.21.15.22:0/3453478497 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5577f100e380 0x5577f0fcabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 1 --2- 172.21.15.22:0/3453478497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100da60 0x5577f10150f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2c6ffd640 0 client.5820 ms_handle_reset on v2:172.21.15.22:6834/2966923505 2024-03-29T04:20:03.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 1 --2- 172.21.15.22:0/3453478497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577f0fc10f0 0x5577f1014bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 1 -- 172.21.15.22:0/3453478497 >> 172.21.15.22:0/3453478497 conn(0x5577f0fea3a0 msgr2=0x5577f0fbbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:03.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 1 -- 172.21.15.22:0/3453478497 shutdown_connections 2024-03-29T04:20:03.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 1 -- 172.21.15.22:0/3453478497 wait complete. 2024-03-29T04:20:03.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.727+0000 7fe2f1b71000 20 client.5820 trim_cache size 0 max 16384 2024-03-29T04:20:03.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.763+0000 7fe2f1b71000 1 Processor -- start 2024-03-29T04:20:03.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.763+0000 7fe2f1b71000 1 -- start start 2024-03-29T04:20:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2f1b71000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5577f0fc10f0 0x5577f0fbb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2f1b71000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100da60 0x5577f0fbbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2f1b71000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577f100e380 0x5577f0fe7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2f1b71000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e8159b40 con 0x5577f100da60 2024-03-29T04:20:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2f1b71000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e8161de0 con 0x5577f100e380 2024-03-29T04:20:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2f1b71000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe2e81594d0 con 0x5577f0fc10f0 2024-03-29T04:20:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2e77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100da60 0x5577f0fbbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2e77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100da60 0x5577f0fbbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52938/0 (socket says 172.21.15.22:52938) 2024-03-29T04:20:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2e77fe640 1 -- 172.21.15.22:0/2951633629 learned_addr learned my addr 172.21.15.22:0/2951633629 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2e7fff640 1 --2- 172.21.15.22:0/2951633629 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5577f0fc10f0 0x5577f0fbb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2ecf8c640 1 --2- 172.21.15.22:0/2951633629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577f100e380 0x5577f0fe7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2e77fe640 1 -- 172.21.15.22:0/2951633629 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5577f0fc10f0 msgr2=0x5577f0fbb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2e77fe640 1 --2- 172.21.15.22:0/2951633629 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5577f0fc10f0 0x5577f0fbb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2e77fe640 1 -- 172.21.15.22:0/2951633629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577f100e380 msgr2=0x5577f0fe7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2e77fe640 1 --2- 172.21.15.22:0/2951633629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577f100e380 0x5577f0fe7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2e77fe640 1 -- 172.21.15.22:0/2951633629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577f100e1b0 con 0x5577f100da60 2024-03-29T04:20:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2e77fe640 1 --2- 172.21.15.22:0/2951633629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100da60 0x5577f0fbbb50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0870e0 tx=0x7fe2dc080000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7494f5f7eceec184 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2cd7fa640 1 -- 172.21.15.22:0/2951633629 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc083070 con 0x5577f100da60 2024-03-29T04:20:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2cd7fa640 1 -- 172.21.15.22:0/2951633629 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2dc05b100 con 0x5577f100da60 2024-03-29T04:20:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2cd7fa640 1 -- 172.21.15.22:0/2951633629 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc06f040 con 0x5577f100da60 2024-03-29T04:20:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2f1b71000 1 -- 172.21.15.22:0/2951633629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100da60 msgr2=0x5577f0fbbb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:03.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2f1b71000 1 --2- 172.21.15.22:0/2951633629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100da60 0x5577f0fbbb50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0870e0 tx=0x7fe2dc080000 comp rx=0 tx=0).stop 2024-03-29T04:20:03.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2f1b71000 1 -- 172.21.15.22:0/2951633629 shutdown_connections 2024-03-29T04:20:03.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2f1b71000 1 --2- 172.21.15.22:0/2951633629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577f100e380 0x5577f0fe7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2f1b71000 1 --2- 172.21.15.22:0/2951633629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100da60 0x5577f0fbbb50 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2f1b71000 1 --2- 172.21.15.22:0/2951633629 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5577f0fc10f0 0x5577f0fbb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2f1b71000 1 -- 172.21.15.22:0/2951633629 >> 172.21.15.22:0/2951633629 conn(0x5577f0fea3a0 msgr2=0x5577f0fbaf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:03.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2f1b71000 1 -- 172.21.15.22:0/2951633629 shutdown_connections 2024-03-29T04:20:03.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2f1b71000 1 -- 172.21.15.22:0/2951633629 wait complete. 2024-03-29T04:20:03.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2f1b71000 1 -- start start 2024-03-29T04:20:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.767+0000 7fe2f1b71000 1 Processor -- start 2024-03-29T04:20:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577f0fc10f0 0x5577f0fbba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100da60 0x5577f10117c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5577f100e380 0x5577f1011d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5577f0dda150 con 0x5577f100da60 2024-03-29T04:20:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5577f0edd750 con 0x5577f0fc10f0 2024-03-29T04:20:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x5577f0dc5750 con 0x5577f100e380 2024-03-29T04:20:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 10 client.0 _ll_register_callbacks cb 0x7fe2dc081070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:20:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2e77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100da60 0x5577f10117c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2e77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100da60 0x5577f10117c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52942/0 (socket says 172.21.15.22:52942) 2024-03-29T04:20:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2e77fe640 1 -- 172.21.15.22:0/2046558190 learned_addr learned my addr 172.21.15.22:0/2046558190 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2ecf8c640 1 --2- 172.21.15.22:0/2046558190 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5577f100e380 0x5577f1011d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2e7fff640 1 --2- 172.21.15.22:0/2046558190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577f0fc10f0 0x5577f0fbba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:03.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2e77fe640 1 -- 172.21.15.22:0/2046558190 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5577f100e380 msgr2=0x5577f1011d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:03.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2e77fe640 1 --2- 172.21.15.22:0/2046558190 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5577f100e380 0x5577f1011d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2e77fe640 1 -- 172.21.15.22:0/2046558190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577f0fc10f0 msgr2=0x5577f0fbba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:03.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2e77fe640 1 --2- 172.21.15.22:0/2046558190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577f0fc10f0 0x5577f0fbba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2e77fe640 1 -- 172.21.15.22:0/2046558190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577f0e8f470 con 0x5577f100da60 2024-03-29T04:20:03.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2e7fff640 1 --2- 172.21.15.22:0/2046558190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577f0fc10f0 0x5577f0fbba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:03.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2e77fe640 1 --2- 172.21.15.22:0/2046558190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100da60 0x5577f10117c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc006cf0 tx=0x7fe2dc000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313d50c11c0cd098 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:03.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-29T04:20:03.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2c6ffd640 1 -- 172.21.15.22:0/2046558190 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc083070 con 0x5577f100da60 2024-03-29T04:20:03.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2c6ffd640 1 -- 172.21.15.22:0/2046558190 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fe2dc07ed30 con 0x5577f100da60 2024-03-29T04:20:03.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 1 -- 172.21.15.22:0/2046558190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2dc05b100 con 0x5577f100da60 2024-03-29T04:20:03.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2c6ffd640 1 -- 172.21.15.22:0/2046558190 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe2dc06f040 con 0x5577f100da60 2024-03-29T04:20:03.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 10 client.5826 fetch_fsmap learned FSMap version 20 2024-03-29T04:20:03.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 1 -- 172.21.15.22:0/2046558190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x5577f0d9d1e0 con 0x5577f100da60 2024-03-29T04:20:03.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2c6ffd640 1 -- 172.21.15.22:0/2046558190 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2dc05b100 con 0x5577f100da60 2024-03-29T04:20:03.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2c6ffd640 1 -- 172.21.15.22:0/2046558190 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fe2dc07d070 con 0x5577f100da60 2024-03-29T04:20:03.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 10 client.5826 fetch_fsmap finished waiting for FSMap version 20 2024-03-29T04:20:03.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 10 client.5826 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-29T04:20:03.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 20 client.5826 populate_metadata read hostname 'smithi022' 2024-03-29T04:20:03.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 1 --2- 172.21.15.22:0/2046558190 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x5577f107f0b0 0x5577f107f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:03.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 4 client.5826 mds_command: new command op to 5529 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T04:20:03.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 1 -- 172.21.15.22:0/2046558190 --> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5577f0e1c040 con 0x5577f107f0b0 2024-03-29T04:20:03.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2e7fff640 1 --2- 172.21.15.22:0/2046558190 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x5577f107f0b0 0x5577f107f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:03.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2c6ffd640 10 client.5826 ms_handle_connect on v2:172.21.15.22:6834/2966923505 2024-03-29T04:20:03.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2e7fff640 1 --2- 172.21.15.22:0/2046558190 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x5577f107f0b0 0x5577f107f4c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88e238cf5478b0c9 server_cookie=908a0f376db42c59 in_seq=0 out_seq=0 2024-03-29T04:20:03.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2c6ffd640 1 -- 172.21.15.22:0/2046558190 <== mds.0 v2:172.21.15.22:6834/2966923505 1 ==== command_reply(tid 0: 0 ) ==== 8+0+376 (crc 0 0 0) 0x5577f0e1c040 con 0x5577f107f0b0 2024-03-29T04:20:03.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2c6ffd640 10 client.5826 handle_command_reply: tid=0 2024-03-29T04:20:03.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 1 client.5826 shutdown 2024-03-29T04:20:03.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 1 -- 172.21.15.22:0/2046558190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100da60 msgr2=0x5577f10117c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:03.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 1 --2- 172.21.15.22:0/2046558190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100da60 0x5577f10117c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc006cf0 tx=0x7fe2dc000bd0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 1 -- 172.21.15.22:0/2046558190 shutdown_connections 2024-03-29T04:20:03.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 1 --2- 172.21.15.22:0/2046558190 >> [v2:172.21.15.22:6834/2966923505,v1:172.21.15.22:6835/2966923505] conn(0x5577f107f0b0 0x5577f107f4c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 1 --2- 172.21.15.22:0/2046558190 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x5577f100e380 0x5577f1011d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 1 --2- 172.21.15.22:0/2046558190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f100da60 0x5577f10117c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2c6ffd640 0 client.5826 ms_handle_reset on v2:172.21.15.22:6834/2966923505 2024-03-29T04:20:03.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 1 --2- 172.21.15.22:0/2046558190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577f0fc10f0 0x5577f0fbba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 1 -- 172.21.15.22:0/2046558190 >> 172.21.15.22:0/2046558190 conn(0x5577f0fea3a0 msgr2=0x5577f0fbaf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:03.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 1 -- 172.21.15.22:0/2046558190 shutdown_connections 2024-03-29T04:20:03.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 1 -- 172.21.15.22:0/2046558190 wait complete. 2024-03-29T04:20:03.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2f1b71000 20 client.5826 trim_cache size 0 max 16384 2024-03-29T04:20:03.760 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-29T04:20:03.760 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-29T04:20:03.760 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {}, 2024-03-29T04:20:03.760 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_stats": { 2024-03-29T04:20:03.760 INFO:teuthology.orchestra.run.smithi022.stdout: "ad732467-bc36-471e-8646-b2f190b4f928": { 2024-03-29T04:20:03.760 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-03-29T04:19:33.307684+0000", 2024-03-29T04:20:03.760 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-29T04:20:03.760 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 23, 2024-03-29T04:20:03.760 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 23, 2024-03-29T04:20:03.760 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-03-29T04:20:03.760 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-03-29T04:20:03.760 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:20:03.761 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-29T04:20:03.761 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-29T04:20:03.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2ee9ef640 1 -- 172.21.15.22:0/1040545389 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe2d4067ff0 msgr2=0x7fe2d406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:03.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2ee9ef640 1 --2- 172.21.15.22:0/1040545389 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe2d4067ff0 0x7fe2d406a4b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe2e00ad000 tx=0x7fe2e0008040 comp rx=0 tx=0).stop 2024-03-29T04:20:03.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2ee9ef640 1 -- 172.21.15.22:0/1040545389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e80b4520 msgr2=0x7fe2e81629d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:03.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2ee9ef640 1 --2- 172.21.15.22:0/1040545389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e80b4520 0x7fe2e81629d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00d830 tx=0x7fe2dc00dcf0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2ee9ef640 1 -- 172.21.15.22:0/1040545389 shutdown_connections 2024-03-29T04:20:03.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2ee9ef640 1 --2- 172.21.15.22:0/1040545389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2e815afc0 0x7fe2e8162f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2ee9ef640 1 --2- 172.21.15.22:0/1040545389 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe2d4067ff0 0x7fe2d406a4b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2ee9ef640 1 --2- 172.21.15.22:0/1040545389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e80b4520 0x7fe2e81629d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2ee9ef640 1 --2- 172.21.15.22:0/1040545389 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe2e806ff20 0x7fe2e8143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:03.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2ee9ef640 1 -- 172.21.15.22:0/1040545389 >> 172.21.15.22:0/1040545389 conn(0x7fe2e80d1070 msgr2=0x7fe2e815b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:03.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2ee9ef640 1 -- 172.21.15.22:0/1040545389 shutdown_connections 2024-03-29T04:20:03.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:03.771+0000 7fe2ee9ef640 1 -- 172.21.15.22:0/1040545389 wait complete. 2024-03-29T04:20:03.770 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-29T04:20:03.770 DEBUG:tasks.cephfs.test_uninlining:extracting inodes 2024-03-29T04:20:03.770 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/Kconfig 2024-03-29T04:20:03.770 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/Kconfig 2024-03-29T04:20:03.778 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627780 linux-5.4/fs/ceph/Kconfig 2024-03-29T04:20:03.778 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/Makefile 2024-03-29T04:20:03.778 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/Makefile 2024-03-29T04:20:03.828 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628281 linux-5.4/fs/ceph/Makefile 2024-03-29T04:20:03.829 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/ceph_frag.c 2024-03-29T04:20:03.829 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/ceph_frag.c 2024-03-29T04:20:03.876 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628287 linux-5.4/fs/ceph/ceph_frag.c 2024-03-29T04:20:03.877 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/io.h 2024-03-29T04:20:03.877 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/io.h 2024-03-29T04:20:03.925 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628294 linux-5.4/fs/ceph/io.h 2024-03-29T04:20:03.925 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/ioctl.h 2024-03-29T04:20:03.925 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/ioctl.h 2024-03-29T04:20:03.973 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628296 linux-5.4/fs/ceph/ioctl.h 2024-03-29T04:20:03.973 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/Kconfig 2024-03-29T04:20:03.973 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/Kconfig 2024-03-29T04:20:04.021 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628308 linux-5.4/fs/ext2/Kconfig 2024-03-29T04:20:04.021 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_user.c 2024-03-29T04:20:04.021 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_user.c 2024-03-29T04:20:04.069 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628326 linux-5.4/fs/ext2/xattr_user.c 2024-03-29T04:20:04.069 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_security.c 2024-03-29T04:20:04.069 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_security.c 2024-03-29T04:20:04.117 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628324 linux-5.4/fs/ext2/xattr_security.c 2024-03-29T04:20:04.117 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/Makefile 2024-03-29T04:20:04.117 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/Makefile 2024-03-29T04:20:04.165 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628309 linux-5.4/fs/ext2/Makefile 2024-03-29T04:20:04.165 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/symlink.c 2024-03-29T04:20:04.165 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/symlink.c 2024-03-29T04:20:04.216 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628321 linux-5.4/fs/ext2/symlink.c 2024-03-29T04:20:04.217 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/acl.h 2024-03-29T04:20:04.217 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/acl.h 2024-03-29T04:20:04.268 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628311 linux-5.4/fs/ext2/acl.h 2024-03-29T04:20:04.268 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_trusted.c 2024-03-29T04:20:04.269 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_trusted.c 2024-03-29T04:20:04.321 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628325 linux-5.4/fs/ext2/xattr_trusted.c 2024-03-29T04:20:04.321 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr.h 2024-03-29T04:20:04.321 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr.h 2024-03-29T04:20:04.373 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628323 linux-5.4/fs/ext2/xattr.h 2024-03-29T04:20:04.374 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/Kconfig 2024-03-29T04:20:04.374 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/Kconfig 2024-03-29T04:20:04.425 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628328 linux-5.4/fs/orangefs/Kconfig 2024-03-29T04:20:04.426 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-29T04:20:04.426 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-29T04:20:04.478 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628341 linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-29T04:20:04.478 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-29T04:20:04.478 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-29T04:20:04.534 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628348 linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-29T04:20:04.534 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/dcache.c 2024-03-29T04:20:04.535 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/dcache.c 2024-03-29T04:20:04.586 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628331 linux-5.4/fs/orangefs/dcache.c 2024-03-29T04:20:04.586 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-29T04:20:04.587 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-29T04:20:04.641 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628339 linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-29T04:20:04.642 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-29T04:20:04.642 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-29T04:20:04.690 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628343 linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-29T04:20:04.690 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-29T04:20:04.690 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-29T04:20:04.743 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628344 linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-29T04:20:04.743 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/Makefile 2024-03-29T04:20:04.743 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/Makefile 2024-03-29T04:20:04.795 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628329 linux-5.4/fs/orangefs/Makefile 2024-03-29T04:20:04.795 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/symlink.c 2024-03-29T04:20:04.795 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/symlink.c 2024-03-29T04:20:04.846 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628352 linux-5.4/fs/orangefs/symlink.c 2024-03-29T04:20:04.846 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/downcall.h 2024-03-29T04:20:04.846 DEBUG:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/downcall.h 2024-03-29T04:20:04.901 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628334 linux-5.4/fs/orangefs/downcall.h 2024-03-29T04:20:04.902 DEBUG:tasks.cephfs.test_uninlining:getting inline data info 2024-03-29T04:20:04.902 DEBUG:tasks.cephfs.test_uninlining:getting inode info #1 of 23:1099511627780 2024-03-29T04:20:04.902 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511627780 2024-03-29T04:20:04.902 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T04:20:04.902 DEBUG:teuthology.orchestra.run.smithi022:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T04:20:04.988 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T04:20:05.051 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T04:20:05.051 INFO:tasks.cephfs.mount:Terminating background process 2024-03-29T04:20:05.052 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:20:05.052 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:20:05.058 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:20:05.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.247+0000 7f09b7196640 1 Processor -- start 2024-03-29T04:20:05.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.247+0000 7f09b7196640 1 -- start start 2024-03-29T04:20:05.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.247+0000 7f09b7196640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f09b015c0a0 0x7f09b006b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:05.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.247+0000 7f09b7196640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f09b006bba0 0x7f09b00cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:05.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.247+0000 7f09b7196640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09b00cdc10 con 0x7f09b00d0040 2024-03-29T04:20:05.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.247+0000 7f09b7196640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f09b00cf670 con 0x7f09b006bba0 2024-03-29T04:20:05.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.247+0000 7f09b7196640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f09b00cf060 con 0x7f09b015c0a0 2024-03-29T04:20:05.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.247+0000 7f09affff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f09b006bba0 0x7f09b00cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:05.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.247+0000 7f09affff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f09b006bba0 0x7f09b00cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38464/0 (socket says 172.21.15.22:38464) 2024-03-29T04:20:05.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.247+0000 7f09affff640 1 -- 172.21.15.22:0/3727327781 learned_addr learned my addr 172.21.15.22:0/3727327781 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:05.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.247+0000 7f09b4f32640 1 --2- 172.21.15.22:0/3727327781 >> v2:172.21.15.151:3300/0 conn(0x7f09b015c0a0 0x7f09b006b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:05.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.247+0000 7f09affff640 1 -- 172.21.15.22:0/3727327781 >> v2:172.21.15.151:3300/0 conn(0x7f09b015c0a0 msgr2=0x7f09b006b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:05.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.247+0000 7f09affff640 1 --2- 172.21.15.22:0/3727327781 >> v2:172.21.15.151:3300/0 conn(0x7f09b015c0a0 0x7f09b006b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.247+0000 7f09affff640 1 -- 172.21.15.22:0/3727327781 >> v1:172.21.15.151:6789/0 conn(0x7f09b00d0040 legacy=0x7f09b0141990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:05.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.247+0000 7f09affff640 1 -- 172.21.15.22:0/3727327781 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b00cd650 con 0x7f09b006bba0 2024-03-29T04:20:05.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.251+0000 7f09affff640 1 --2- 172.21.15.22:0/3727327781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09b006bba0 0x7f09b00cfb00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f09b0068120 tx=0x7f099801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3006038e758fa740 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:05.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.251+0000 7f09af7fe640 1 -- 172.21.15.22:0/3727327781 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09980aa610 con 0x7f09b006bba0 2024-03-29T04:20:05.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.251+0000 7f09af7fe640 1 -- 172.21.15.22:0/3727327781 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09980af070 con 0x7f09b006bba0 2024-03-29T04:20:05.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.251+0000 7f09af7fe640 1 -- 172.21.15.22:0/3727327781 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09980aa9d0 con 0x7f09b006bba0 2024-03-29T04:20:05.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.251+0000 7f09b7196640 1 -- 172.21.15.22:0/3727327781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09b006bba0 msgr2=0x7f09b00cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:05.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.251+0000 7f09b7196640 1 --2- 172.21.15.22:0/3727327781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09b006bba0 0x7f09b00cfb00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f09b0068120 tx=0x7f099801c4b0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.251+0000 7f09b7196640 1 -- 172.21.15.22:0/3727327781 shutdown_connections 2024-03-29T04:20:05.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.251+0000 7f09b7196640 1 --2- 172.21.15.22:0/3727327781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09b006bba0 0x7f09b00cfb00 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.251+0000 7f09b7196640 1 --2- 172.21.15.22:0/3727327781 >> v2:172.21.15.151:3300/0 conn(0x7f09b015c0a0 0x7f09b006b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.251+0000 7f09b7196640 1 -- 172.21.15.22:0/3727327781 >> 172.21.15.22:0/3727327781 conn(0x7f09b00b2980 msgr2=0x7f09b00d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:05.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.251+0000 7f09b7196640 1 -- 172.21.15.22:0/3727327781 shutdown_connections 2024-03-29T04:20:05.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.251+0000 7f09b7196640 1 -- 172.21.15.22:0/3727327781 wait complete. 2024-03-29T04:20:05.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.251+0000 7f09b7196640 1 Processor -- start 2024-03-29T04:20:05.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.251+0000 7f09b7196640 1 -- start start 2024-03-29T04:20:05.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09b7196640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09b006bba0 0x7f09b014d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:05.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09b7196640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09b00d0040 0x7f09b013c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:05.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09b7196640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09b015c0a0 0x7f09b013ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:05.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09b7196640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f09b0142220 con 0x7f09b006bba0 2024-03-29T04:20:05.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09b7196640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f09b01420a0 con 0x7f09b015c0a0 2024-03-29T04:20:05.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09b7196640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f09b00cd0f0 con 0x7f09b00d0040 2024-03-29T04:20:05.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09b4f32640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09b006bba0 0x7f09b014d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:05.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09b4f32640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09b006bba0 0x7f09b014d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38474/0 (socket says 172.21.15.22:38474) 2024-03-29T04:20:05.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09b4f32640 1 -- 172.21.15.22:0/461062960 learned_addr learned my addr 172.21.15.22:0/461062960 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:05.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09b5733640 1 --2- 172.21.15.22:0/461062960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09b015c0a0 0x7f09b013ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:05.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09affff640 1 --2- 172.21.15.22:0/461062960 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09b00d0040 0x7f09b013c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:05.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09b4f32640 1 -- 172.21.15.22:0/461062960 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09b00d0040 msgr2=0x7f09b013c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:05.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09b4f32640 1 --2- 172.21.15.22:0/461062960 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09b00d0040 0x7f09b013c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09b4f32640 1 -- 172.21.15.22:0/461062960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09b015c0a0 msgr2=0x7f09b013ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:05.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09b4f32640 1 --2- 172.21.15.22:0/461062960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09b015c0a0 0x7f09b013ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09b4f32640 1 -- 172.21.15.22:0/461062960 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b0166f00 con 0x7f09b006bba0 2024-03-29T04:20:05.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09b4f32640 1 --2- 172.21.15.22:0/461062960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09b006bba0 0x7f09b014d810 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f09a000d620 tx=0x7f09a000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:05.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09ad7fa640 1 -- 172.21.15.22:0/461062960 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a000b620 con 0x7f09b006bba0 2024-03-29T04:20:05.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09ad7fa640 1 -- 172.21.15.22:0/461062960 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a0019070 con 0x7f09b006bba0 2024-03-29T04:20:05.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09b7196640 1 -- 172.21.15.22:0/461062960 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b014dd50 con 0x7f09b006bba0 2024-03-29T04:20:05.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09ad7fa640 1 -- 172.21.15.22:0/461062960 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a000b9e0 con 0x7f09b006bba0 2024-03-29T04:20:05.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09b7196640 1 -- 172.21.15.22:0/461062960 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b013bf90 con 0x7f09b006bba0 2024-03-29T04:20:05.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09b7196640 1 -- 172.21.15.22:0/461062960 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b0068890 con 0x7f09b006bba0 2024-03-29T04:20:05.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09ad7fa640 1 -- 172.21.15.22:0/461062960 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f09a000bb80 con 0x7f09b006bba0 2024-03-29T04:20:05.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09ad7fa640 1 --2- 172.21.15.22:0/461062960 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0990067ff0 0x7f099006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:05.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.255+0000 7f09ad7fa640 1 -- 172.21.15.22:0/461062960 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f09a00875b0 con 0x7f09b006bba0 2024-03-29T04:20:05.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.259+0000 7f09affff640 1 --2- 172.21.15.22:0/461062960 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0990067ff0 0x7f099006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:05.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.259+0000 7f09affff640 1 --2- 172.21.15.22:0/461062960 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0990067ff0 0x7f099006a4b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f09980ae000 tx=0x7f0998008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:05.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.259+0000 7f09ad7fa640 1 -- 172.21.15.22:0/461062960 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f09a0057c00 con 0x7f09b006bba0 2024-03-29T04:20:05.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.423+0000 7f09b7196640 1 -- 172.21.15.22:0/461062960 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09b006f9c0 con 0x7f09b006bba0 2024-03-29T04:20:05.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.427+0000 7f09ad7fa640 1 -- 172.21.15.22:0/461062960 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16991 (secure 0 0 0) 0x7f09a0057500 con 0x7f09b006bba0 2024-03-29T04:20:05.408 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:20:05.408 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":42,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:19:17.809849+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:18:56.336876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:18:56.950670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:20:05.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.427+0000 7f09b7196640 1 -- 172.21.15.22:0/461062960 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0990067ff0 msgr2=0x7f099006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:05.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.427+0000 7f09b7196640 1 --2- 172.21.15.22:0/461062960 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0990067ff0 0x7f099006a4b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f09980ae000 tx=0x7f0998008040 comp rx=0 tx=0).stop 2024-03-29T04:20:05.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.427+0000 7f09b7196640 1 -- 172.21.15.22:0/461062960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09b006bba0 msgr2=0x7f09b014d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:05.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.427+0000 7f09b7196640 1 --2- 172.21.15.22:0/461062960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09b006bba0 0x7f09b014d810 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f09a000d620 tx=0x7f09a000dae0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.427+0000 7f09b7196640 1 -- 172.21.15.22:0/461062960 shutdown_connections 2024-03-29T04:20:05.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.427+0000 7f09b7196640 1 --2- 172.21.15.22:0/461062960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09b015c0a0 0x7f09b013ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.427+0000 7f09b7196640 1 --2- 172.21.15.22:0/461062960 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f09b00d0040 0x7f09b013c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.427+0000 7f09b7196640 1 --2- 172.21.15.22:0/461062960 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0990067ff0 0x7f099006a4b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.427+0000 7f09b7196640 1 --2- 172.21.15.22:0/461062960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09b006bba0 0x7f09b014d810 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.427+0000 7f09b7196640 1 -- 172.21.15.22:0/461062960 >> 172.21.15.22:0/461062960 conn(0x7f09b00b2980 msgr2=0x7f09b015b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:05.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.427+0000 7f09b7196640 1 -- 172.21.15.22:0/461062960 shutdown_connections 2024-03-29T04:20:05.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.427+0000 7f09b7196640 1 -- 172.21.15.22:0/461062960 wait complete. 2024-03-29T04:20:05.427 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:18:56.336876+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:18:56.950670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:20:05.427 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:20:05.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.571+0000 7fabb3b35640 1 Processor -- start 2024-03-29T04:20:05.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.571+0000 7fabb3b35640 1 -- start start 2024-03-29T04:20:05.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.571+0000 7fabb3b35640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fabac15d6d0 0x7fabac160970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.571+0000 7fabb3b35640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabac15c970 con 0x7fabac159c90 2024-03-29T04:20:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.571+0000 7fabb3b35640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabac0ccbe0 con 0x7fabac070800 2024-03-29T04:20:05.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.571+0000 7fabb3b35640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fabac15d240 con 0x7fabac15d6d0 2024-03-29T04:20:05.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.571+0000 7fabb18d1640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fabac159c90 0x7fabac0700f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:45524/0 (socket says 172.21.15.22:45524) 2024-03-29T04:20:05.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.571+0000 7fabb18d1640 1 -- 172.21.15.22:0/3550321935 learned_addr learned my addr 172.21.15.22:0/3550321935 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:05.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.571+0000 7fabb20d2640 1 --2- 172.21.15.22:0/3550321935 >> v2:172.21.15.22:3300/0 conn(0x7fabac15d6d0 0x7fabac160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:05.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.571+0000 7fabb08cf640 1 -- 172.21.15.22:0/3550321935 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2473135396 0 0) 0x7faba8001120 con 0x7fabac159c90 2024-03-29T04:20:05.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.571+0000 7fabb08cf640 1 -- 172.21.15.22:0/3550321935 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab98003530 con 0x7fabac159c90 2024-03-29T04:20:05.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.571+0000 7fabb20d2640 1 -- 172.21.15.22:0/3550321935 >> v1:172.21.15.151:6789/0 conn(0x7fabac070800 legacy=0x7fabac0b6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:05.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.571+0000 7fabb20d2640 1 -- 172.21.15.22:0/3550321935 >> v1:172.21.15.22:6789/0 conn(0x7fabac159c90 legacy=0x7fabac0700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:05.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.571+0000 7fabb20d2640 1 -- 172.21.15.22:0/3550321935 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabac15cbf0 con 0x7fabac15d6d0 2024-03-29T04:20:05.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.575+0000 7fabb20d2640 1 --2- 172.21.15.22:0/3550321935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabac15d6d0 0x7fabac160970 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fabac15d3c0 tx=0x7faba401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b70c4a8233b7002 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:05.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.575+0000 7fabb08cf640 1 -- 172.21.15.22:0/3550321935 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba40a6710 con 0x7fabac15d6d0 2024-03-29T04:20:05.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.575+0000 7fabb08cf640 1 -- 172.21.15.22:0/3550321935 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba40a68b0 con 0x7fabac15d6d0 2024-03-29T04:20:05.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.575+0000 7fabb08cf640 1 -- 172.21.15.22:0/3550321935 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba40a6c80 con 0x7fabac15d6d0 2024-03-29T04:20:05.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.575+0000 7fabb3b35640 1 -- 172.21.15.22:0/3550321935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabac15d6d0 msgr2=0x7fabac160970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:05.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.575+0000 7fabb3b35640 1 --2- 172.21.15.22:0/3550321935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabac15d6d0 0x7fabac160970 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fabac15d3c0 tx=0x7faba401c950 comp rx=0 tx=0).stop 2024-03-29T04:20:05.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.575+0000 7fabb3b35640 1 -- 172.21.15.22:0/3550321935 shutdown_connections 2024-03-29T04:20:05.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.575+0000 7fabb3b35640 1 --2- 172.21.15.22:0/3550321935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabac15d6d0 0x7fabac160970 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.575+0000 7fabb3b35640 1 -- 172.21.15.22:0/3550321935 >> 172.21.15.22:0/3550321935 conn(0x7fabac0ceae0 msgr2=0x7fabac0ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:05.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.575+0000 7fabb3b35640 1 -- 172.21.15.22:0/3550321935 shutdown_connections 2024-03-29T04:20:05.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.575+0000 7fabb3b35640 1 -- 172.21.15.22:0/3550321935 wait complete. 2024-03-29T04:20:05.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.575+0000 7fabb3b35640 1 Processor -- start 2024-03-29T04:20:05.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.575+0000 7fabb3b35640 1 -- start start 2024-03-29T04:20:05.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.575+0000 7fabb3b35640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabac070800 0x7fabac1663e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:05.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7fabb3b35640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fabac159c90 0x7fabac06e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7fabb3b35640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabac15d6d0 0x7fabac15ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7fabb3b35640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fabac0b7a50 con 0x7fabac15d6d0 2024-03-29T04:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7fabb3b35640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fabac0b78d0 con 0x7fabac070800 2024-03-29T04:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7fabb3b35640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fabac0b7bd0 con 0x7fabac159c90 2024-03-29T04:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7fabb20d2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabac15d6d0 0x7fabac15ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7fabb20d2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabac15d6d0 0x7fabac15ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38482/0 (socket says 172.21.15.22:38482) 2024-03-29T04:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7fabb20d2640 1 -- 172.21.15.22:0/1126365805 learned_addr learned my addr 172.21.15.22:0/1126365805 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7fabb18d1640 1 --2- 172.21.15.22:0/1126365805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabac070800 0x7fabac1663e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7fabb20d2640 1 -- 172.21.15.22:0/1126365805 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fabac159c90 msgr2=0x7fabac06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7fabb10d0640 1 --2- 172.21.15.22:0/1126365805 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fabac159c90 0x7fabac06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7fabb20d2640 1 --2- 172.21.15.22:0/1126365805 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fabac159c90 0x7fabac06e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7fabb20d2640 1 -- 172.21.15.22:0/1126365805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabac070800 msgr2=0x7fabac1663e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7fabb20d2640 1 --2- 172.21.15.22:0/1126365805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabac070800 0x7fabac1663e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7fabb20d2640 1 -- 172.21.15.22:0/1126365805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabac06e540 con 0x7fabac15d6d0 2024-03-29T04:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7fabb20d2640 1 --2- 172.21.15.22:0/1126365805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabac15d6d0 0x7fabac15ea10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7faba4019560 tx=0x7faba40196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:05.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7faba27fc640 1 -- 172.21.15.22:0/1126365805 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba40ae8b0 con 0x7fabac15d6d0 2024-03-29T04:20:05.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7faba27fc640 1 -- 172.21.15.22:0/1126365805 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba40c1070 con 0x7fabac15d6d0 2024-03-29T04:20:05.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7fabb3b35640 1 -- 172.21.15.22:0/1126365805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabac165620 con 0x7fabac15d6d0 2024-03-29T04:20:05.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7faba27fc640 1 -- 172.21.15.22:0/1126365805 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba40aecb0 con 0x7fabac15d6d0 2024-03-29T04:20:05.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7fabb3b35640 1 -- 172.21.15.22:0/1126365805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabac165be0 con 0x7fabac15d6d0 2024-03-29T04:20:05.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7fabb3b35640 1 -- 172.21.15.22:0/1126365805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabac068900 con 0x7fabac15d6d0 2024-03-29T04:20:05.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7faba27fc640 1 -- 172.21.15.22:0/1126365805 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7faba40c6460 con 0x7fabac15d6d0 2024-03-29T04:20:05.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7faba27fc640 1 --2- 172.21.15.22:0/1126365805 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fab90067fc0 0x7fab9006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:05.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7faba27fc640 1 -- 172.21.15.22:0/1126365805 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7faba4121ba0 con 0x7fabac15d6d0 2024-03-29T04:20:05.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.579+0000 7fabb18d1640 1 --2- 172.21.15.22:0/1126365805 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fab90067fc0 0x7fab9006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:05.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.583+0000 7fabb18d1640 1 --2- 172.21.15.22:0/1126365805 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fab90067fc0 0x7fab9006a480 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faba80012c0 tx=0x7faba8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:05.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.583+0000 7faba27fc640 1 -- 172.21.15.22:0/1126365805 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7faba4016540 con 0x7fabac15d6d0 2024-03-29T04:20:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.747+0000 7fabb3b35640 1 -- 172.21.15.22:0/1126365805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fabac1680e0 con 0x7fabac15d6d0 2024-03-29T04:20:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.747+0000 7faba27fc640 1 -- 172.21.15.22:0/1126365805 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) ==== 93+0+10 (secure 0 0 0) 0x7faba40f21a0 con 0x7fabac15d6d0 2024-03-29T04:20:05.728 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:20:05.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.747+0000 7fabb3b35640 1 -- 172.21.15.22:0/1126365805 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fab90067fc0 msgr2=0x7fab9006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:05.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.747+0000 7fabb3b35640 1 --2- 172.21.15.22:0/1126365805 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fab90067fc0 0x7fab9006a480 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faba80012c0 tx=0x7faba8009040 comp rx=0 tx=0).stop 2024-03-29T04:20:05.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.747+0000 7fabb3b35640 1 -- 172.21.15.22:0/1126365805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabac15d6d0 msgr2=0x7fabac15ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:05.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.747+0000 7fabb3b35640 1 --2- 172.21.15.22:0/1126365805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabac15d6d0 0x7fabac15ea10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7faba4019560 tx=0x7faba40196f0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.747+0000 7fabb3b35640 1 -- 172.21.15.22:0/1126365805 shutdown_connections 2024-03-29T04:20:05.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.747+0000 7fabb3b35640 1 --2- 172.21.15.22:0/1126365805 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fab90067fc0 0x7fab9006a480 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.747+0000 7fabb3b35640 1 --2- 172.21.15.22:0/1126365805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabac15d6d0 0x7fabac15ea10 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.747+0000 7fabb3b35640 1 --2- 172.21.15.22:0/1126365805 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fabac159c90 0x7fabac06e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.747+0000 7fabb3b35640 1 --2- 172.21.15.22:0/1126365805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabac070800 0x7fabac1663e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.747+0000 7fabb3b35640 1 -- 172.21.15.22:0/1126365805 >> 172.21.15.22:0/1126365805 conn(0x7fabac0ceae0 msgr2=0x7fabac06f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:05.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.747+0000 7fabb3b35640 1 -- 172.21.15.22:0/1126365805 shutdown_connections 2024-03-29T04:20:05.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.747+0000 7fabb3b35640 1 -- 172.21.15.22:0/1126365805 wait complete. 2024-03-29T04:20:05.747 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:20:05.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.899+0000 7f04d7fbf640 1 Processor -- start 2024-03-29T04:20:05.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.899+0000 7f04d7fbf640 1 -- start start 2024-03-29T04:20:05.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.899+0000 7f04d7fbf640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f04d015afc0 0x7f04d00b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:05.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.899+0000 7f04d7fbf640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f04d00b4520 0x7f04d006f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:05.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.899+0000 7f04d7fbf640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04d00b4ba0 con 0x7f04d006ff20 2024-03-29T04:20:05.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.899+0000 7f04d7fbf640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f04d0070c30 con 0x7f04d00b4520 2024-03-29T04:20:05.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.899+0000 7f04d7fbf640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f04d00b4960 con 0x7f04d015afc0 2024-03-29T04:20:05.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.899+0000 7f04d555a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f04d00b4520 0x7f04d006f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:05.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.899+0000 7f04d555a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f04d00b4520 0x7f04d006f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38494/0 (socket says 172.21.15.22:38494) 2024-03-29T04:20:05.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.899+0000 7f04d555a640 1 -- 172.21.15.22:0/4271562560 learned_addr learned my addr 172.21.15.22:0/4271562560 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:05.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.899+0000 7f04d5d5b640 1 --2- 172.21.15.22:0/4271562560 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f04d015afc0 0x7f04d00b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:05.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.899+0000 7f04d555a640 1 -- 172.21.15.22:0/4271562560 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f04d015afc0 msgr2=0x7f04d00b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:05.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.899+0000 7f04d555a640 1 --2- 172.21.15.22:0/4271562560 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f04d015afc0 0x7f04d00b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.899+0000 7f04d555a640 1 -- 172.21.15.22:0/4271562560 >> v1:172.21.15.151:6789/0 conn(0x7f04d006ff20 legacy=0x7f04d006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:05.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.899+0000 7f04d555a640 1 -- 172.21.15.22:0/4271562560 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d015ec60 con 0x7f04d00b4520 2024-03-29T04:20:05.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.899+0000 7f04d5d5b640 1 --2- 172.21.15.22:0/4271562560 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f04d015afc0 0x7f04d00b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:05.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.903+0000 7f04d555a640 1 --2- 172.21.15.22:0/4271562560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04d00b4520 0x7f04d006f9e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f04c000bb10 tx=0x7f04c001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fa757d8e86f110 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:05.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.903+0000 7f04d4d59640 1 -- 172.21.15.22:0/4271562560 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c00b4070 con 0x7f04d00b4520 2024-03-29T04:20:05.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.903+0000 7f04d4d59640 1 -- 172.21.15.22:0/4271562560 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c00a7a90 con 0x7f04d00b4520 2024-03-29T04:20:05.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.903+0000 7f04d4d59640 1 -- 172.21.15.22:0/4271562560 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c00a7e60 con 0x7f04d00b4520 2024-03-29T04:20:05.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.903+0000 7f04d7fbf640 1 -- 172.21.15.22:0/4271562560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04d00b4520 msgr2=0x7f04d006f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:05.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.903+0000 7f04d7fbf640 1 --2- 172.21.15.22:0/4271562560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04d00b4520 0x7f04d006f9e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f04c000bb10 tx=0x7f04c001ebf0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.903+0000 7f04d7fbf640 1 -- 172.21.15.22:0/4271562560 shutdown_connections 2024-03-29T04:20:05.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.903+0000 7f04d7fbf640 1 --2- 172.21.15.22:0/4271562560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04d00b4520 0x7f04d006f9e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.903+0000 7f04d7fbf640 1 --2- 172.21.15.22:0/4271562560 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f04d015afc0 0x7f04d00b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.903+0000 7f04d7fbf640 1 -- 172.21.15.22:0/4271562560 >> 172.21.15.22:0/4271562560 conn(0x7f04d00d1070 msgr2=0x7f04d00d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:05.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.903+0000 7f04d7fbf640 1 -- 172.21.15.22:0/4271562560 shutdown_connections 2024-03-29T04:20:05.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.903+0000 7f04d7fbf640 1 -- 172.21.15.22:0/4271562560 wait complete. 2024-03-29T04:20:05.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.903+0000 7f04d7fbf640 1 Processor -- start 2024-03-29T04:20:05.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.903+0000 7f04d7fbf640 1 -- start start 2024-03-29T04:20:05.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04d7fbf640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f04d006ff20 0x7f04d01492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:05.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04d7fbf640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04d00b4520 0x7f04d015b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:05.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04d7fbf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04d015afc0 0x7f04d015bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:05.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04d7fbf640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f04d015fa20 con 0x7f04d015afc0 2024-03-29T04:20:05.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04d7fbf640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f04d00cf7c0 con 0x7f04d00b4520 2024-03-29T04:20:05.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04d7fbf640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f04d014ef80 con 0x7f04d006ff20 2024-03-29T04:20:05.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04d655c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04d015afc0 0x7f04d015bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:05.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04d655c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04d015afc0 0x7f04d015bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38504/0 (socket says 172.21.15.22:38504) 2024-03-29T04:20:05.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04d655c640 1 -- 172.21.15.22:0/2487269220 learned_addr learned my addr 172.21.15.22:0/2487269220 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:05.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04d5d5b640 1 --2- 172.21.15.22:0/2487269220 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f04d006ff20 0x7f04d01492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:05.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04d555a640 1 --2- 172.21.15.22:0/2487269220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04d00b4520 0x7f04d015b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:05.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04d655c640 1 -- 172.21.15.22:0/2487269220 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f04d006ff20 msgr2=0x7f04d01492d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:05.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04d655c640 1 --2- 172.21.15.22:0/2487269220 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f04d006ff20 0x7f04d01492d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04d655c640 1 -- 172.21.15.22:0/2487269220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04d00b4520 msgr2=0x7f04d015b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:05.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04d655c640 1 --2- 172.21.15.22:0/2487269220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04d00b4520 0x7f04d015b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:05.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04d655c640 1 -- 172.21.15.22:0/2487269220 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d015c250 con 0x7f04d015afc0 2024-03-29T04:20:05.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04d655c640 1 --2- 172.21.15.22:0/2487269220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04d015afc0 0x7f04d015bc20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f04c800c780 tx=0x7f04c800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:05.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04c67fc640 1 -- 172.21.15.22:0/2487269220 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8004b70 con 0x7f04d015afc0 2024-03-29T04:20:05.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04c67fc640 1 -- 172.21.15.22:0/2487269220 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c8004d10 con 0x7f04d015afc0 2024-03-29T04:20:05.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04d7fbf640 1 -- 172.21.15.22:0/2487269220 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d015c530 con 0x7f04d015afc0 2024-03-29T04:20:05.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04c67fc640 1 -- 172.21.15.22:0/2487269220 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c800aa90 con 0x7f04d015afc0 2024-03-29T04:20:05.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04d7fbf640 1 -- 172.21.15.22:0/2487269220 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d015caf0 con 0x7f04d015afc0 2024-03-29T04:20:05.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04d7fbf640 1 -- 172.21.15.22:0/2487269220 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04d0068900 con 0x7f04d015afc0 2024-03-29T04:20:05.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04c67fc640 1 -- 172.21.15.22:0/2487269220 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f04c800ac30 con 0x7f04d015afc0 2024-03-29T04:20:05.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04c67fc640 1 --2- 172.21.15.22:0/2487269220 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f04b4067fc0 0x7f04b406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:05.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04c67fc640 1 -- 172.21.15.22:0/2487269220 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f04c8086ce0 con 0x7f04d015afc0 2024-03-29T04:20:05.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.907+0000 7f04d5d5b640 1 --2- 172.21.15.22:0/2487269220 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f04b4067fc0 0x7f04b406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:05.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.911+0000 7f04d5d5b640 1 --2- 172.21.15.22:0/2487269220 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f04b4067fc0 0x7f04b406a480 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f04cc006fd0 tx=0x7f04cc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:05.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:05.915+0000 7f04c67fc640 1 -- 172.21.15.22:0/2487269220 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f04c80572c0 con 0x7f04d015afc0 2024-03-29T04:20:06.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.075+0000 7f04d7fbf640 1 -- 172.21.15.22:0/2487269220 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f04d015b4c0 con 0x7f04d015afc0 2024-03-29T04:20:06.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.079+0000 7f04c67fc640 1 -- 172.21.15.22:0/2487269220 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) ==== 104+0+11 (secure 0 0 0) 0x7f04c8056bc0 con 0x7f04d015afc0 2024-03-29T04:20:06.059 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:20:06.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.079+0000 7f04d7fbf640 1 -- 172.21.15.22:0/2487269220 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f04b4067fc0 msgr2=0x7f04b406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:06.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.079+0000 7f04d7fbf640 1 --2- 172.21.15.22:0/2487269220 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f04b4067fc0 0x7f04b406a480 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f04cc006fd0 tx=0x7f04cc008040 comp rx=0 tx=0).stop 2024-03-29T04:20:06.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.079+0000 7f04d7fbf640 1 -- 172.21.15.22:0/2487269220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04d015afc0 msgr2=0x7f04d015bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:06.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.079+0000 7f04d7fbf640 1 --2- 172.21.15.22:0/2487269220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04d015afc0 0x7f04d015bc20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f04c800c780 tx=0x7f04c800cc40 comp rx=0 tx=0).stop 2024-03-29T04:20:06.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.079+0000 7f04d7fbf640 1 -- 172.21.15.22:0/2487269220 shutdown_connections 2024-03-29T04:20:06.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.079+0000 7f04d7fbf640 1 --2- 172.21.15.22:0/2487269220 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f04b4067fc0 0x7f04b406a480 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.079+0000 7f04d7fbf640 1 --2- 172.21.15.22:0/2487269220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04d015afc0 0x7f04d015bc20 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.079+0000 7f04d7fbf640 1 --2- 172.21.15.22:0/2487269220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04d00b4520 0x7f04d015b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.079+0000 7f04d7fbf640 1 --2- 172.21.15.22:0/2487269220 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f04d006ff20 0x7f04d01492d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.079+0000 7f04d7fbf640 1 -- 172.21.15.22:0/2487269220 >> 172.21.15.22:0/2487269220 conn(0x7f04d00d1070 msgr2=0x7f04d006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:06.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.079+0000 7f04d7fbf640 1 -- 172.21.15.22:0/2487269220 shutdown_connections 2024-03-29T04:20:06.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.079+0000 7f04d7fbf640 1 -- 172.21.15.22:0/2487269220 wait complete. 2024-03-29T04:20:06.078 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:20:06.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.227+0000 7f83d823e640 1 Processor -- start 2024-03-29T04:20:06.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.227+0000 7f83d823e640 1 -- start start 2024-03-29T04:20:06.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.227+0000 7f83d823e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f83d006fd20 0x7f83d00cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:06.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.227+0000 7f83d823e640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d006ec80 con 0x7f83d00cbe80 2024-03-29T04:20:06.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.227+0000 7f83d823e640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d006ef60 con 0x7f83d006e780 2024-03-29T04:20:06.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.227+0000 7f83d823e640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f83d006f240 con 0x7f83d006fd20 2024-03-29T04:20:06.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.227+0000 7f83d5fda640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f83d00cbe80 0x7f83d00cc260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:45534/0 (socket says 172.21.15.22:45534) 2024-03-29T04:20:06.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.227+0000 7f83d5fda640 1 -- 172.21.15.22:0/1446004513 learned_addr learned my addr 172.21.15.22:0/1446004513 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:06.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.227+0000 7f83d67db640 1 --2- 172.21.15.22:0/1446004513 >> v2:172.21.15.22:3300/0 conn(0x7f83d006fd20 0x7f83d00cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:06.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.227+0000 7f83d4fd8640 1 -- 172.21.15.22:0/1446004513 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2413955538 0 0) 0x7f83d006ec80 con 0x7f83d00cbe80 2024-03-29T04:20:06.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.227+0000 7f83d4fd8640 1 -- 172.21.15.22:0/1446004513 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83bc003530 con 0x7f83d00cbe80 2024-03-29T04:20:06.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.227+0000 7f83d67db640 1 -- 172.21.15.22:0/1446004513 >> v1:172.21.15.151:6789/0 conn(0x7f83d006e780 legacy=0x7f83d015d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:06.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.227+0000 7f83d67db640 1 -- 172.21.15.22:0/1446004513 >> v1:172.21.15.22:6789/0 conn(0x7f83d00cbe80 legacy=0x7f83d00cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:06.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.227+0000 7f83d67db640 1 -- 172.21.15.22:0/1446004513 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d006f520 con 0x7f83d006fd20 2024-03-29T04:20:06.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.231+0000 7f83d67db640 1 --2- 172.21.15.22:0/1446004513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83d006fd20 0x7f83d00cf480 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009d90 tx=0x7f83c801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162cec1c65cf2e23 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:06.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.231+0000 7f83d4fd8640 1 -- 172.21.15.22:0/1446004513 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c80b2070 con 0x7f83d006fd20 2024-03-29T04:20:06.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.231+0000 7f83d4fd8640 1 -- 172.21.15.22:0/1446004513 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c80a5900 con 0x7f83d006fd20 2024-03-29T04:20:06.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.231+0000 7f83d4fd8640 1 -- 172.21.15.22:0/1446004513 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c80a5cd0 con 0x7f83d006fd20 2024-03-29T04:20:06.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.231+0000 7f83d823e640 1 -- 172.21.15.22:0/1446004513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83d006fd20 msgr2=0x7f83d00cf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:06.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.231+0000 7f83d823e640 1 --2- 172.21.15.22:0/1446004513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83d006fd20 0x7f83d00cf480 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009d90 tx=0x7f83c801c930 comp rx=0 tx=0).stop 2024-03-29T04:20:06.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.231+0000 7f83d823e640 1 -- 172.21.15.22:0/1446004513 shutdown_connections 2024-03-29T04:20:06.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.231+0000 7f83d823e640 1 --2- 172.21.15.22:0/1446004513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83d006fd20 0x7f83d00cf480 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.231+0000 7f83d823e640 1 -- 172.21.15.22:0/1446004513 >> 172.21.15.22:0/1446004513 conn(0x7f83d00cdac0 msgr2=0x7f83d00cded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:06.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.231+0000 7f83d823e640 1 -- 172.21.15.22:0/1446004513 shutdown_connections 2024-03-29T04:20:06.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.231+0000 7f83d823e640 1 -- 172.21.15.22:0/1446004513 wait complete. 2024-03-29T04:20:06.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.231+0000 7f83d823e640 1 Processor -- start 2024-03-29T04:20:06.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.231+0000 7f83d823e640 1 -- start start 2024-03-29T04:20:06.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83d823e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83d006e780 0x7f83d01445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:06.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83d823e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83d006fd20 0x7f83d0143e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:06.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83d823e640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f83d00cbe80 0x7f83d01659d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:06.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83d823e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f83d016a5f0 con 0x7f83d006e780 2024-03-29T04:20:06.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83d823e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f83d016a470 con 0x7f83d006fd20 2024-03-29T04:20:06.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83d823e640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f83d016a770 con 0x7f83d00cbe80 2024-03-29T04:20:06.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83d5fda640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83d006e780 0x7f83d01445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:06.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83d5fda640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83d006e780 0x7f83d01445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38530/0 (socket says 172.21.15.22:38530) 2024-03-29T04:20:06.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83d5fda640 1 -- 172.21.15.22:0/874466474 learned_addr learned my addr 172.21.15.22:0/874466474 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:06.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83d57d9640 1 --2- 172.21.15.22:0/874466474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83d006fd20 0x7f83d0143e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:06.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83d67db640 1 --2- 172.21.15.22:0/874466474 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f83d00cbe80 0x7f83d01659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:06.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83d5fda640 1 -- 172.21.15.22:0/874466474 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f83d00cbe80 msgr2=0x7f83d01659d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:06.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83d5fda640 1 --2- 172.21.15.22:0/874466474 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f83d00cbe80 0x7f83d01659d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83d5fda640 1 -- 172.21.15.22:0/874466474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83d006fd20 msgr2=0x7f83d0143e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:06.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83d5fda640 1 --2- 172.21.15.22:0/874466474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83d006fd20 0x7f83d0143e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83d5fda640 1 -- 172.21.15.22:0/874466474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d00b7070 con 0x7f83d006e780 2024-03-29T04:20:06.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83d5fda640 1 --2- 172.21.15.22:0/874466474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83d006e780 0x7f83d01445b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00c3f0 tx=0x7f83cc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:06.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83c67fc640 1 -- 172.21.15.22:0/874466474 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc017070 con 0x7f83d006e780 2024-03-29T04:20:06.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83c67fc640 1 -- 172.21.15.22:0/874466474 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83cc005cf0 con 0x7f83d006e780 2024-03-29T04:20:06.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83c67fc640 1 -- 172.21.15.22:0/874466474 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0072f0 con 0x7f83d006e780 2024-03-29T04:20:06.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83d823e640 1 -- 172.21.15.22:0/874466474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d00b6d90 con 0x7f83d006e780 2024-03-29T04:20:06.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83d823e640 1 -- 172.21.15.22:0/874466474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d0144350 con 0x7f83d006e780 2024-03-29T04:20:06.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83d823e640 1 -- 172.21.15.22:0/874466474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d0068e60 con 0x7f83d006e780 2024-03-29T04:20:06.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83c67fc640 1 -- 172.21.15.22:0/874466474 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f83cc01a070 con 0x7f83d006e780 2024-03-29T04:20:06.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83c67fc640 1 --2- 172.21.15.22:0/874466474 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f83b4067ff0 0x7f83b406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:06.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.235+0000 7f83c67fc640 1 -- 172.21.15.22:0/874466474 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f83cc085a70 con 0x7f83d006e780 2024-03-29T04:20:06.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.239+0000 7f83d57d9640 1 --2- 172.21.15.22:0/874466474 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f83b4067ff0 0x7f83b406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:06.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.239+0000 7f83d57d9640 1 --2- 172.21.15.22:0/874466474 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f83b4067ff0 0x7f83b406a4b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f83d00b6af0 tx=0x7f83c0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:06.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.243+0000 7f83c67fc640 1 -- 172.21.15.22:0/874466474 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f83cc056100 con 0x7f83d006e780 2024-03-29T04:20:06.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.423+0000 7f83d823e640 1 -- 172.21.15.22:0/874466474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f83d00c6880 con 0x7f83d006e780 2024-03-29T04:20:06.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.427+0000 7f83c67fc640 1 -- 172.21.15.22:0/874466474 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) ==== 100+0+11 (secure 0 0 0) 0x7f83cc055a00 con 0x7f83d006e780 2024-03-29T04:20:06.407 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-29T04:20:06.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.427+0000 7f83d823e640 1 -- 172.21.15.22:0/874466474 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f83b4067ff0 msgr2=0x7f83b406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:06.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.427+0000 7f83d823e640 1 --2- 172.21.15.22:0/874466474 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f83b4067ff0 0x7f83b406a4b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f83d00b6af0 tx=0x7f83c0009040 comp rx=0 tx=0).stop 2024-03-29T04:20:06.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.427+0000 7f83d823e640 1 -- 172.21.15.22:0/874466474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83d006e780 msgr2=0x7f83d01445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:06.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.427+0000 7f83d823e640 1 --2- 172.21.15.22:0/874466474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83d006e780 0x7f83d01445b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00c3f0 tx=0x7f83cc00c8b0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.427+0000 7f83d823e640 1 -- 172.21.15.22:0/874466474 shutdown_connections 2024-03-29T04:20:06.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.427+0000 7f83d823e640 1 --2- 172.21.15.22:0/874466474 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f83d00cbe80 0x7f83d01659d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.427+0000 7f83d823e640 1 --2- 172.21.15.22:0/874466474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83d006fd20 0x7f83d0143e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.427+0000 7f83d823e640 1 --2- 172.21.15.22:0/874466474 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f83b4067ff0 0x7f83b406a4b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.427+0000 7f83d823e640 1 --2- 172.21.15.22:0/874466474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83d006e780 0x7f83d01445b0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.427+0000 7f83d823e640 1 -- 172.21.15.22:0/874466474 >> 172.21.15.22:0/874466474 conn(0x7f83d00cdac0 msgr2=0x7f83d00c7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:06.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.427+0000 7f83d823e640 1 -- 172.21.15.22:0/874466474 shutdown_connections 2024-03-29T04:20:06.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.427+0000 7f83d823e640 1 -- 172.21.15.22:0/874466474 wait complete. 2024-03-29T04:20:06.427 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:20:06.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.567+0000 7f0ff9e7f640 1 Processor -- start 2024-03-29T04:20:06.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.567+0000 7f0ff9e7f640 1 -- start start 2024-03-29T04:20:06.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.567+0000 7f0ff9e7f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0ff415bf50 0x7f0ff415e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:06.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.567+0000 7f0ff9e7f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0ff40b3d90 0x7f0ff406bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:06.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.567+0000 7f0ff9e7f640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ff415c330 con 0x7f0ff40b42d0 2024-03-29T04:20:06.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.567+0000 7f0ff9e7f640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f0ff406b720 con 0x7f0ff40b3d90 2024-03-29T04:20:06.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.567+0000 7f0ff9e7f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff41624e0 con 0x7f0ff415bf50 2024-03-29T04:20:06.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.567+0000 7f0ff3fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0ff40b3d90 0x7f0ff406bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:06.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.567+0000 7f0ff2ffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f0ff40b42d0 0x7f0ff40b3680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:45538/0 (socket says 172.21.15.22:45538) 2024-03-29T04:20:06.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.567+0000 7f0ff2ffd640 1 -- 172.21.15.22:0/3663000307 learned_addr learned my addr 172.21.15.22:0/3663000307 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:06.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.567+0000 7f0ff37fe640 1 --2- 172.21.15.22:0/3663000307 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0ff415bf50 0x7f0ff415e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:06.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.567+0000 7f0ff3fff640 1 -- 172.21.15.22:0/3663000307 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0ff415bf50 msgr2=0x7f0ff415e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:06.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.567+0000 7f0ff3fff640 1 --2- 172.21.15.22:0/3663000307 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0ff415bf50 0x7f0ff415e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.567+0000 7f0ff3fff640 1 -- 172.21.15.22:0/3663000307 >> v1:172.21.15.22:6789/0 conn(0x7f0ff40b42d0 legacy=0x7f0ff40b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:06.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.567+0000 7f0ff3fff640 1 -- 172.21.15.22:0/3663000307 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff41627c0 con 0x7f0ff40b3d90 2024-03-29T04:20:06.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.567+0000 7f0ff37fe640 1 --2- 172.21.15.22:0/3663000307 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0ff415bf50 0x7f0ff415e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:06.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.567+0000 7f0ff3fff640 1 --2- 172.21.15.22:0/3663000307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ff40b3d90 0x7f0ff406bbb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009d90 tx=0x7f0fec01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c54361308052244 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:06.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.567+0000 7f0ff27fc640 1 -- 172.21.15.22:0/3663000307 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0b2070 con 0x7f0ff40b3d90 2024-03-29T04:20:06.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.567+0000 7f0ff27fc640 1 -- 172.21.15.22:0/3663000307 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fec0a5a20 con 0x7f0ff40b3d90 2024-03-29T04:20:06.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.567+0000 7f0ff27fc640 1 -- 172.21.15.22:0/3663000307 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0a5df0 con 0x7f0ff40b3d90 2024-03-29T04:20:06.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff9e7f640 1 -- 172.21.15.22:0/3663000307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ff40b3d90 msgr2=0x7f0ff406bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:06.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff9e7f640 1 --2- 172.21.15.22:0/3663000307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ff40b3d90 0x7f0ff406bbb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009d90 tx=0x7f0fec01c940 comp rx=0 tx=0).stop 2024-03-29T04:20:06.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff9e7f640 1 -- 172.21.15.22:0/3663000307 shutdown_connections 2024-03-29T04:20:06.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff9e7f640 1 --2- 172.21.15.22:0/3663000307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ff40b3d90 0x7f0ff406bbb0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff9e7f640 1 --2- 172.21.15.22:0/3663000307 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0ff415bf50 0x7f0ff415e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff9e7f640 1 -- 172.21.15.22:0/3663000307 >> 172.21.15.22:0/3663000307 conn(0x7f0ff40b2980 msgr2=0x7f0ff40d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:06.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff9e7f640 1 -- 172.21.15.22:0/3663000307 shutdown_connections 2024-03-29T04:20:06.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff9e7f640 1 -- 172.21.15.22:0/3663000307 wait complete. 2024-03-29T04:20:06.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff9e7f640 1 Processor -- start 2024-03-29T04:20:06.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff9e7f640 1 -- start start 2024-03-29T04:20:06.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff9e7f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ff40b3d90 0x7f0ff413e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:06.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff9e7f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0ff40b42d0 0x7f0ff40ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:06.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff9e7f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ff415bf50 0x7f0ff40aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:06.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff9e7f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff406cc90 con 0x7f0ff415bf50 2024-03-29T04:20:06.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff9e7f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff406cb10 con 0x7f0ff40b3d90 2024-03-29T04:20:06.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff9e7f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff406ce10 con 0x7f0ff40b42d0 2024-03-29T04:20:06.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ff415bf50 0x7f0ff40aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:06.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ff415bf50 0x7f0ff40aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38544/0 (socket says 172.21.15.22:38544) 2024-03-29T04:20:06.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff3fff640 1 -- 172.21.15.22:0/1272955547 learned_addr learned my addr 172.21.15.22:0/1272955547 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:06.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff2ffd640 1 --2- 172.21.15.22:0/1272955547 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0ff40b42d0 0x7f0ff40ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:06.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff37fe640 1 --2- 172.21.15.22:0/1272955547 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ff40b3d90 0x7f0ff413e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:06.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff3fff640 1 -- 172.21.15.22:0/1272955547 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0ff40b42d0 msgr2=0x7f0ff40ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:06.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff3fff640 1 --2- 172.21.15.22:0/1272955547 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0ff40b42d0 0x7f0ff40ae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff3fff640 1 -- 172.21.15.22:0/1272955547 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ff40b3d90 msgr2=0x7f0ff413e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:06.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff3fff640 1 --2- 172.21.15.22:0/1272955547 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ff40b3d90 0x7f0ff413e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff3fff640 1 -- 172.21.15.22:0/1272955547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff4140720 con 0x7f0ff415bf50 2024-03-29T04:20:06.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff2ffd640 1 --2- 172.21.15.22:0/1272955547 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0ff40b42d0 0x7f0ff40ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:06.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff3fff640 1 --2- 172.21.15.22:0/1272955547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ff415bf50 0x7f0ff40aee70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0fec01aff0 tx=0x7f0fec0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:06.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0fdbfff640 1 -- 172.21.15.22:0/1272955547 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0b2050 con 0x7f0ff415bf50 2024-03-29T04:20:06.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0fdbfff640 1 -- 172.21.15.22:0/1272955547 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fec0be070 con 0x7f0ff415bf50 2024-03-29T04:20:06.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff9e7f640 1 -- 172.21.15.22:0/1272955547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff4140a00 con 0x7f0ff415bf50 2024-03-29T04:20:06.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0fdbfff640 1 -- 172.21.15.22:0/1272955547 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0adcb0 con 0x7f0ff415bf50 2024-03-29T04:20:06.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff9e7f640 1 -- 172.21.15.22:0/1272955547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff4156250 con 0x7f0ff415bf50 2024-03-29T04:20:06.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.571+0000 7f0ff9e7f640 1 -- 172.21.15.22:0/1272955547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ff4068890 con 0x7f0ff415bf50 2024-03-29T04:20:06.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.575+0000 7f0fdbfff640 1 -- 172.21.15.22:0/1272955547 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f0fec0c3460 con 0x7f0ff415bf50 2024-03-29T04:20:06.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.575+0000 7f0fdbfff640 1 --2- 172.21.15.22:0/1272955547 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0fe0067fc0 0x7f0fe006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:06.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.575+0000 7f0fdbfff640 1 -- 172.21.15.22:0/1272955547 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f0fec121c10 con 0x7f0ff415bf50 2024-03-29T04:20:06.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.575+0000 7f0ff37fe640 1 --2- 172.21.15.22:0/1272955547 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0fe0067fc0 0x7f0fe006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:06.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.575+0000 7f0ff37fe640 1 --2- 172.21.15.22:0/1272955547 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0fe0067fc0 0x7f0fe006a480 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0ff415b9b0 tx=0x7f0fe4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:06.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.583+0000 7f0fdbfff640 1 -- 172.21.15.22:0/1272955547 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f0fec016560 con 0x7f0ff415bf50 2024-03-29T04:20:06.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.751+0000 7f0ff9e7f640 1 -- 172.21.15.22:0/1272955547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0ff40ccb20 con 0x7f0ff415bf50 2024-03-29T04:20:06.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.751+0000 7f0fdbfff640 1 -- 172.21.15.22:0/1272955547 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) ==== 80+0+0 (secure 0 0 0) 0x7f0fec125080 con 0x7f0ff415bf50 2024-03-29T04:20:06.732 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-03-29T04:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.751+0000 7f0ff9e7f640 1 -- 172.21.15.22:0/1272955547 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0fe0067fc0 msgr2=0x7f0fe006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.751+0000 7f0ff9e7f640 1 --2- 172.21.15.22:0/1272955547 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0fe0067fc0 0x7f0fe006a480 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0ff415b9b0 tx=0x7f0fe4008040 comp rx=0 tx=0).stop 2024-03-29T04:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.751+0000 7f0ff9e7f640 1 -- 172.21.15.22:0/1272955547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ff415bf50 msgr2=0x7f0ff40aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.751+0000 7f0ff9e7f640 1 --2- 172.21.15.22:0/1272955547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ff415bf50 0x7f0ff40aee70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0fec01aff0 tx=0x7f0fec0196e0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.755+0000 7f0ff9e7f640 1 -- 172.21.15.22:0/1272955547 shutdown_connections 2024-03-29T04:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.755+0000 7f0ff9e7f640 1 --2- 172.21.15.22:0/1272955547 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0fe0067fc0 0x7f0fe006a480 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.755+0000 7f0ff9e7f640 1 --2- 172.21.15.22:0/1272955547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ff415bf50 0x7f0ff40aee70 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.755+0000 7f0ff9e7f640 1 --2- 172.21.15.22:0/1272955547 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0ff40b42d0 0x7f0ff40ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.755+0000 7f0ff9e7f640 1 --2- 172.21.15.22:0/1272955547 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ff40b3d90 0x7f0ff413e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.755+0000 7f0ff9e7f640 1 -- 172.21.15.22:0/1272955547 >> 172.21.15.22:0/1272955547 conn(0x7f0ff40b2980 msgr2=0x7f0ff415efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.755+0000 7f0ff9e7f640 1 -- 172.21.15.22:0/1272955547 shutdown_connections 2024-03-29T04:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:06.755+0000 7f0ff9e7f640 1 -- 172.21.15.22:0/1272955547 wait complete. 2024-03-29T04:20:06.750 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi022.front.sepia.ceph.com... 2024-03-29T04:20:06.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:20:06.750 DEBUG:teuthology.orchestra.run.smithi022:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T04:20:06.761 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[38693]: fuse finished with error 0 and tester_r 0 2024-03-29T04:20:06.776 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-03-29T04:20:06.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:20:06.776 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T04:20:06.903 INFO:tasks.cephfs.mount:Terminating background process 2024-03-29T04:20:06.903 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:20:06.903 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:20:06.909 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:20:07.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.047+0000 7f3785321640 1 Processor -- start 2024-03-29T04:20:07.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.047+0000 7f3785321640 1 -- start start 2024-03-29T04:20:07.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f3785321640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f37801420a0 0x7f37800cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:07.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f3785321640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37800cdc10 con 0x7f378015c0a0 2024-03-29T04:20:07.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f3785321640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37800cf060 con 0x7f37800cfb00 2024-03-29T04:20:07.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f3785321640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f37800cf670 con 0x7f37801420a0 2024-03-29T04:20:07.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377effd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f378015c0a0 0x7f378006b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:45552/0 (socket says 172.21.15.22:45552) 2024-03-29T04:20:07.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377effd640 1 -- 172.21.15.22:0/3750079035 learned_addr learned my addr 172.21.15.22:0/3750079035 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:07.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377f7fe640 1 --2- 172.21.15.22:0/3750079035 >> v2:172.21.15.151:3300/0 conn(0x7f37801420a0 0x7f37800cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:07.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377dffb640 1 -- 172.21.15.22:0/3750079035 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3192058293 0 0) 0x7f37800cdc10 con 0x7f378015c0a0 2024-03-29T04:20:07.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377dffb640 1 -- 172.21.15.22:0/3750079035 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f376c003530 con 0x7f378015c0a0 2024-03-29T04:20:07.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377dffb640 1 -- 172.21.15.22:0/3750079035 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1829865525 0 0) 0x7f376c003530 con 0x7f378015c0a0 2024-03-29T04:20:07.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377dffb640 1 -- 172.21.15.22:0/3750079035 >> v2:172.21.15.151:3300/0 conn(0x7f37801420a0 msgr2=0x7f37800cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:07.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377dffb640 1 --2- 172.21.15.22:0/3750079035 >> v2:172.21.15.151:3300/0 conn(0x7f37801420a0 0x7f37800cff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377dffb640 1 -- 172.21.15.22:0/3750079035 >> v1:172.21.15.151:6789/0 conn(0x7f37800cfb00 legacy=0x7f3780141990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:07.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377dffb640 1 -- 172.21.15.22:0/3750079035 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37800cd650 con 0x7f378015c0a0 2024-03-29T04:20:07.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377dffb640 1 -- 172.21.15.22:0/3750079035 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f3778001680 con 0x7f378015c0a0 2024-03-29T04:20:07.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377dffb640 1 -- 172.21.15.22:0/3750079035 >> v1:172.21.15.22:6789/0 conn(0x7f378015c0a0 legacy=0x7f378006b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:07.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377dffb640 1 --2- 172.21.15.22:0/3750079035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f376c0039c0 0x7f376c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:07.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377dffb640 1 --2- 172.21.15.22:0/3750079035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f376c004810 0x7f376c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:07.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377dffb640 1 --2- 172.21.15.22:0/3750079035 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f376c0051d0 0x7f376c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:07.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377dffb640 1 -- 172.21.15.22:0/3750079035 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f37800cf670 con 0x7f376c004810 2024-03-29T04:20:07.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377dffb640 1 -- 172.21.15.22:0/3750079035 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f376c00c070 con 0x7f376c0039c0 2024-03-29T04:20:07.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377dffb640 1 -- 172.21.15.22:0/3750079035 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f376c00c270 con 0x7f376c0051d0 2024-03-29T04:20:07.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377e7fc640 1 --2- 172.21.15.22:0/3750079035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f376c004810 0x7f376c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:07.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377effd640 1 --2- 172.21.15.22:0/3750079035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f376c0039c0 0x7f376c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:07.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377f7fe640 1 --2- 172.21.15.22:0/3750079035 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f376c0051d0 0x7f376c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:07.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377e7fc640 1 -- 172.21.15.22:0/3750079035 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f376c0051d0 msgr2=0x7f376c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:07.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377e7fc640 1 --2- 172.21.15.22:0/3750079035 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f376c0051d0 0x7f376c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377e7fc640 1 -- 172.21.15.22:0/3750079035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f376c0039c0 msgr2=0x7f376c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:07.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377e7fc640 1 --2- 172.21.15.22:0/3750079035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f376c0039c0 0x7f376c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377e7fc640 1 -- 172.21.15.22:0/3750079035 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376c003530 con 0x7f376c004810 2024-03-29T04:20:07.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.051+0000 7f377f7fe640 1 -- 172.21.15.22:0/3750079035 reap_dead start 2024-03-29T04:20:07.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f377e7fc640 1 --2- 172.21.15.22:0/3750079035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f376c004810 0x7f376c004c90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3768001750 tx=0x7f376801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c89047d7cef63f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:07.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f377dffb640 1 -- 172.21.15.22:0/3750079035 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37680013a0 con 0x7f376c004810 2024-03-29T04:20:07.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f377dffb640 1 -- 172.21.15.22:0/3750079035 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37680052d0 con 0x7f376c004810 2024-03-29T04:20:07.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f377dffb640 1 -- 172.21.15.22:0/3750079035 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37680054c0 con 0x7f376c004810 2024-03-29T04:20:07.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f3785321640 1 -- 172.21.15.22:0/3750079035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f376c004810 msgr2=0x7f376c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:07.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f3785321640 1 --2- 172.21.15.22:0/3750079035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f376c004810 0x7f376c004c90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3768001750 tx=0x7f376801c3c0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f3785321640 1 -- 172.21.15.22:0/3750079035 shutdown_connections 2024-03-29T04:20:07.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f3785321640 1 --2- 172.21.15.22:0/3750079035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f376c004810 0x7f376c004c90 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f3785321640 1 -- 172.21.15.22:0/3750079035 >> 172.21.15.22:0/3750079035 conn(0x7f37800b2980 msgr2=0x7f37800d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:07.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f3785321640 1 -- 172.21.15.22:0/3750079035 shutdown_connections 2024-03-29T04:20:07.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f3785321640 1 -- 172.21.15.22:0/3750079035 wait complete. 2024-03-29T04:20:07.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f3785321640 1 Processor -- start 2024-03-29T04:20:07.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f3785321640 1 -- start start 2024-03-29T04:20:07.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f3785321640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f376c004810 0x7f3780157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:07.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f3785321640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37801e6630 0x7f3780144080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:07.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f3785321640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37801445c0 0x7f37801e48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:07.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f3785321640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f378015c480 con 0x7f37801e6630 2024-03-29T04:20:07.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f3785321640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f37800cf300 con 0x7f37801445c0 2024-03-29T04:20:07.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f3785321640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f378015c600 con 0x7f376c004810 2024-03-29T04:20:07.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f377e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37801e6630 0x7f3780144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:07.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f377e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37801e6630 0x7f3780144080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38564/0 (socket says 172.21.15.22:38564) 2024-03-29T04:20:07.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f377e7fc640 1 -- 172.21.15.22:0/448345271 learned_addr learned my addr 172.21.15.22:0/448345271 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:07.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f377e7fc640 1 -- 172.21.15.22:0/448345271 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f376c004810 msgr2=0x7f3780157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:07.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f377e7fc640 1 --2- 172.21.15.22:0/448345271 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f376c004810 0x7f3780157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f377e7fc640 1 -- 172.21.15.22:0/448345271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37801445c0 msgr2=0x7f37801e48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:07.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f377e7fc640 1 --2- 172.21.15.22:0/448345271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37801445c0 0x7f37801e48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.055+0000 7f377e7fc640 1 -- 172.21.15.22:0/448345271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3780157e70 con 0x7f37801e6630 2024-03-29T04:20:07.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.059+0000 7f377e7fc640 1 --2- 172.21.15.22:0/448345271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37801e6630 0x7f3780144080 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3768005040 tx=0x7f376c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:07.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.059+0000 7f37777fe640 1 -- 172.21.15.22:0/448345271 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768007a60 con 0x7f37801e6630 2024-03-29T04:20:07.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.059+0000 7f37777fe640 1 -- 172.21.15.22:0/448345271 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3768005480 con 0x7f37801e6630 2024-03-29T04:20:07.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.059+0000 7f3785321640 1 -- 172.21.15.22:0/448345271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37801e5590 con 0x7f37801e6630 2024-03-29T04:20:07.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.059+0000 7f37777fe640 1 -- 172.21.15.22:0/448345271 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37680067e0 con 0x7f37801e6630 2024-03-29T04:20:07.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.059+0000 7f3785321640 1 -- 172.21.15.22:0/448345271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37801e5b50 con 0x7f37801e6630 2024-03-29T04:20:07.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.059+0000 7f3785321640 1 -- 172.21.15.22:0/448345271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3780068890 con 0x7f37801e6630 2024-03-29T04:20:07.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.059+0000 7f37777fe640 1 -- 172.21.15.22:0/448345271 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f37680b4430 con 0x7f37801e6630 2024-03-29T04:20:07.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.059+0000 7f37777fe640 1 --2- 172.21.15.22:0/448345271 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3760067fc0 0x7f376006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:07.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.059+0000 7f37777fe640 1 -- 172.21.15.22:0/448345271 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f3768120640 con 0x7f37801e6630 2024-03-29T04:20:07.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.059+0000 7f377effd640 1 --2- 172.21.15.22:0/448345271 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3760067fc0 0x7f376006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:07.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.059+0000 7f377effd640 1 --2- 172.21.15.22:0/448345271 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3760067fc0 0x7f376006a480 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3778001b90 tx=0x7f37780047f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:07.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.067+0000 7f37777fe640 1 -- 172.21.15.22:0/448345271 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f37680c1030 con 0x7f37801e6630 2024-03-29T04:20:07.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.267+0000 7f3785321640 1 -- 172.21.15.22:0/448345271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37800d0f20 con 0x7f37801e6630 2024-03-29T04:20:07.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.267+0000 7f37777fe640 1 -- 172.21.15.22:0/448345271 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16991 (secure 0 0 0) 0x7f37680b8030 con 0x7f37801e6630 2024-03-29T04:20:07.249 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:20:07.249 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":42,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:19:17.809849+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:18:56.336876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:18:56.950670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:20:07.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.271+0000 7f3785321640 1 -- 172.21.15.22:0/448345271 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3760067fc0 msgr2=0x7f376006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:07.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.271+0000 7f3785321640 1 --2- 172.21.15.22:0/448345271 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3760067fc0 0x7f376006a480 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3778001b90 tx=0x7f37780047f0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.271+0000 7f3785321640 1 -- 172.21.15.22:0/448345271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37801e6630 msgr2=0x7f3780144080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:07.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.271+0000 7f3785321640 1 --2- 172.21.15.22:0/448345271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37801e6630 0x7f3780144080 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3768005040 tx=0x7f376c005840 comp rx=0 tx=0).stop 2024-03-29T04:20:07.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.271+0000 7f3785321640 1 -- 172.21.15.22:0/448345271 shutdown_connections 2024-03-29T04:20:07.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.271+0000 7f3785321640 1 --2- 172.21.15.22:0/448345271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37801445c0 0x7f37801e48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.271+0000 7f3785321640 1 --2- 172.21.15.22:0/448345271 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3760067fc0 0x7f376006a480 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.271+0000 7f3785321640 1 --2- 172.21.15.22:0/448345271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37801e6630 0x7f3780144080 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.271+0000 7f3785321640 1 --2- 172.21.15.22:0/448345271 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f376c004810 0x7f3780157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.271+0000 7f3785321640 1 -- 172.21.15.22:0/448345271 >> 172.21.15.22:0/448345271 conn(0x7f37800b2980 msgr2=0x7f37800b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:07.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.271+0000 7f3785321640 1 -- 172.21.15.22:0/448345271 shutdown_connections 2024-03-29T04:20:07.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.271+0000 7f3785321640 1 -- 172.21.15.22:0/448345271 wait complete. 2024-03-29T04:20:07.267 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:18:56.336876+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:18:56.950670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:20:07.268 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:20:07.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.419+0000 7f85be0c9640 1 Processor -- start 2024-03-29T04:20:07.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.419+0000 7f85be0c9640 1 -- start start 2024-03-29T04:20:07.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85be0c9640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f85b815f880 0x7f85b80b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:07.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85be0c9640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85b80ce4d0 con 0x7f85b8070640 2024-03-29T04:20:07.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85be0c9640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85b80cc7f0 con 0x7f85b806f090 2024-03-29T04:20:07.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85be0c9640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f85b8070ad0 con 0x7f85b815f880 2024-03-29T04:20:07.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b6ffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f85b8070640 0x7f85b806e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:45554/0 (socket says 172.21.15.22:45554) 2024-03-29T04:20:07.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b6ffd640 1 -- 172.21.15.22:0/4293767865 learned_addr learned my addr 172.21.15.22:0/4293767865 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:07.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b77fe640 1 --2- 172.21.15.22:0/4293767865 >> v2:172.21.15.151:3300/0 conn(0x7f85b815f880 0x7f85b80b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:07.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b67fc640 1 -- 172.21.15.22:0/4293767865 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4158196718 0 0) 0x7f85b80ce4d0 con 0x7f85b8070640 2024-03-29T04:20:07.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b67fc640 1 -- 172.21.15.22:0/4293767865 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85a0003530 con 0x7f85b8070640 2024-03-29T04:20:07.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b67fc640 1 -- 172.21.15.22:0/4293767865 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3355785875 0 0) 0x7f85a0003530 con 0x7f85b8070640 2024-03-29T04:20:07.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b67fc640 1 -- 172.21.15.22:0/4293767865 >> v2:172.21.15.151:3300/0 conn(0x7f85b815f880 msgr2=0x7f85b80b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:07.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b67fc640 1 --2- 172.21.15.22:0/4293767865 >> v2:172.21.15.151:3300/0 conn(0x7f85b815f880 0x7f85b80b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b67fc640 1 -- 172.21.15.22:0/4293767865 >> v1:172.21.15.151:6789/0 conn(0x7f85b806f090 legacy=0x7f85b80ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:07.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b67fc640 1 -- 172.21.15.22:0/4293767865 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b80ccad0 con 0x7f85b8070640 2024-03-29T04:20:07.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b67fc640 1 -- 172.21.15.22:0/4293767865 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f85ac0016a0 con 0x7f85b8070640 2024-03-29T04:20:07.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b67fc640 1 -- 172.21.15.22:0/4293767865 >> v1:172.21.15.22:6789/0 conn(0x7f85b8070640 legacy=0x7f85b806e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:07.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b67fc640 1 --2- 172.21.15.22:0/4293767865 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85a00039c0 0x7f85a0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:07.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b67fc640 1 --2- 172.21.15.22:0/4293767865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85a0004810 0x7f85a0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:07.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b67fc640 1 --2- 172.21.15.22:0/4293767865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85a00052f0 0x7f85a000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:07.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b67fc640 1 -- 172.21.15.22:0/4293767865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f85b8070ad0 con 0x7f85a00052f0 2024-03-29T04:20:07.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b67fc640 1 -- 172.21.15.22:0/4293767865 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f85a000c1c0 con 0x7f85a0004810 2024-03-29T04:20:07.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b67fc640 1 -- 172.21.15.22:0/4293767865 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f85a000c3c0 con 0x7f85a00039c0 2024-03-29T04:20:07.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b7fff640 1 --2- 172.21.15.22:0/4293767865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85a00052f0 0x7f85a000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:07.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b6ffd640 1 --2- 172.21.15.22:0/4293767865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85a0004810 0x7f85a0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:07.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b77fe640 1 --2- 172.21.15.22:0/4293767865 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85a00039c0 0x7f85a0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:07.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b7fff640 1 -- 172.21.15.22:0/4293767865 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85a00039c0 msgr2=0x7f85a0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:07.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b7fff640 1 --2- 172.21.15.22:0/4293767865 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85a00039c0 0x7f85a0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b7fff640 1 -- 172.21.15.22:0/4293767865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85a0004810 msgr2=0x7f85a0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:07.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b7fff640 1 --2- 172.21.15.22:0/4293767865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85a0004810 0x7f85a0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b7fff640 1 -- 172.21.15.22:0/4293767865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a0003530 con 0x7f85a00052f0 2024-03-29T04:20:07.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.423+0000 7f85b77fe640 1 --2- 172.21.15.22:0/4293767865 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85a00039c0 0x7f85a0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:07.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.427+0000 7f85b7fff640 1 -- 172.21.15.22:0/4293767865 reap_dead start 2024-03-29T04:20:07.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.427+0000 7f85b7fff640 1 --2- 172.21.15.22:0/4293767865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85a00052f0 0x7f85a000b9d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f85b80705f0 tx=0x7f85b001c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17399231ac7431ee server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:07.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.427+0000 7f85b67fc640 1 -- 172.21.15.22:0/4293767865 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b0001a90 con 0x7f85a00052f0 2024-03-29T04:20:07.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.427+0000 7f85b67fc640 1 -- 172.21.15.22:0/4293767865 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85b0003b80 con 0x7f85a00052f0 2024-03-29T04:20:07.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.427+0000 7f85b67fc640 1 -- 172.21.15.22:0/4293767865 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b00050d0 con 0x7f85a00052f0 2024-03-29T04:20:07.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.427+0000 7f85be0c9640 1 -- 172.21.15.22:0/4293767865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85a00052f0 msgr2=0x7f85a000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:07.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.427+0000 7f85be0c9640 1 --2- 172.21.15.22:0/4293767865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85a00052f0 0x7f85a000b9d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f85b80705f0 tx=0x7f85b001c720 comp rx=0 tx=0).stop 2024-03-29T04:20:07.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.427+0000 7f85be0c9640 1 -- 172.21.15.22:0/4293767865 shutdown_connections 2024-03-29T04:20:07.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.427+0000 7f85be0c9640 1 --2- 172.21.15.22:0/4293767865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85a00052f0 0x7f85a000b9d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.427+0000 7f85be0c9640 1 -- 172.21.15.22:0/4293767865 >> 172.21.15.22:0/4293767865 conn(0x7f85b80b3cf0 msgr2=0x7f85b80b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:07.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.427+0000 7f85be0c9640 1 -- 172.21.15.22:0/4293767865 shutdown_connections 2024-03-29T04:20:07.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.427+0000 7f85be0c9640 1 -- 172.21.15.22:0/4293767865 wait complete. 2024-03-29T04:20:07.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.427+0000 7f85be0c9640 1 Processor -- start 2024-03-29T04:20:07.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.427+0000 7f85be0c9640 1 -- start start 2024-03-29T04:20:07.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f85be0c9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85a00052f0 0x7f85b815e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:07.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f85be0c9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85b815a390 0x7f85b81532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:07.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f85be0c9640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85b8153e60 0x7f85b81537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:07.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f85be0c9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f85b80cf650 con 0x7f85a00052f0 2024-03-29T04:20:07.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f85be0c9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f85b80cf4d0 con 0x7f85b815a390 2024-03-29T04:20:07.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f85be0c9640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f85b8160850 con 0x7f85b8153e60 2024-03-29T04:20:07.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f85b77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85a00052f0 0x7f85b815e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:07.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f85b77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85a00052f0 0x7f85b815e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38590/0 (socket says 172.21.15.22:38590) 2024-03-29T04:20:07.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f85b77fe640 1 -- 172.21.15.22:0/4287968805 learned_addr learned my addr 172.21.15.22:0/4287968805 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:07.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f85b6ffd640 1 --2- 172.21.15.22:0/4287968805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85b815a390 0x7f85b81532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:07.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f85b77fe640 1 -- 172.21.15.22:0/4287968805 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85b8153e60 msgr2=0x7f85b81537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:07.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f85b7fff640 1 --2- 172.21.15.22:0/4287968805 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85b8153e60 0x7f85b81537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:07.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f85b77fe640 1 --2- 172.21.15.22:0/4287968805 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85b8153e60 0x7f85b81537f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f85b77fe640 1 -- 172.21.15.22:0/4287968805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85b815a390 msgr2=0x7f85b81532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:07.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f85b77fe640 1 --2- 172.21.15.22:0/4287968805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85b815a390 0x7f85b81532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f85b77fe640 1 -- 172.21.15.22:0/4287968805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b815d7c0 con 0x7f85a00052f0 2024-03-29T04:20:07.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f85b77fe640 1 --2- 172.21.15.22:0/4287968805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85a00052f0 0x7f85b815e6c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f85a8010900 tx=0x7f85a8010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:07.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f859ffff640 1 -- 172.21.15.22:0/4287968805 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a800f040 con 0x7f85a00052f0 2024-03-29T04:20:07.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f859ffff640 1 -- 172.21.15.22:0/4287968805 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a800ec20 con 0x7f85a00052f0 2024-03-29T04:20:07.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f85be0c9640 1 -- 172.21.15.22:0/4287968805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b806e6b0 con 0x7f85a00052f0 2024-03-29T04:20:07.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f859ffff640 1 -- 172.21.15.22:0/4287968805 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a801c400 con 0x7f85a00052f0 2024-03-29T04:20:07.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f85be0c9640 1 -- 172.21.15.22:0/4287968805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b815da60 con 0x7f85a00052f0 2024-03-29T04:20:07.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f85be0c9640 1 -- 172.21.15.22:0/4287968805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b8068900 con 0x7f85a00052f0 2024-03-29T04:20:07.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f859ffff640 1 -- 172.21.15.22:0/4287968805 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f85a801c5a0 con 0x7f85a00052f0 2024-03-29T04:20:07.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f859ffff640 1 --2- 172.21.15.22:0/4287968805 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f85a4067ff0 0x7f85a406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:07.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f859ffff640 1 -- 172.21.15.22:0/4287968805 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f85a8086e00 con 0x7f85a00052f0 2024-03-29T04:20:07.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.431+0000 7f85b6ffd640 1 --2- 172.21.15.22:0/4287968805 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f85a4067ff0 0x7f85a406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:07.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.435+0000 7f85b6ffd640 1 --2- 172.21.15.22:0/4287968805 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f85a4067ff0 0x7f85a406a4b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f85b80c8c50 tx=0x7f85ac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:07.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.439+0000 7f859ffff640 1 -- 172.21.15.22:0/4287968805 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f85a8018030 con 0x7f85a00052f0 2024-03-29T04:20:07.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.759+0000 7f85be0c9640 1 -- 172.21.15.22:0/4287968805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f85b815dca0 con 0x7f85a00052f0 2024-03-29T04:20:07.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.759+0000 7f859ffff640 1 -- 172.21.15.22:0/4287968805 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) ==== 93+0+10 (secure 0 0 0) 0x7f85a8057400 con 0x7f85a00052f0 2024-03-29T04:20:07.740 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:20:07.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.763+0000 7f85be0c9640 1 -- 172.21.15.22:0/4287968805 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f85a4067ff0 msgr2=0x7f85a406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:07.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.763+0000 7f85be0c9640 1 --2- 172.21.15.22:0/4287968805 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f85a4067ff0 0x7f85a406a4b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f85b80c8c50 tx=0x7f85ac009040 comp rx=0 tx=0).stop 2024-03-29T04:20:07.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.763+0000 7f85be0c9640 1 -- 172.21.15.22:0/4287968805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85a00052f0 msgr2=0x7f85b815e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:07.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.763+0000 7f85be0c9640 1 --2- 172.21.15.22:0/4287968805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85a00052f0 0x7f85b815e6c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f85a8010900 tx=0x7f85a8010dc0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.763+0000 7f85be0c9640 1 -- 172.21.15.22:0/4287968805 shutdown_connections 2024-03-29T04:20:07.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.763+0000 7f85be0c9640 1 --2- 172.21.15.22:0/4287968805 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f85b8153e60 0x7f85b81537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.763+0000 7f85be0c9640 1 --2- 172.21.15.22:0/4287968805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85b815a390 0x7f85b81532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.763+0000 7f85be0c9640 1 --2- 172.21.15.22:0/4287968805 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f85a4067ff0 0x7f85a406a4b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.763+0000 7f85be0c9640 1 --2- 172.21.15.22:0/4287968805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85a00052f0 0x7f85b815e6c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.763+0000 7f85be0c9640 1 -- 172.21.15.22:0/4287968805 >> 172.21.15.22:0/4287968805 conn(0x7f85b80b3cf0 msgr2=0x7f85b81676e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:07.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.763+0000 7f85be0c9640 1 -- 172.21.15.22:0/4287968805 shutdown_connections 2024-03-29T04:20:07.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.763+0000 7f85be0c9640 1 -- 172.21.15.22:0/4287968805 wait complete. 2024-03-29T04:20:07.768 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:20:07.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.903+0000 7ff9bfc39640 1 Processor -- start 2024-03-29T04:20:07.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.903+0000 7ff9bfc39640 1 -- start start 2024-03-29T04:20:07.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9bfc39640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff9b815bd70 0x7ff9b806eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:07.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9bfc39640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9b806dc00 con 0x7ff9b806f610 2024-03-29T04:20:07.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9bfc39640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9b806dee0 con 0x7ff9b80afb90 2024-03-29T04:20:07.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9bfc39640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7ff9b806e1c0 con 0x7ff9b815bd70 2024-03-29T04:20:07.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9be1d6640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff9b806f610 0x7ff9b806fa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:45558/0 (socket says 172.21.15.22:45558) 2024-03-29T04:20:07.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9be1d6640 1 -- 172.21.15.22:0/28485045 learned_addr learned my addr 172.21.15.22:0/28485045 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:07.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9bd9d5640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff9b815bd70 0x7ff9b806eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:07.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9bc9d3640 1 -- 172.21.15.22:0/28485045 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2232298980 0 0) 0x7ff9b806dc00 con 0x7ff9b806f610 2024-03-29T04:20:07.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9bc9d3640 1 -- 172.21.15.22:0/28485045 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9a4003530 con 0x7ff9b806f610 2024-03-29T04:20:07.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9bd9d5640 1 -- 172.21.15.22:0/28485045 >> v1:172.21.15.151:6789/0 conn(0x7ff9b80afb90 legacy=0x7ff9b815c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:07.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9bd9d5640 1 -- 172.21.15.22:0/28485045 >> v1:172.21.15.22:6789/0 conn(0x7ff9b806f610 legacy=0x7ff9b806fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:07.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9bd9d5640 1 -- 172.21.15.22:0/28485045 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b806e4a0 con 0x7ff9b815bd70 2024-03-29T04:20:07.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9bd9d5640 1 --2- 172.21.15.22:0/28485045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9b815bd70 0x7ff9b806eca0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40097f0 tx=0x7ff9b401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a16fac2ccf8aef8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:07.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9bc9d3640 1 -- 172.21.15.22:0/28485045 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b40b0070 con 0x7ff9b815bd70 2024-03-29T04:20:07.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9bc9d3640 1 -- 172.21.15.22:0/28485045 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9b40ab600 con 0x7ff9b815bd70 2024-03-29T04:20:07.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9bc9d3640 1 -- 172.21.15.22:0/28485045 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b40ab9d0 con 0x7ff9b815bd70 2024-03-29T04:20:07.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9bfc39640 1 -- 172.21.15.22:0/28485045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9b815bd70 msgr2=0x7ff9b806eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:07.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9bfc39640 1 --2- 172.21.15.22:0/28485045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9b815bd70 0x7ff9b806eca0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40097f0 tx=0x7ff9b401c460 comp rx=0 tx=0).stop 2024-03-29T04:20:07.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9bfc39640 1 -- 172.21.15.22:0/28485045 shutdown_connections 2024-03-29T04:20:07.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9bfc39640 1 --2- 172.21.15.22:0/28485045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9b815bd70 0x7ff9b806eca0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9bfc39640 1 -- 172.21.15.22:0/28485045 >> 172.21.15.22:0/28485045 conn(0x7ff9b80d0c90 msgr2=0x7ff9b80d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:07.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9bfc39640 1 -- 172.21.15.22:0/28485045 shutdown_connections 2024-03-29T04:20:07.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.907+0000 7ff9bfc39640 1 -- 172.21.15.22:0/28485045 wait complete. 2024-03-29T04:20:07.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9bfc39640 1 Processor -- start 2024-03-29T04:20:07.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9bfc39640 1 -- start start 2024-03-29T04:20:07.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9bfc39640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9b806f610 0x7ff9b8148a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:07.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9bfc39640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9b80afb90 0x7ff9b8153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:07.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9bfc39640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff9b815bd70 0x7ff9b814d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:07.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9bfc39640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b80b43c0 con 0x7ff9b806f610 2024-03-29T04:20:07.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9bfc39640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b80b4240 con 0x7ff9b80afb90 2024-03-29T04:20:07.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9bfc39640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b80b4540 con 0x7ff9b815bd70 2024-03-29T04:20:07.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9bd9d5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9b806f610 0x7ff9b8148a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:07.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9bd9d5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9b806f610 0x7ff9b8148a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38602/0 (socket says 172.21.15.22:38602) 2024-03-29T04:20:07.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9bd9d5640 1 -- 172.21.15.22:0/2204371593 learned_addr learned my addr 172.21.15.22:0/2204371593 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:07.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9bd9d5640 1 -- 172.21.15.22:0/2204371593 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff9b815bd70 msgr2=0x7ff9b814d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:07.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9bd9d5640 1 --2- 172.21.15.22:0/2204371593 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff9b815bd70 0x7ff9b814d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9bd9d5640 1 -- 172.21.15.22:0/2204371593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9b80afb90 msgr2=0x7ff9b8153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:07.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9bd1d4640 1 --2- 172.21.15.22:0/2204371593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9b80afb90 0x7ff9b8153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:07.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9bd9d5640 1 --2- 172.21.15.22:0/2204371593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9b80afb90 0x7ff9b8153810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:07.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9bd9d5640 1 -- 172.21.15.22:0/2204371593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b8153d50 con 0x7ff9b806f610 2024-03-29T04:20:07.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9bd1d4640 1 --2- 172.21.15.22:0/2204371593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9b80afb90 0x7ff9b8153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:07.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9bd9d5640 1 --2- 172.21.15.22:0/2204371593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9b806f610 0x7ff9b8148a50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4019070 tx=0x7ff9b4019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:07.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9ae7fc640 1 -- 172.21.15.22:0/2204371593 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b40b0040 con 0x7ff9b806f610 2024-03-29T04:20:07.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9ae7fc640 1 -- 172.21.15.22:0/2204371593 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9b40bb070 con 0x7ff9b806f610 2024-03-29T04:20:07.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9bfc39640 1 -- 172.21.15.22:0/2204371593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b8147260 con 0x7ff9b806f610 2024-03-29T04:20:07.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9ae7fc640 1 -- 172.21.15.22:0/2204371593 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b40b58b0 con 0x7ff9b806f610 2024-03-29T04:20:07.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff9bfc39640 1 -- 172.21.15.22:0/2204371593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b814cae0 con 0x7ff9b806f610 2024-03-29T04:20:07.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.911+0000 7ff99bfff640 1 -- 172.21.15.22:0/2204371593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b8068900 con 0x7ff9b806f610 2024-03-29T04:20:07.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.915+0000 7ff9ae7fc640 1 -- 172.21.15.22:0/2204371593 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff9b40b3070 con 0x7ff9b806f610 2024-03-29T04:20:07.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.915+0000 7ff9ae7fc640 1 --2- 172.21.15.22:0/2204371593 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff99c067fc0 0x7ff99c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:07.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.915+0000 7ff9ae7fc640 1 -- 172.21.15.22:0/2204371593 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7ff9b4120300 con 0x7ff9b806f610 2024-03-29T04:20:07.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.915+0000 7ff9bd1d4640 1 --2- 172.21.15.22:0/2204371593 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff99c067fc0 0x7ff99c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:07.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.915+0000 7ff9bd1d4640 1 --2- 172.21.15.22:0/2204371593 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff99c067fc0 0x7ff99c06a480 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80af750 tx=0x7ff9a8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:07.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:07.919+0000 7ff9ae7fc640 1 -- 172.21.15.22:0/2204371593 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff9b4016050 con 0x7ff9b806f610 2024-03-29T04:20:08.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.083+0000 7ff99bfff640 1 -- 172.21.15.22:0/2204371593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff9b806eef0 con 0x7ff9b806f610 2024-03-29T04:20:08.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.083+0000 7ff9ae7fc640 1 -- 172.21.15.22:0/2204371593 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) ==== 104+0+11 (secure 0 0 0) 0x7ff9b40f0900 con 0x7ff9b806f610 2024-03-29T04:20:08.065 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:20:08.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.087+0000 7ff99bfff640 1 -- 172.21.15.22:0/2204371593 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff99c067fc0 msgr2=0x7ff99c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:08.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.087+0000 7ff99bfff640 1 --2- 172.21.15.22:0/2204371593 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff99c067fc0 0x7ff99c06a480 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80af750 tx=0x7ff9a8008040 comp rx=0 tx=0).stop 2024-03-29T04:20:08.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.087+0000 7ff99bfff640 1 -- 172.21.15.22:0/2204371593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9b806f610 msgr2=0x7ff9b8148a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:08.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.087+0000 7ff99bfff640 1 --2- 172.21.15.22:0/2204371593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9b806f610 0x7ff9b8148a50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4019070 tx=0x7ff9b4019200 comp rx=0 tx=0).stop 2024-03-29T04:20:08.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.087+0000 7ff99bfff640 1 -- 172.21.15.22:0/2204371593 shutdown_connections 2024-03-29T04:20:08.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.087+0000 7ff99bfff640 1 --2- 172.21.15.22:0/2204371593 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff9b815bd70 0x7ff9b814d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.087+0000 7ff99bfff640 1 --2- 172.21.15.22:0/2204371593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9b80afb90 0x7ff9b8153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.087+0000 7ff99bfff640 1 --2- 172.21.15.22:0/2204371593 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff99c067fc0 0x7ff99c06a480 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.087+0000 7ff99bfff640 1 --2- 172.21.15.22:0/2204371593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9b806f610 0x7ff9b8148a50 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.087+0000 7ff99bfff640 1 -- 172.21.15.22:0/2204371593 >> 172.21.15.22:0/2204371593 conn(0x7ff9b80d0c90 msgr2=0x7ff9b8155470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:08.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.087+0000 7ff99bfff640 1 -- 172.21.15.22:0/2204371593 shutdown_connections 2024-03-29T04:20:08.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.087+0000 7ff99bfff640 1 -- 172.21.15.22:0/2204371593 wait complete. 2024-03-29T04:20:08.083 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:20:08.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.223+0000 7f3d5110b640 1 Processor -- start 2024-03-29T04:20:08.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.223+0000 7f3d5110b640 1 -- start start 2024-03-29T04:20:08.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.227+0000 7f3d5110b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3d4c1450f0 0x7f3d4c15ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:08.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.227+0000 7f3d5110b640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3d4c0d04a0 0x7f3d4c06c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:08.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.227+0000 7f3d5110b640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d4c0d0b50 con 0x7f3d4c160440 2024-03-29T04:20:08.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.227+0000 7f3d5110b640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f3d4c0d0930 con 0x7f3d4c1450f0 2024-03-29T04:20:08.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.227+0000 7f3d5110b640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f3d4c0d0e30 con 0x7f3d4c0d04a0 2024-03-29T04:20:08.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.227+0000 7f3d4ad9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3d4c1450f0 0x7f3d4c15ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:08.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.227+0000 7f3d4ad9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3d4c1450f0 0x7f3d4c15ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38604/0 (socket says 172.21.15.22:38604) 2024-03-29T04:20:08.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.227+0000 7f3d4ad9d640 1 -- 172.21.15.22:0/2441661341 learned_addr learned my addr 172.21.15.22:0/2441661341 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:08.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.227+0000 7f3d4b59e640 1 --2- 172.21.15.22:0/2441661341 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3d4c0d04a0 0x7f3d4c06c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:08.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.227+0000 7f3d4ad9d640 1 -- 172.21.15.22:0/2441661341 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3d4c0d04a0 msgr2=0x7f3d4c06c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:08.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.227+0000 7f3d4ad9d640 1 --2- 172.21.15.22:0/2441661341 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3d4c0d04a0 0x7f3d4c06c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.227+0000 7f3d4ad9d640 1 -- 172.21.15.22:0/2441661341 >> v1:172.21.15.151:6789/0 conn(0x7f3d4c160440 legacy=0x7f3d4c0cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:08.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.227+0000 7f3d4ad9d640 1 -- 172.21.15.22:0/2441661341 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d4c06b700 con 0x7f3d4c1450f0 2024-03-29T04:20:08.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.231+0000 7f3d4ad9d640 1 --2- 172.21.15.22:0/2441661341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d4c1450f0 0x7f3d4c15ff00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009870 tx=0x7f3d4401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15fb874211266d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:08.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.231+0000 7f3d49d9b640 1 -- 172.21.15.22:0/2441661341 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d440b2070 con 0x7f3d4c1450f0 2024-03-29T04:20:08.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.231+0000 7f3d49d9b640 1 -- 172.21.15.22:0/2441661341 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d440ad640 con 0x7f3d4c1450f0 2024-03-29T04:20:08.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.231+0000 7f3d49d9b640 1 -- 172.21.15.22:0/2441661341 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d440ada10 con 0x7f3d4c1450f0 2024-03-29T04:20:08.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.231+0000 7f3d5110b640 1 -- 172.21.15.22:0/2441661341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d4c1450f0 msgr2=0x7f3d4c15ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:08.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.231+0000 7f3d5110b640 1 --2- 172.21.15.22:0/2441661341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d4c1450f0 0x7f3d4c15ff00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009870 tx=0x7f3d4401c4e0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.231+0000 7f3d5110b640 1 -- 172.21.15.22:0/2441661341 shutdown_connections 2024-03-29T04:20:08.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.231+0000 7f3d5110b640 1 --2- 172.21.15.22:0/2441661341 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3d4c0d04a0 0x7f3d4c06c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.231+0000 7f3d5110b640 1 --2- 172.21.15.22:0/2441661341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d4c1450f0 0x7f3d4c15ff00 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.231+0000 7f3d5110b640 1 -- 172.21.15.22:0/2441661341 >> 172.21.15.22:0/2441661341 conn(0x7f3d4c0d1070 msgr2=0x7f3d4c0d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:08.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.231+0000 7f3d5110b640 1 -- 172.21.15.22:0/2441661341 shutdown_connections 2024-03-29T04:20:08.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.231+0000 7f3d5110b640 1 -- 172.21.15.22:0/2441661341 wait complete. 2024-03-29T04:20:08.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.231+0000 7f3d5110b640 1 Processor -- start 2024-03-29T04:20:08.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.231+0000 7f3d5110b640 1 -- start start 2024-03-29T04:20:08.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d5110b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d4c0d04a0 0x7f3d4c13aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:08.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d5110b640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3d4c1450f0 0x7f3d4c143ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:08.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d5110b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d4c160440 0x7f3d4c15c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:08.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d5110b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c06d0b0 con 0x7f3d4c0d04a0 2024-03-29T04:20:08.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d5110b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c06cf30 con 0x7f3d4c160440 2024-03-29T04:20:08.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d5110b640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f3d4c06d230 con 0x7f3d4c1450f0 2024-03-29T04:20:08.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d4ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d4c0d04a0 0x7f3d4c13aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:08.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d4ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d4c0d04a0 0x7f3d4c13aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38618/0 (socket says 172.21.15.22:38618) 2024-03-29T04:20:08.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d4ad9d640 1 -- 172.21.15.22:0/3701896660 learned_addr learned my addr 172.21.15.22:0/3701896660 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:08.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d4b59e640 1 --2- 172.21.15.22:0/3701896660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d4c160440 0x7f3d4c15c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:08.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d4a59c640 1 --2- 172.21.15.22:0/3701896660 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3d4c1450f0 0x7f3d4c143ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:08.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d4ad9d640 1 -- 172.21.15.22:0/3701896660 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3d4c1450f0 msgr2=0x7f3d4c143ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:08.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d4ad9d640 1 --2- 172.21.15.22:0/3701896660 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3d4c1450f0 0x7f3d4c143ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d4ad9d640 1 -- 172.21.15.22:0/3701896660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d4c160440 msgr2=0x7f3d4c15c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:08.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d4ad9d640 1 --2- 172.21.15.22:0/3701896660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d4c160440 0x7f3d4c15c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d4ad9d640 1 -- 172.21.15.22:0/3701896660 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d4c142e20 con 0x7f3d4c0d04a0 2024-03-29T04:20:08.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d4a59c640 1 --2- 172.21.15.22:0/3701896660 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3d4c1450f0 0x7f3d4c143ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:08.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d4ad9d640 1 --2- 172.21.15.22:0/3701896660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d4c0d04a0 0x7f3d4c13aee0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3d440b0040 tx=0x7f3d44019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:08.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d337fe640 1 -- 172.21.15.22:0/3701896660 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d440ba070 con 0x7f3d4c0d04a0 2024-03-29T04:20:08.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d337fe640 1 -- 172.21.15.22:0/3701896660 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d440b6630 con 0x7f3d4c0d04a0 2024-03-29T04:20:08.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d337fe640 1 -- 172.21.15.22:0/3701896660 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d440b2040 con 0x7f3d4c0d04a0 2024-03-29T04:20:08.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d5110b640 1 -- 172.21.15.22:0/3701896660 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d4c143100 con 0x7f3d4c0d04a0 2024-03-29T04:20:08.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d5110b640 1 -- 172.21.15.22:0/3701896660 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d4c1436c0 con 0x7f3d4c0d04a0 2024-03-29T04:20:08.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d5110b640 1 -- 172.21.15.22:0/3701896660 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d4c068900 con 0x7f3d4c0d04a0 2024-03-29T04:20:08.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d337fe640 1 -- 172.21.15.22:0/3701896660 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f3d440bd070 con 0x7f3d4c0d04a0 2024-03-29T04:20:08.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d337fe640 1 --2- 172.21.15.22:0/3701896660 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3d38067ff0 0x7f3d3806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:08.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d337fe640 1 -- 172.21.15.22:0/3701896660 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f3d441206d0 con 0x7f3d4c0d04a0 2024-03-29T04:20:08.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.235+0000 7f3d4a59c640 1 --2- 172.21.15.22:0/3701896660 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3d38067ff0 0x7f3d3806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:08.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.239+0000 7f3d4a59c640 1 --2- 172.21.15.22:0/3701896660 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3d38067ff0 0x7f3d3806a4b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0d0270 tx=0x7f3d40008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:08.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.239+0000 7f3d337fe640 1 -- 172.21.15.22:0/3701896660 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f3d440160d0 con 0x7f3d4c0d04a0 2024-03-29T04:20:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.419+0000 7f3d5110b640 1 -- 172.21.15.22:0/3701896660 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3d4c0b2820 con 0x7f3d4c0d04a0 2024-03-29T04:20:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.419+0000 7f3d337fe640 1 -- 172.21.15.22:0/3701896660 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) ==== 100+0+11 (secure 0 0 0) 0x7f3d440f0d60 con 0x7f3d4c0d04a0 2024-03-29T04:20:08.402 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-29T04:20:08.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.423+0000 7f3d5110b640 1 -- 172.21.15.22:0/3701896660 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3d38067ff0 msgr2=0x7f3d3806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:08.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.423+0000 7f3d5110b640 1 --2- 172.21.15.22:0/3701896660 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3d38067ff0 0x7f3d3806a4b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0d0270 tx=0x7f3d40008040 comp rx=0 tx=0).stop 2024-03-29T04:20:08.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.423+0000 7f3d5110b640 1 -- 172.21.15.22:0/3701896660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d4c0d04a0 msgr2=0x7f3d4c13aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:08.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.423+0000 7f3d5110b640 1 --2- 172.21.15.22:0/3701896660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d4c0d04a0 0x7f3d4c13aee0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3d440b0040 tx=0x7f3d44019280 comp rx=0 tx=0).stop 2024-03-29T04:20:08.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.423+0000 7f3d5110b640 1 -- 172.21.15.22:0/3701896660 shutdown_connections 2024-03-29T04:20:08.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.423+0000 7f3d5110b640 1 --2- 172.21.15.22:0/3701896660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d4c160440 0x7f3d4c15c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.423+0000 7f3d5110b640 1 --2- 172.21.15.22:0/3701896660 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3d4c1450f0 0x7f3d4c143ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.423+0000 7f3d5110b640 1 --2- 172.21.15.22:0/3701896660 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3d38067ff0 0x7f3d3806a4b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.423+0000 7f3d5110b640 1 --2- 172.21.15.22:0/3701896660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d4c0d04a0 0x7f3d4c13aee0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.423+0000 7f3d5110b640 1 -- 172.21.15.22:0/3701896660 >> 172.21.15.22:0/3701896660 conn(0x7f3d4c0d1070 msgr2=0x7f3d4c0730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:08.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.423+0000 7f3d5110b640 1 -- 172.21.15.22:0/3701896660 shutdown_connections 2024-03-29T04:20:08.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.423+0000 7f3d5110b640 1 -- 172.21.15.22:0/3701896660 wait complete. 2024-03-29T04:20:08.422 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:20:08.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.559+0000 7f03d7e08640 1 Processor -- start 2024-03-29T04:20:08.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.559+0000 7f03d7e08640 1 -- start start 2024-03-29T04:20:08.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.559+0000 7f03d7e08640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f03d015a840 0x7f03d015e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:08.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.559+0000 7f03d7e08640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f03d015e940 0x7f03d006da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:08.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.559+0000 7f03d7e08640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03d0072cb0 con 0x7f03d006df90 2024-03-29T04:20:08.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.559+0000 7f03d7e08640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f03d015eda0 con 0x7f03d015a840 2024-03-29T04:20:08.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.559+0000 7f03d7e08640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f03d0072f90 con 0x7f03d015e940 2024-03-29T04:20:08.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.559+0000 7f03d5ba4640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f03d015a840 0x7f03d015e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:08.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.559+0000 7f03d5ba4640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f03d015a840 0x7f03d015e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38624/0 (socket says 172.21.15.22:38624) 2024-03-29T04:20:08.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.559+0000 7f03d5ba4640 1 -- 172.21.15.22:0/3272054597 learned_addr learned my addr 172.21.15.22:0/3272054597 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:08.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.559+0000 7f03d53a3640 1 --2- 172.21.15.22:0/3272054597 >> v2:172.21.15.151:3300/0 conn(0x7f03d015e940 0x7f03d006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:08.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.559+0000 7f03d5ba4640 1 -- 172.21.15.22:0/3272054597 >> v2:172.21.15.151:3300/0 conn(0x7f03d015e940 msgr2=0x7f03d006da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:08.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.559+0000 7f03d5ba4640 1 --2- 172.21.15.22:0/3272054597 >> v2:172.21.15.151:3300/0 conn(0x7f03d015e940 0x7f03d006da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.559+0000 7f03d5ba4640 1 -- 172.21.15.22:0/3272054597 >> v1:172.21.15.151:6789/0 conn(0x7f03d006df90 legacy=0x7f03d006e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:08.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.559+0000 7f03d5ba4640 1 -- 172.21.15.22:0/3272054597 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d0073270 con 0x7f03d015a840 2024-03-29T04:20:08.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.559+0000 7f03d53a3640 1 --2- 172.21.15.22:0/3272054597 >> v2:172.21.15.151:3300/0 conn(0x7f03d015e940 0x7f03d006da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:08.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.563+0000 7f03d5ba4640 1 --2- 172.21.15.22:0/3272054597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d015a840 0x7f03d015e400 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f03d00cfac0 tx=0x7f03cc01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f873ab7982328e9b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:08.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.563+0000 7f03d4ba2640 1 -- 172.21.15.22:0/3272054597 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc0ab640 con 0x7f03d015a840 2024-03-29T04:20:08.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.563+0000 7f03d4ba2640 1 -- 172.21.15.22:0/3272054597 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03cc0b0070 con 0x7f03d015a840 2024-03-29T04:20:08.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.563+0000 7f03d4ba2640 1 -- 172.21.15.22:0/3272054597 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc0aba00 con 0x7f03d015a840 2024-03-29T04:20:08.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.563+0000 7f03d7e08640 1 -- 172.21.15.22:0/3272054597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d015a840 msgr2=0x7f03d015e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:08.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.563+0000 7f03d7e08640 1 --2- 172.21.15.22:0/3272054597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d015a840 0x7f03d015e400 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f03d00cfac0 tx=0x7f03cc01c420 comp rx=0 tx=0).stop 2024-03-29T04:20:08.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.563+0000 7f03d7e08640 1 -- 172.21.15.22:0/3272054597 shutdown_connections 2024-03-29T04:20:08.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.563+0000 7f03d7e08640 1 --2- 172.21.15.22:0/3272054597 >> v2:172.21.15.151:3300/0 conn(0x7f03d015e940 0x7f03d006da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.563+0000 7f03d7e08640 1 --2- 172.21.15.22:0/3272054597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d015a840 0x7f03d015e400 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.563+0000 7f03d7e08640 1 -- 172.21.15.22:0/3272054597 >> 172.21.15.22:0/3272054597 conn(0x7f03d00cfee0 msgr2=0x7f03d00d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:08.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.563+0000 7f03d7e08640 1 -- 172.21.15.22:0/3272054597 shutdown_connections 2024-03-29T04:20:08.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.563+0000 7f03d7e08640 1 -- 172.21.15.22:0/3272054597 wait complete. 2024-03-29T04:20:08.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d7e08640 1 Processor -- start 2024-03-29T04:20:08.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d7e08640 1 -- start start 2024-03-29T04:20:08.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d7e08640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d006df90 0x7f03d0142460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:08.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d7e08640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03d015a840 0x7f03d0146e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:08.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d7e08640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03d015e940 0x7f03d014d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:08.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d7e08640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f03d0072a50 con 0x7f03d006df90 2024-03-29T04:20:08.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d7e08640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f03d00728d0 con 0x7f03d015a840 2024-03-29T04:20:08.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d7e08640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f03d00b4310 con 0x7f03d015e940 2024-03-29T04:20:08.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d5ba4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d006df90 0x7f03d0142460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:08.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d5ba4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d006df90 0x7f03d0142460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38634/0 (socket says 172.21.15.22:38634) 2024-03-29T04:20:08.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d5ba4640 1 -- 172.21.15.22:0/3666134628 learned_addr learned my addr 172.21.15.22:0/3666134628 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:08.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d53a3640 1 --2- 172.21.15.22:0/3666134628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03d015a840 0x7f03d0146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:08.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d5ba4640 1 -- 172.21.15.22:0/3666134628 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03d015e940 msgr2=0x7f03d014d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:08.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d63a5640 1 --2- 172.21.15.22:0/3666134628 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03d015e940 0x7f03d014d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:08.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d5ba4640 1 --2- 172.21.15.22:0/3666134628 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03d015e940 0x7f03d014d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d5ba4640 1 -- 172.21.15.22:0/3666134628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03d015a840 msgr2=0x7f03d0146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:08.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d5ba4640 1 --2- 172.21.15.22:0/3666134628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03d015a840 0x7f03d0146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d5ba4640 1 -- 172.21.15.22:0/3666134628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d00af990 con 0x7f03d006df90 2024-03-29T04:20:08.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d63a5640 1 --2- 172.21.15.22:0/3666134628 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03d015e940 0x7f03d014d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:08.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d53a3640 1 --2- 172.21.15.22:0/3666134628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03d015a840 0x7f03d0146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:08.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d5ba4640 1 --2- 172.21.15.22:0/3666134628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d006df90 0x7f03d0142460 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0ad040 tx=0x7f03cc01ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:08.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03c67fc640 1 -- 172.21.15.22:0/3666134628 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc0be5f0 con 0x7f03d006df90 2024-03-29T04:20:08.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03c67fc640 1 -- 172.21.15.22:0/3666134628 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03cc0be7b0 con 0x7f03d006df90 2024-03-29T04:20:08.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d7e08640 1 -- 172.21.15.22:0/3666134628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d00afc70 con 0x7f03d006df90 2024-03-29T04:20:08.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03c67fc640 1 -- 172.21.15.22:0/3666134628 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc0beba0 con 0x7f03d006df90 2024-03-29T04:20:08.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d7e08640 1 -- 172.21.15.22:0/3666134628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d00b12c0 con 0x7f03d006df90 2024-03-29T04:20:08.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.567+0000 7f03d7e08640 1 -- 172.21.15.22:0/3666134628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d0068890 con 0x7f03d006df90 2024-03-29T04:20:08.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.571+0000 7f03c67fc640 1 -- 172.21.15.22:0/3666134628 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f03cc0b43f0 con 0x7f03d006df90 2024-03-29T04:20:08.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.571+0000 7f03c67fc640 1 --2- 172.21.15.22:0/3666134628 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f03b4067ff0 0x7f03b406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:08.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.571+0000 7f03c67fc640 1 -- 172.21.15.22:0/3666134628 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f03cc121250 con 0x7f03d006df90 2024-03-29T04:20:08.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.571+0000 7f03d53a3640 1 --2- 172.21.15.22:0/3666134628 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f03b4067ff0 0x7f03b406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:08.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.571+0000 7f03d53a3640 1 --2- 172.21.15.22:0/3666134628 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f03b4067ff0 0x7f03b406a4b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f03d014e9a0 tx=0x7f03c0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:08.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.575+0000 7f03c67fc640 1 -- 172.21.15.22:0/3666134628 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f03cc0160d0 con 0x7f03d006df90 2024-03-29T04:20:08.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.747+0000 7f03d7e08640 1 -- 172.21.15.22:0/3666134628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f03d00afe40 con 0x7f03d006df90 2024-03-29T04:20:08.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.751+0000 7f03c67fc640 1 -- 172.21.15.22:0/3666134628 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) ==== 80+0+0 (secure 0 0 0) 0x7f03cc0f18e0 con 0x7f03d006df90 2024-03-29T04:20:08.731 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-03-29T04:20:08.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.751+0000 7f03d7e08640 1 -- 172.21.15.22:0/3666134628 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f03b4067ff0 msgr2=0x7f03b406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:08.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.751+0000 7f03d7e08640 1 --2- 172.21.15.22:0/3666134628 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f03b4067ff0 0x7f03b406a4b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f03d014e9a0 tx=0x7f03c0008040 comp rx=0 tx=0).stop 2024-03-29T04:20:08.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.751+0000 7f03d7e08640 1 -- 172.21.15.22:0/3666134628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d006df90 msgr2=0x7f03d0142460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:08.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.751+0000 7f03d7e08640 1 --2- 172.21.15.22:0/3666134628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d006df90 0x7f03d0142460 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0ad040 tx=0x7f03cc01ac60 comp rx=0 tx=0).stop 2024-03-29T04:20:08.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.751+0000 7f03d7e08640 1 -- 172.21.15.22:0/3666134628 shutdown_connections 2024-03-29T04:20:08.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.751+0000 7f03d7e08640 1 --2- 172.21.15.22:0/3666134628 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f03d015e940 0x7f03d014d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.751+0000 7f03d7e08640 1 --2- 172.21.15.22:0/3666134628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03d015a840 0x7f03d0146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.751+0000 7f03d7e08640 1 --2- 172.21.15.22:0/3666134628 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f03b4067ff0 0x7f03b406a4b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.751+0000 7f03d7e08640 1 --2- 172.21.15.22:0/3666134628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d006df90 0x7f03d0142460 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:08.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.751+0000 7f03d7e08640 1 -- 172.21.15.22:0/3666134628 >> 172.21.15.22:0/3666134628 conn(0x7f03d00cfee0 msgr2=0x7f03d0158740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:08.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.751+0000 7f03d7e08640 1 -- 172.21.15.22:0/3666134628 shutdown_connections 2024-03-29T04:20:08.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:08.751+0000 7f03d7e08640 1 -- 172.21.15.22:0/3666134628 wait complete. 2024-03-29T04:20:08.749 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi151.front.sepia.ceph.com... 2024-03-29T04:20:08.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:20:08.750 DEBUG:teuthology.orchestra.run.smithi151:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-03-29T04:20:08.763 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi151.stderr:ceph-fuse[34129]: fuse finished with error 0 and tester_r 0 2024-03-29T04:20:08.764 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-03-29T04:20:08.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:20:08.764 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-29T04:20:08.933 INFO:tasks.cephfs.mount:Terminating background process 2024-03-29T04:20:08.933 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:20:08.933 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:20:08.939 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339616k,nr_inodes=4084904,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275316k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16090 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275312k,nr_inodes=818828,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275316k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T04:20:08.939 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-03-29T04:20:08.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:20:08.940 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-29T04:20:08.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:20:08.992 INFO:teuthology.orchestra.run.smithi022.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-29T04:20:08.992 INFO:tasks.cephfs.mount:Terminating background process 2024-03-29T04:20:08.992 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:20:08.992 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:20:09.002 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336516k,nr_inodes=4084129,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T04:20:09.002 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-03-29T04:20:09.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:20:09.002 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-29T04:20:09.053 INFO:teuthology.orchestra.run.smithi151.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-29T04:20:09.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:20:09.054 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:20:09.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.203+0000 7f0a0a6a7640 1 Processor -- start 2024-03-29T04:20:09.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.203+0000 7f0a0a6a7640 1 -- start start 2024-03-29T04:20:09.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.203+0000 7f0a0a6a7640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0a04070490 0x7f0a0415c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:09.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.203+0000 7f0a0a6a7640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f0a0406de90 0x7f0a0406e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:09.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.203+0000 7f0a0a6a7640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a0415c670 con 0x7f0a040b5b20 2024-03-29T04:20:09.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.203+0000 7f0a0a6a7640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f0a0415edd0 con 0x7f0a0406de90 2024-03-29T04:20:09.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.203+0000 7f0a0a6a7640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0a0415f0b0 con 0x7f0a04070490 2024-03-29T04:20:09.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.203+0000 7f0a08c44640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f0a0406de90 0x7f0a0406e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:09.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.203+0000 7f0a03fff640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0a04070490 0x7f0a0415c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:09.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.203+0000 7f0a08c44640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f0a0406de90 0x7f0a0406e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.22:40970/0 (socket says 172.21.15.22:40970) 2024-03-29T04:20:09.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.203+0000 7f0a037fe640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7f0a040b5b20 0x7f0a0406d780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.22:50024/0 (socket says 172.21.15.22:50024) 2024-03-29T04:20:09.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.203+0000 7f0a08c44640 1 -- 172.21.15.22:0/3910137025 learned_addr learned my addr 172.21.15.22:0/3910137025 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:09.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.203+0000 7f0a08c44640 1 -- 172.21.15.22:0/3910137025 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0a04070490 msgr2=0x7f0a0415c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:09.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.203+0000 7f0a08c44640 1 --2- 172.21.15.22:0/3910137025 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0a04070490 0x7f0a0415c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.203+0000 7f0a08c44640 1 -- 172.21.15.22:0/3910137025 >> v1:172.21.15.151:6789/0 conn(0x7f0a040b5b20 legacy=0x7f0a0406d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:09.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.203+0000 7f0a08c44640 1 -- 172.21.15.22:0/3910137025 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0415f390 con 0x7f0a0406de90 2024-03-29T04:20:09.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a08c44640 1 --2- 172.21.15.22:0/3910137025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a0406de90 0x7f0a0406e810 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009870 tx=0x7f09f401c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88915c305c338d80 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:09.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a02ffd640 1 -- 172.21.15.22:0/3910137025 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f40b0070 con 0x7f0a0406de90 2024-03-29T04:20:09.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a02ffd640 1 -- 172.21.15.22:0/3910137025 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09f4004230 con 0x7f0a0406de90 2024-03-29T04:20:09.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a02ffd640 1 -- 172.21.15.22:0/3910137025 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4003620 con 0x7f0a0406de90 2024-03-29T04:20:09.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a0a6a7640 1 -- 172.21.15.22:0/3910137025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a0406de90 msgr2=0x7f0a0406e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:09.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a0a6a7640 1 --2- 172.21.15.22:0/3910137025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a0406de90 0x7f0a0406e810 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009870 tx=0x7f09f401c440 comp rx=0 tx=0).stop 2024-03-29T04:20:09.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a0a6a7640 1 -- 172.21.15.22:0/3910137025 shutdown_connections 2024-03-29T04:20:09.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a0a6a7640 1 --2- 172.21.15.22:0/3910137025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a0406de90 0x7f0a0406e810 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a0a6a7640 1 --2- 172.21.15.22:0/3910137025 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0a04070490 0x7f0a0415c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a0a6a7640 1 -- 172.21.15.22:0/3910137025 >> 172.21.15.22:0/3910137025 conn(0x7f0a04072500 msgr2=0x7f0a04072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:09.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a0a6a7640 1 -- 172.21.15.22:0/3910137025 shutdown_connections 2024-03-29T04:20:09.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a0a6a7640 1 -- 172.21.15.22:0/3910137025 wait complete. 2024-03-29T04:20:09.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a0a6a7640 1 Processor -- start 2024-03-29T04:20:09.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a0a6a7640 1 -- start start 2024-03-29T04:20:09.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a0a6a7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a0406de90 0x7f0a0414d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:09.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a0a6a7640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0a04070490 0x7f0a04156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:09.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a0a6a7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a040b5b20 0x7f0a041569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:09.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a0a6a7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0406f8f0 con 0x7f0a0406de90 2024-03-29T04:20:09.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a0a6a7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0406f770 con 0x7f0a040b5b20 2024-03-29T04:20:09.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a0a6a7640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0a0406fa70 con 0x7f0a04070490 2024-03-29T04:20:09.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a03fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a0406de90 0x7f0a0414d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:09.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a03fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a0406de90 0x7f0a0414d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38640/0 (socket says 172.21.15.22:38640) 2024-03-29T04:20:09.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a03fff640 1 -- 172.21.15.22:0/4258282678 learned_addr learned my addr 172.21.15.22:0/4258282678 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:09.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a08c44640 1 --2- 172.21.15.22:0/4258282678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a040b5b20 0x7f0a041569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:09.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a037fe640 1 --2- 172.21.15.22:0/4258282678 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0a04070490 0x7f0a04156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a03fff640 1 -- 172.21.15.22:0/4258282678 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0a04070490 msgr2=0x7f0a04156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a03fff640 1 --2- 172.21.15.22:0/4258282678 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0a04070490 0x7f0a04156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a03fff640 1 -- 172.21.15.22:0/4258282678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a040b5b20 msgr2=0x7f0a041569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a03fff640 1 --2- 172.21.15.22:0/4258282678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a040b5b20 0x7f0a041569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.207+0000 7f0a03fff640 1 -- 172.21.15.22:0/4258282678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0415a920 con 0x7f0a0406de90 2024-03-29T04:20:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.211+0000 7f0a03fff640 1 --2- 172.21.15.22:0/4258282678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a0406de90 0x7f0a0414d380 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00efc0 tx=0x7f09fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.211+0000 7f0a00ff9640 1 -- 172.21.15.22:0/4258282678 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc018020 con 0x7f0a0406de90 2024-03-29T04:20:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.211+0000 7f0a00ff9640 1 -- 172.21.15.22:0/4258282678 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09fc014480 con 0x7f0a0406de90 2024-03-29T04:20:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.211+0000 7f0a0a6a7640 1 -- 172.21.15.22:0/4258282678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a0406cac0 con 0x7f0a0406de90 2024-03-29T04:20:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.211+0000 7f0a00ff9640 1 -- 172.21.15.22:0/4258282678 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc014850 con 0x7f0a0406de90 2024-03-29T04:20:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.211+0000 7f0a0a6a7640 1 -- 172.21.15.22:0/4258282678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a0406d080 con 0x7f0a0406de90 2024-03-29T04:20:09.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.211+0000 7f0a0a6a7640 1 -- 172.21.15.22:0/4258282678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a04068890 con 0x7f0a0406de90 2024-03-29T04:20:09.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.211+0000 7f0a00ff9640 1 -- 172.21.15.22:0/4258282678 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f09fc01b020 con 0x7f0a0406de90 2024-03-29T04:20:09.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.211+0000 7f0a00ff9640 1 --2- 172.21.15.22:0/4258282678 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f09e4067fc0 0x7f09e406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:09.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.211+0000 7f0a00ff9640 1 -- 172.21.15.22:0/4258282678 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f09fc0871a0 con 0x7f0a0406de90 2024-03-29T04:20:09.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.211+0000 7f0a037fe640 1 --2- 172.21.15.22:0/4258282678 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f09e4067fc0 0x7f09e406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:09.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.211+0000 7f0a037fe640 1 --2- 172.21.15.22:0/4258282678 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f09e4067fc0 0x7f09e406a480 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0a04165460 tx=0x7f09ec007cf0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:09.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.215+0000 7f0a00ff9640 1 -- 172.21.15.22:0/4258282678 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f09fc022070 con 0x7f0a0406de90 2024-03-29T04:20:09.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.407+0000 7f0a0a6a7640 1 -- 172.21.15.22:0/4258282678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a040b53f0 con 0x7f0a0406de90 2024-03-29T04:20:09.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.407+0000 7f0a00ff9640 1 -- 172.21.15.22:0/4258282678 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5181 (secure 0 0 0) 0x7f09fc010040 con 0x7f0a0406de90 2024-03-29T04:20:09.390 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:20:09.390 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/3112156479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":3112156479},{"type":"v1","addr":"172.21.15.151:6835","nonce":3112156479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5308,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/491273202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":491273202},{"type":"v1","addr":"172.21.15.151:6837","nonce":491273202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/434199693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":434199693},{"type":"v1","addr":"172.21.15.22:6833","nonce":434199693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:18:57.990202+0000","modified":"2024-03-29T04:19:24.799606+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6835/2966923505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2966923505},{"type":"v1","addr":"172.21.15.22:6835","nonce":2966923505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529]},"id":2}]} 2024-03-29T04:20:09.390 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-03-29T04:20:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.411+0000 7f0a0a6a7640 1 -- 172.21.15.22:0/4258282678 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f09e4067fc0 msgr2=0x7f09e406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.411+0000 7f0a0a6a7640 1 --2- 172.21.15.22:0/4258282678 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f09e4067fc0 0x7f09e406a480 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0a04165460 tx=0x7f09ec007cf0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.411+0000 7f0a0a6a7640 1 -- 172.21.15.22:0/4258282678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a0406de90 msgr2=0x7f0a0414d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.411+0000 7f0a0a6a7640 1 --2- 172.21.15.22:0/4258282678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a0406de90 0x7f0a0414d380 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00efc0 tx=0x7f09fc00c490 comp rx=0 tx=0).stop 2024-03-29T04:20:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.411+0000 7f0a0a6a7640 1 -- 172.21.15.22:0/4258282678 shutdown_connections 2024-03-29T04:20:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.411+0000 7f0a0a6a7640 1 --2- 172.21.15.22:0/4258282678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a040b5b20 0x7f0a041569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.411+0000 7f0a0a6a7640 1 --2- 172.21.15.22:0/4258282678 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0a04070490 0x7f0a04156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.411+0000 7f0a0a6a7640 1 --2- 172.21.15.22:0/4258282678 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f09e4067fc0 0x7f09e406a480 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.411+0000 7f0a0a6a7640 1 --2- 172.21.15.22:0/4258282678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a0406de90 0x7f0a0414d380 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.411+0000 7f0a0a6a7640 1 -- 172.21.15.22:0/4258282678 >> 172.21.15.22:0/4258282678 conn(0x7f0a04072500 msgr2=0x7f0a04164200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.411+0000 7f0a0a6a7640 1 -- 172.21.15.22:0/4258282678 shutdown_connections 2024-03-29T04:20:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.411+0000 7f0a0a6a7640 1 -- 172.21.15.22:0/4258282678 wait complete. 2024-03-29T04:20:09.408 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:20:09.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf146d9640 1 Processor -- start 2024-03-29T04:20:09.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf146d9640 1 -- start start 2024-03-29T04:20:09.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf146d9640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c1420a0 0x7fcf0c0cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:09.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf146d9640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf0c0cdc10 con 0x7fcf0c15c0a0 2024-03-29T04:20:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf146d9640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf0c0cf060 con 0x7fcf0c0cfb00 2024-03-29T04:20:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf146d9640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fcf0c0cf670 con 0x7fcf0c1420a0 2024-03-29T04:20:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf12475640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fcf0c15c0a0 0x7fcf0c06b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:45570/0 (socket says 172.21.15.22:45570) 2024-03-29T04:20:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf12475640 1 -- 172.21.15.22:0/3802110286 learned_addr learned my addr 172.21.15.22:0/3802110286 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf12c76640 1 --2- 172.21.15.22:0/3802110286 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c1420a0 0x7fcf0c0cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf11473640 1 -- 172.21.15.22:0/3802110286 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1835487703 0 0) 0x7fcf0c0cdc10 con 0x7fcf0c15c0a0 2024-03-29T04:20:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf11473640 1 -- 172.21.15.22:0/3802110286 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcef8003530 con 0x7fcf0c15c0a0 2024-03-29T04:20:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf11473640 1 -- 172.21.15.22:0/3802110286 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1381367335 0 0) 0x7fcef8003530 con 0x7fcf0c15c0a0 2024-03-29T04:20:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf11473640 1 -- 172.21.15.22:0/3802110286 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c1420a0 msgr2=0x7fcf0c0cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf11473640 1 --2- 172.21.15.22:0/3802110286 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c1420a0 0x7fcf0c0cff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf11473640 1 -- 172.21.15.22:0/3802110286 >> v1:172.21.15.151:6789/0 conn(0x7fcf0c0cfb00 legacy=0x7fcf0c141990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf11473640 1 -- 172.21.15.22:0/3802110286 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0c0cd650 con 0x7fcf0c15c0a0 2024-03-29T04:20:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf11473640 1 -- 172.21.15.22:0/3802110286 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7fcf04001a00 con 0x7fcf0c15c0a0 2024-03-29T04:20:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf11473640 1 -- 172.21.15.22:0/3802110286 >> v1:172.21.15.22:6789/0 conn(0x7fcf0c15c0a0 legacy=0x7fcf0c06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf11473640 1 --2- 172.21.15.22:0/3802110286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcef80039f0 0x7fcef8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf11473640 1 --2- 172.21.15.22:0/3802110286 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c1420a0 0x7fcef8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf11473640 1 --2- 172.21.15.22:0/3802110286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcef8004d50 0x7fcef80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf11473640 1 -- 172.21.15.22:0/3802110286 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c0cf670 con 0x7fcef80039f0 2024-03-29T04:20:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf11473640 1 -- 172.21.15.22:0/3802110286 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcef800ba00 con 0x7fcef8004d50 2024-03-29T04:20:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf11473640 1 -- 172.21.15.22:0/3802110286 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fcef800bc00 con 0x7fcf0c1420a0 2024-03-29T04:20:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf12475640 1 --2- 172.21.15.22:0/3802110286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcef80039f0 0x7fcef8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf12475640 1 -- 172.21.15.22:0/3802110286 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c1420a0 msgr2=0x7fcef8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf12475640 1 --2- 172.21.15.22:0/3802110286 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c1420a0 0x7fcef8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf12475640 1 -- 172.21.15.22:0/3802110286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcef8004d50 msgr2=0x7fcef80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf12475640 1 --2- 172.21.15.22:0/3802110286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcef8004d50 0x7fcef80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.555+0000 7fcf12475640 1 -- 172.21.15.22:0/3802110286 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef8003530 con 0x7fcef80039f0 2024-03-29T04:20:09.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf12475640 1 --2- 172.21.15.22:0/3802110286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcef80039f0 0x7fcef8003dd0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400a920 tx=0x7fcf0401d350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3535dd6471dd266e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:09.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf11473640 1 -- 172.21.15.22:0/3802110286 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0400a4c0 con 0x7fcef80039f0 2024-03-29T04:20:09.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf11473640 1 -- 172.21.15.22:0/3802110286 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf040ad650 con 0x7fcef80039f0 2024-03-29T04:20:09.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf11473640 1 -- 172.21.15.22:0/3802110286 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf040a5e60 con 0x7fcef80039f0 2024-03-29T04:20:09.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf146d9640 1 -- 172.21.15.22:0/3802110286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcef80039f0 msgr2=0x7fcef8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:09.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf146d9640 1 --2- 172.21.15.22:0/3802110286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcef80039f0 0x7fcef8003dd0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400a920 tx=0x7fcf0401d350 comp rx=0 tx=0).stop 2024-03-29T04:20:09.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf12c76640 1 -- 172.21.15.22:0/3802110286 reap_dead start 2024-03-29T04:20:09.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf146d9640 1 -- 172.21.15.22:0/3802110286 shutdown_connections 2024-03-29T04:20:09.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf146d9640 1 -- 172.21.15.22:0/3802110286 >> 172.21.15.22:0/3802110286 conn(0x7fcf0c0b2980 msgr2=0x7fcf0c0d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:09.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf146d9640 1 -- 172.21.15.22:0/3802110286 shutdown_connections 2024-03-29T04:20:09.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf146d9640 1 -- 172.21.15.22:0/3802110286 wait complete. 2024-03-29T04:20:09.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf146d9640 1 Processor -- start 2024-03-29T04:20:09.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf146d9640 1 -- start start 2024-03-29T04:20:09.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf146d9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcef80039f0 0x7fcf0c156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:09.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf146d9640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c157290 0x7fcf0c157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:09.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf146d9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf0c159b30 0x7fcf0c159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:09.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf146d9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c15c480 con 0x7fcef80039f0 2024-03-29T04:20:09.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf146d9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c0d0310 con 0x7fcf0c159b30 2024-03-29T04:20:09.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf146d9640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fcf0c15c600 con 0x7fcf0c157290 2024-03-29T04:20:09.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf12475640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcef80039f0 0x7fcf0c156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:09.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf12475640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcef80039f0 0x7fcf0c156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38660/0 (socket says 172.21.15.22:38660) 2024-03-29T04:20:09.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.559+0000 7fcf12475640 1 -- 172.21.15.22:0/3402461875 learned_addr learned my addr 172.21.15.22:0/3402461875 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:09.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.563+0000 7fcf12c76640 1 --2- 172.21.15.22:0/3402461875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf0c159b30 0x7fcf0c159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:09.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.563+0000 7fcf11c74640 1 --2- 172.21.15.22:0/3402461875 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c157290 0x7fcf0c157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:09.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.563+0000 7fcf12475640 1 -- 172.21.15.22:0/3402461875 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c157290 msgr2=0x7fcf0c157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:09.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.563+0000 7fcf12475640 1 --2- 172.21.15.22:0/3402461875 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c157290 0x7fcf0c157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.563+0000 7fcf12475640 1 -- 172.21.15.22:0/3402461875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf0c159b30 msgr2=0x7fcf0c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:09.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.563+0000 7fcf12475640 1 --2- 172.21.15.22:0/3402461875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf0c159b30 0x7fcf0c159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.563+0000 7fcf12475640 1 -- 172.21.15.22:0/3402461875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0c159090 con 0x7fcef80039f0 2024-03-29T04:20:09.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.563+0000 7fcf11c74640 1 --2- 172.21.15.22:0/3402461875 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c157290 0x7fcf0c157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:09.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.563+0000 7fcf12c76640 1 --2- 172.21.15.22:0/3402461875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf0c159b30 0x7fcf0c159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:09.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.563+0000 7fcf12475640 1 --2- 172.21.15.22:0/3402461875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcef80039f0 0x7fcf0c156b80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcf0401e960 tx=0x7fcef8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:09.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.563+0000 7fcefeffd640 1 -- 172.21.15.22:0/3402461875 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf040b2040 con 0x7fcef80039f0 2024-03-29T04:20:09.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.563+0000 7fcefeffd640 1 -- 172.21.15.22:0/3402461875 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf040bc070 con 0x7fcef80039f0 2024-03-29T04:20:09.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.563+0000 7fcf146d9640 1 -- 172.21.15.22:0/3402461875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf0c0cc200 con 0x7fcef80039f0 2024-03-29T04:20:09.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.563+0000 7fcefeffd640 1 -- 172.21.15.22:0/3402461875 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf040017c0 con 0x7fcef80039f0 2024-03-29T04:20:09.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.563+0000 7fcf146d9640 1 -- 172.21.15.22:0/3402461875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf0c159610 con 0x7fcef80039f0 2024-03-29T04:20:09.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.563+0000 7fcf146d9640 1 -- 172.21.15.22:0/3402461875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf0c068890 con 0x7fcef80039f0 2024-03-29T04:20:09.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.563+0000 7fcefeffd640 1 -- 172.21.15.22:0/3402461875 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fcf04004b70 con 0x7fcef80039f0 2024-03-29T04:20:09.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.563+0000 7fcefeffd640 1 --2- 172.21.15.22:0/3402461875 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcef0067ff0 0x7fcef006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:09.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.563+0000 7fcefeffd640 1 -- 172.21.15.22:0/3402461875 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fcf041212e0 con 0x7fcef80039f0 2024-03-29T04:20:09.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.563+0000 7fcf11c74640 1 --2- 172.21.15.22:0/3402461875 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcef0067ff0 0x7fcef006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:09.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.563+0000 7fcf11c74640 1 --2- 172.21.15.22:0/3402461875 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcef0067ff0 0x7fcef006a4b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c1398c0 tx=0x7fcf00009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:09.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.571+0000 7fcefeffd640 1 -- 172.21.15.22:0/3402461875 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fcf04017090 con 0x7fcef80039f0 2024-03-29T04:20:09.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.743+0000 7fcf146d9640 1 -- 172.21.15.22:0/3402461875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf0c158e80 con 0x7fcef80039f0 2024-03-29T04:20:09.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.747+0000 7fcefeffd640 1 -- 172.21.15.22:0/3402461875 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16991 (secure 0 0 0) 0x7fcf040f1a00 con 0x7fcef80039f0 2024-03-29T04:20:09.727 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:20:09.727 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":42,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:19:17.809849+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:18:56.336876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:18:56.950670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:20:09.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.747+0000 7fcf146d9640 1 -- 172.21.15.22:0/3402461875 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcef0067ff0 msgr2=0x7fcef006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:09.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.747+0000 7fcf146d9640 1 --2- 172.21.15.22:0/3402461875 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcef0067ff0 0x7fcef006a4b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c1398c0 tx=0x7fcf00009040 comp rx=0 tx=0).stop 2024-03-29T04:20:09.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.747+0000 7fcf146d9640 1 -- 172.21.15.22:0/3402461875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcef80039f0 msgr2=0x7fcf0c156b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:09.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.747+0000 7fcf146d9640 1 --2- 172.21.15.22:0/3402461875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcef80039f0 0x7fcf0c156b80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcf0401e960 tx=0x7fcef8005840 comp rx=0 tx=0).stop 2024-03-29T04:20:09.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.747+0000 7fcf146d9640 1 -- 172.21.15.22:0/3402461875 shutdown_connections 2024-03-29T04:20:09.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.747+0000 7fcf146d9640 1 --2- 172.21.15.22:0/3402461875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf0c159b30 0x7fcf0c159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.747+0000 7fcf146d9640 1 --2- 172.21.15.22:0/3402461875 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fcf0c157290 0x7fcf0c157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.747+0000 7fcf146d9640 1 --2- 172.21.15.22:0/3402461875 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fcef0067ff0 0x7fcef006a4b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.747+0000 7fcf146d9640 1 --2- 172.21.15.22:0/3402461875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcef80039f0 0x7fcf0c156b80 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.747+0000 7fcf146d9640 1 -- 172.21.15.22:0/3402461875 >> 172.21.15.22:0/3402461875 conn(0x7fcf0c0b2980 msgr2=0x7fcf0c145390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:09.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.747+0000 7fcf146d9640 1 -- 172.21.15.22:0/3402461875 shutdown_connections 2024-03-29T04:20:09.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.747+0000 7fcf146d9640 1 -- 172.21.15.22:0/3402461875 wait complete. 2024-03-29T04:20:09.746 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:17:23.735304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:18:56.336876+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:18:56.950670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:20:09.747 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:20:09.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.883+0000 7fe65e20f640 1 Processor -- start 2024-03-29T04:20:09.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.883+0000 7fe65e20f640 1 -- start start 2024-03-29T04:20:09.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.883+0000 7fe65e20f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe65806c790 0x7fe65806b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:09.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.883+0000 7fe65e20f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe65806f9e0 0x7fe65806fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:09.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.883+0000 7fe65e20f640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe658070340 con 0x7fe658161fd0 2024-03-29T04:20:09.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.883+0000 7fe65e20f640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fe65815f8b0 con 0x7fe65806f9e0 2024-03-29T04:20:09.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.883+0000 7fe65e20f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe65815e9e0 con 0x7fe65806c790 2024-03-29T04:20:09.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.883+0000 7fe656ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe65806f9e0 0x7fe65806fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:09.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.883+0000 7fe656ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe65806f9e0 0x7fe65806fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38672/0 (socket says 172.21.15.22:38672) 2024-03-29T04:20:09.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.883+0000 7fe656ffd640 1 -- 172.21.15.22:0/3172397389 learned_addr learned my addr 172.21.15.22:0/3172397389 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:09.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.883+0000 7fe6577fe640 1 --2- 172.21.15.22:0/3172397389 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe65806c790 0x7fe65806b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:09.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.883+0000 7fe656ffd640 1 -- 172.21.15.22:0/3172397389 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe65806c790 msgr2=0x7fe65806b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:09.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.883+0000 7fe656ffd640 1 --2- 172.21.15.22:0/3172397389 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe65806c790 0x7fe65806b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.883+0000 7fe656ffd640 1 -- 172.21.15.22:0/3172397389 >> v1:172.21.15.151:6789/0 conn(0x7fe658161fd0 legacy=0x7fe658162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:09.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.883+0000 7fe656ffd640 1 -- 172.21.15.22:0/3172397389 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65815eca0 con 0x7fe65806f9e0 2024-03-29T04:20:09.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.883+0000 7fe6577fe640 1 --2- 172.21.15.22:0/3172397389 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe65806c790 0x7fe65806b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:09.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.887+0000 7fe656ffd640 1 --2- 172.21.15.22:0/3172397389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe65806f9e0 0x7fe65806fe00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe648009e50 tx=0x7fe64801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652bb2549ae479f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:09.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.887+0000 7fe6567fc640 1 -- 172.21.15.22:0/3172397389 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6480a5840 con 0x7fe65806f9e0 2024-03-29T04:20:09.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.887+0000 7fe6567fc640 1 -- 172.21.15.22:0/3172397389 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6480a59e0 con 0x7fe65806f9e0 2024-03-29T04:20:09.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.887+0000 7fe6567fc640 1 -- 172.21.15.22:0/3172397389 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6480a5db0 con 0x7fe65806f9e0 2024-03-29T04:20:09.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.887+0000 7fe65e20f640 1 -- 172.21.15.22:0/3172397389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe65806f9e0 msgr2=0x7fe65806fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:09.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.887+0000 7fe65e20f640 1 --2- 172.21.15.22:0/3172397389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe65806f9e0 0x7fe65806fe00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe648009e50 tx=0x7fe64801c9c0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.887+0000 7fe65e20f640 1 -- 172.21.15.22:0/3172397389 shutdown_connections 2024-03-29T04:20:09.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.887+0000 7fe65e20f640 1 --2- 172.21.15.22:0/3172397389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe65806f9e0 0x7fe65806fe00 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.887+0000 7fe65e20f640 1 --2- 172.21.15.22:0/3172397389 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe65806c790 0x7fe65806b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.887+0000 7fe65e20f640 1 -- 172.21.15.22:0/3172397389 >> 172.21.15.22:0/3172397389 conn(0x7fe6580d1070 msgr2=0x7fe6580d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:09.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.887+0000 7fe65e20f640 1 -- 172.21.15.22:0/3172397389 shutdown_connections 2024-03-29T04:20:09.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.887+0000 7fe65e20f640 1 -- 172.21.15.22:0/3172397389 wait complete. 2024-03-29T04:20:09.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.887+0000 7fe65e20f640 1 Processor -- start 2024-03-29T04:20:09.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.887+0000 7fe65e20f640 1 -- start start 2024-03-29T04:20:09.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe65e20f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe65806c790 0x7fe65814bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:09.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe65e20f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe65806f9e0 0x7fe6580b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:09.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe65e20f640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe658161fd0 0x7fe658153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:09.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe65e20f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe65806cb70 con 0x7fe65806c790 2024-03-29T04:20:09.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe65e20f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe658070b20 con 0x7fe65806f9e0 2024-03-29T04:20:09.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe65e20f640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe65806ccf0 con 0x7fe658161fd0 2024-03-29T04:20:09.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe6577fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe65806c790 0x7fe65814bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:09.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe6577fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe65806c790 0x7fe65814bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38688/0 (socket says 172.21.15.22:38688) 2024-03-29T04:20:09.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe6577fe640 1 -- 172.21.15.22:0/3725490106 learned_addr learned my addr 172.21.15.22:0/3725490106 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:09.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe6577fe640 1 -- 172.21.15.22:0/3725490106 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe658161fd0 msgr2=0x7fe658153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:09.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe6577fe640 1 --2- 172.21.15.22:0/3725490106 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe658161fd0 0x7fe658153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe656ffd640 1 --2- 172.21.15.22:0/3725490106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe65806f9e0 0x7fe6580b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:09.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe6577fe640 1 -- 172.21.15.22:0/3725490106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe65806f9e0 msgr2=0x7fe6580b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:09.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe6577fe640 1 --2- 172.21.15.22:0/3725490106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe65806f9e0 0x7fe6580b1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:09.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe6577fe640 1 -- 172.21.15.22:0/3725490106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65814f6c0 con 0x7fe65806c790 2024-03-29T04:20:09.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe6577fe640 1 --2- 172.21.15.22:0/3725490106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe65806c790 0x7fe65814bf30 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe64000c820 tx=0x7fe64000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:09.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe64ffff640 1 -- 172.21.15.22:0/3725490106 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe640015070 con 0x7fe65806c790 2024-03-29T04:20:09.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe64ffff640 1 -- 172.21.15.22:0/3725490106 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe640007a30 con 0x7fe65806c790 2024-03-29T04:20:09.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe65e20f640 1 -- 172.21.15.22:0/3725490106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe65814e180 con 0x7fe65806c790 2024-03-29T04:20:09.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe64ffff640 1 -- 172.21.15.22:0/3725490106 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe640007e40 con 0x7fe65806c790 2024-03-29T04:20:09.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe65e20f640 1 -- 172.21.15.22:0/3725490106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe658153510 con 0x7fe65806c790 2024-03-29T04:20:09.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe65e20f640 1 -- 172.21.15.22:0/3725490106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe658068900 con 0x7fe65806c790 2024-03-29T04:20:09.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe64ffff640 1 -- 172.21.15.22:0/3725490106 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe640018070 con 0x7fe65806c790 2024-03-29T04:20:09.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe64ffff640 1 --2- 172.21.15.22:0/3725490106 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe63c067fc0 0x7fe63c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:09.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe64ffff640 1 -- 172.21.15.22:0/3725490106 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fe640086500 con 0x7fe65806c790 2024-03-29T04:20:09.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe656ffd640 1 --2- 172.21.15.22:0/3725490106 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe63c067fc0 0x7fe63c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:09.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.891+0000 7fe656ffd640 1 --2- 172.21.15.22:0/3725490106 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe63c067fc0 0x7fe63c06a480 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe6480ae000 tx=0x7fe6480a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:09.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:09.895+0000 7fe64ffff640 1 -- 172.21.15.22:0/3725490106 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fe640056b90 con 0x7fe65806c790 2024-03-29T04:20:10.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.063+0000 7fe65e20f640 1 -- 172.21.15.22:0/3725490106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe6580706c0 con 0x7fe65806c790 2024-03-29T04:20:10.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.063+0000 7fe64ffff640 1 -- 172.21.15.22:0/3725490106 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) ==== 93+0+10 (secure 0 0 0) 0x7fe640056490 con 0x7fe65806c790 2024-03-29T04:20:10.045 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-29T04:20:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.067+0000 7fe65e20f640 1 -- 172.21.15.22:0/3725490106 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe63c067fc0 msgr2=0x7fe63c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.067+0000 7fe65e20f640 1 --2- 172.21.15.22:0/3725490106 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe63c067fc0 0x7fe63c06a480 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe6480ae000 tx=0x7fe6480a7070 comp rx=0 tx=0).stop 2024-03-29T04:20:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.067+0000 7fe65e20f640 1 -- 172.21.15.22:0/3725490106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe65806c790 msgr2=0x7fe65814bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.067+0000 7fe65e20f640 1 --2- 172.21.15.22:0/3725490106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe65806c790 0x7fe65814bf30 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe64000c820 tx=0x7fe64000cce0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.067+0000 7fe65e20f640 1 -- 172.21.15.22:0/3725490106 shutdown_connections 2024-03-29T04:20:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.067+0000 7fe65e20f640 1 --2- 172.21.15.22:0/3725490106 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe658161fd0 0x7fe658153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.067+0000 7fe65e20f640 1 --2- 172.21.15.22:0/3725490106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe65806f9e0 0x7fe6580b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.067+0000 7fe65e20f640 1 --2- 172.21.15.22:0/3725490106 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe63c067fc0 0x7fe63c06a480 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.067+0000 7fe65e20f640 1 --2- 172.21.15.22:0/3725490106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe65806c790 0x7fe65814bf30 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.067+0000 7fe65e20f640 1 -- 172.21.15.22:0/3725490106 >> 172.21.15.22:0/3725490106 conn(0x7fe6580d1070 msgr2=0x7fe65815fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.067+0000 7fe65e20f640 1 -- 172.21.15.22:0/3725490106 shutdown_connections 2024-03-29T04:20:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.067+0000 7fe65e20f640 1 -- 172.21.15.22:0/3725490106 wait complete. 2024-03-29T04:20:10.072 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:20:10.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.215+0000 7f8143a40640 1 Processor -- start 2024-03-29T04:20:10.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.215+0000 7f8143a40640 1 -- start start 2024-03-29T04:20:10.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.215+0000 7f8143a40640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f813c147e00 0x7f813c0b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:10.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.215+0000 7f8143a40640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f813c0b4f30 0x7f813c06c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:10.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.215+0000 7f8143a40640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f813c0b5470 con 0x7f813c06c5b0 2024-03-29T04:20:10.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.215+0000 7f8143a40640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f813c06ca00 con 0x7f813c147e00 2024-03-29T04:20:10.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.215+0000 7f8143a40640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f813c0b5750 con 0x7f813c0b4f30 2024-03-29T04:20:10.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.215+0000 7f81417dc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f813c147e00 0x7f813c0b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:10.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.215+0000 7f81417dc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f813c147e00 0x7f813c0b49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38694/0 (socket says 172.21.15.22:38694) 2024-03-29T04:20:10.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.215+0000 7f81417dc640 1 -- 172.21.15.22:0/2703053263 learned_addr learned my addr 172.21.15.22:0/2703053263 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:10.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.215+0000 7f8140fdb640 1 --2- 172.21.15.22:0/2703053263 >> v2:172.21.15.151:3300/0 conn(0x7f813c0b4f30 0x7f813c06c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:10.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.215+0000 7f81417dc640 1 -- 172.21.15.22:0/2703053263 >> v2:172.21.15.151:3300/0 conn(0x7f813c0b4f30 msgr2=0x7f813c06c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:10.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.215+0000 7f81417dc640 1 --2- 172.21.15.22:0/2703053263 >> v2:172.21.15.151:3300/0 conn(0x7f813c0b4f30 0x7f813c06c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.215+0000 7f81417dc640 1 -- 172.21.15.22:0/2703053263 >> v1:172.21.15.151:6789/0 conn(0x7f813c06c5b0 legacy=0x7f813c06f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:10.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.215+0000 7f81417dc640 1 -- 172.21.15.22:0/2703053263 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813c0b5a30 con 0x7f813c147e00 2024-03-29T04:20:10.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.215+0000 7f81417dc640 1 --2- 172.21.15.22:0/2703053263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f813c147e00 0x7f813c0b49f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f813c0b72a0 tx=0x7f813801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff7ef939d466c6a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:10.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8133fff640 1 -- 172.21.15.22:0/2703053263 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81380b0070 con 0x7f813c147e00 2024-03-29T04:20:10.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8133fff640 1 -- 172.21.15.22:0/2703053263 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81380ab600 con 0x7f813c147e00 2024-03-29T04:20:10.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8133fff640 1 -- 172.21.15.22:0/2703053263 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81380ab9d0 con 0x7f813c147e00 2024-03-29T04:20:10.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8143a40640 1 -- 172.21.15.22:0/2703053263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f813c147e00 msgr2=0x7f813c0b49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:10.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8143a40640 1 --2- 172.21.15.22:0/2703053263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f813c147e00 0x7f813c0b49f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f813c0b72a0 tx=0x7f813801c4e0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8143a40640 1 -- 172.21.15.22:0/2703053263 shutdown_connections 2024-03-29T04:20:10.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8143a40640 1 --2- 172.21.15.22:0/2703053263 >> v2:172.21.15.151:3300/0 conn(0x7f813c0b4f30 0x7f813c06c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8143a40640 1 --2- 172.21.15.22:0/2703053263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f813c147e00 0x7f813c0b49f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8143a40640 1 -- 172.21.15.22:0/2703053263 >> 172.21.15.22:0/2703053263 conn(0x7f813c0d1070 msgr2=0x7f813c0d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:10.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8143a40640 1 -- 172.21.15.22:0/2703053263 shutdown_connections 2024-03-29T04:20:10.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8143a40640 1 -- 172.21.15.22:0/2703053263 wait complete. 2024-03-29T04:20:10.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8143a40640 1 Processor -- start 2024-03-29T04:20:10.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8143a40640 1 -- start start 2024-03-29T04:20:10.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8143a40640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f813c06c5b0 0x7f813c15bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:10.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8143a40640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f813c0b4f30 0x7f813c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:10.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8143a40640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f813c147e00 0x7f813c15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:10.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8143a40640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f813c16b4b0 con 0x7f813c147e00 2024-03-29T04:20:10.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8143a40640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f813c16b330 con 0x7f813c06c5b0 2024-03-29T04:20:10.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8143a40640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f813c16b630 con 0x7f813c0b4f30 2024-03-29T04:20:10.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8141fdd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f813c147e00 0x7f813c15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:10.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8141fdd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f813c147e00 0x7f813c15ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38708/0 (socket says 172.21.15.22:38708) 2024-03-29T04:20:10.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8141fdd640 1 -- 172.21.15.22:0/356093505 learned_addr learned my addr 172.21.15.22:0/356093505 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:10.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8140fdb640 1 --2- 172.21.15.22:0/356093505 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f813c0b4f30 0x7f813c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:10.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f81417dc640 1 --2- 172.21.15.22:0/356093505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f813c06c5b0 0x7f813c15bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:10.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8141fdd640 1 -- 172.21.15.22:0/356093505 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f813c0b4f30 msgr2=0x7f813c15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:10.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8141fdd640 1 --2- 172.21.15.22:0/356093505 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f813c0b4f30 0x7f813c15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8141fdd640 1 -- 172.21.15.22:0/356093505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f813c06c5b0 msgr2=0x7f813c15bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:10.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8141fdd640 1 --2- 172.21.15.22:0/356093505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f813c06c5b0 0x7f813c15bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8141fdd640 1 -- 172.21.15.22:0/356093505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813c164100 con 0x7f813c147e00 2024-03-29T04:20:10.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8141fdd640 1 --2- 172.21.15.22:0/356093505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f813c147e00 0x7f813c15ed60 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f813400c780 tx=0x7f813400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:10.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.219+0000 7f8131ffb640 1 -- 172.21.15.22:0/356093505 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8134004b70 con 0x7f813c147e00 2024-03-29T04:20:10.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.223+0000 7f8131ffb640 1 -- 172.21.15.22:0/356093505 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8134004d10 con 0x7f813c147e00 2024-03-29T04:20:10.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.223+0000 7f8143a40640 1 -- 172.21.15.22:0/356093505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f813c146e30 con 0x7f813c147e00 2024-03-29T04:20:10.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.223+0000 7f8131ffb640 1 -- 172.21.15.22:0/356093505 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813400aa90 con 0x7f813c147e00 2024-03-29T04:20:10.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.223+0000 7f8143a40640 1 -- 172.21.15.22:0/356093505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f813c15d8b0 con 0x7f813c147e00 2024-03-29T04:20:10.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.223+0000 7f8143a40640 1 -- 172.21.15.22:0/356093505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f813c068900 con 0x7f813c147e00 2024-03-29T04:20:10.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.223+0000 7f8131ffb640 1 -- 172.21.15.22:0/356093505 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f813400ac30 con 0x7f813c147e00 2024-03-29T04:20:10.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.223+0000 7f8131ffb640 1 --2- 172.21.15.22:0/356093505 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8120067fc0 0x7f812006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:10.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.223+0000 7f8131ffb640 1 -- 172.21.15.22:0/356093505 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f8134086860 con 0x7f813c147e00 2024-03-29T04:20:10.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.223+0000 7f81417dc640 1 --2- 172.21.15.22:0/356093505 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8120067fc0 0x7f812006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:10.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.223+0000 7f81417dc640 1 --2- 172.21.15.22:0/356093505 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8120067fc0 0x7f812006a480 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f81380ad000 tx=0x7f8138008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:10.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.227+0000 7f8131ffb640 1 -- 172.21.15.22:0/356093505 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f8134020070 con 0x7f813c147e00 2024-03-29T04:20:10.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.383+0000 7f8143a40640 1 -- 172.21.15.22:0/356093505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f813c070a90 con 0x7f813c147e00 2024-03-29T04:20:10.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.383+0000 7f8131ffb640 1 -- 172.21.15.22:0/356093505 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) ==== 104+0+11 (secure 0 0 0) 0x7f813c164100 con 0x7f813c147e00 2024-03-29T04:20:10.365 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-29T04:20:10.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.387+0000 7f8143a40640 1 -- 172.21.15.22:0/356093505 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8120067fc0 msgr2=0x7f812006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:10.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.387+0000 7f8143a40640 1 --2- 172.21.15.22:0/356093505 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8120067fc0 0x7f812006a480 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f81380ad000 tx=0x7f8138008040 comp rx=0 tx=0).stop 2024-03-29T04:20:10.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.387+0000 7f8143a40640 1 -- 172.21.15.22:0/356093505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f813c147e00 msgr2=0x7f813c15ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:10.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.387+0000 7f8143a40640 1 --2- 172.21.15.22:0/356093505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f813c147e00 0x7f813c15ed60 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f813400c780 tx=0x7f813400cc40 comp rx=0 tx=0).stop 2024-03-29T04:20:10.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.387+0000 7f8143a40640 1 -- 172.21.15.22:0/356093505 shutdown_connections 2024-03-29T04:20:10.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.387+0000 7f8143a40640 1 --2- 172.21.15.22:0/356093505 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f8120067fc0 0x7f812006a480 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.387+0000 7f8143a40640 1 --2- 172.21.15.22:0/356093505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f813c147e00 0x7f813c15ed60 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.387+0000 7f8143a40640 1 --2- 172.21.15.22:0/356093505 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f813c0b4f30 0x7f813c15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.387+0000 7f8143a40640 1 --2- 172.21.15.22:0/356093505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f813c06c5b0 0x7f813c15bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.387+0000 7f8143a40640 1 -- 172.21.15.22:0/356093505 >> 172.21.15.22:0/356093505 conn(0x7f813c0d1070 msgr2=0x7f813c0b3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:10.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.387+0000 7f8143a40640 1 -- 172.21.15.22:0/356093505 shutdown_connections 2024-03-29T04:20:10.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.387+0000 7f8143a40640 1 -- 172.21.15.22:0/356093505 wait complete. 2024-03-29T04:20:10.383 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:20:10.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.523+0000 7f2265523640 1 Processor -- start 2024-03-29T04:20:10.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.523+0000 7f2265523640 1 -- start start 2024-03-29T04:20:10.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.523+0000 7f2265523640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f226015f880 0x7f22600b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:10.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.523+0000 7f2265523640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f226006f090 0x7f22600ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:10.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.523+0000 7f2265523640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22600ce4d0 con 0x7f2260070640 2024-03-29T04:20:10.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.523+0000 7f2265523640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f2260070ad0 con 0x7f226015f880 2024-03-29T04:20:10.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.523+0000 7f2265523640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f22600cc7f0 con 0x7f226006f090 2024-03-29T04:20:10.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.523+0000 7f225effd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f226015f880 0x7f22600b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:10.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.523+0000 7f225effd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f226015f880 0x7f22600b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38722/0 (socket says 172.21.15.22:38722) 2024-03-29T04:20:10.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.523+0000 7f225effd640 1 -- 172.21.15.22:0/4115689756 learned_addr learned my addr 172.21.15.22:0/4115689756 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:10.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.523+0000 7f225e7fc640 1 --1- 172.21.15.22:0/4115689756 >> v1:172.21.15.22:6789/0 conn(0x7f2260070640 0x7f226006e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:45574/0 (socket says 172.21.15.22:45574) 2024-03-29T04:20:10.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.523+0000 7f225f7fe640 1 --2- 172.21.15.22:0/4115689756 >> v2:172.21.15.151:3300/0 conn(0x7f226006f090 0x7f22600ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:10.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.523+0000 7f225effd640 1 -- 172.21.15.22:0/4115689756 >> v2:172.21.15.151:3300/0 conn(0x7f226006f090 msgr2=0x7f22600ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:10.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.523+0000 7f225effd640 1 --2- 172.21.15.22:0/4115689756 >> v2:172.21.15.151:3300/0 conn(0x7f226006f090 0x7f22600ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.523+0000 7f225effd640 1 -- 172.21.15.22:0/4115689756 >> v1:172.21.15.22:6789/0 conn(0x7f2260070640 legacy=0x7f226006e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:10.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.523+0000 7f225dffb640 1 -- 172.21.15.22:0/4115689756 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 773823682 0 0) 0x7f22600ce4d0 con 0x7f2260070640 2024-03-29T04:20:10.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.523+0000 7f225effd640 1 -- 172.21.15.22:0/4115689756 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22600ccad0 con 0x7f226015f880 2024-03-29T04:20:10.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.527+0000 7f225effd640 1 --2- 172.21.15.22:0/4115689756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f226015f880 0x7f22600b4ff0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f22600b1c10 tx=0x7f225001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0bfeeb34ecffa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:10.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.527+0000 7f225dffb640 1 -- 172.21.15.22:0/4115689756 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22500b1070 con 0x7f226015f880 2024-03-29T04:20:10.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.527+0000 7f225dffb640 1 -- 172.21.15.22:0/4115689756 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22500ac640 con 0x7f226015f880 2024-03-29T04:20:10.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.527+0000 7f225dffb640 1 -- 172.21.15.22:0/4115689756 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22500aca10 con 0x7f226015f880 2024-03-29T04:20:10.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.527+0000 7f2265523640 1 -- 172.21.15.22:0/4115689756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f226015f880 msgr2=0x7f22600b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:10.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.527+0000 7f2265523640 1 --2- 172.21.15.22:0/4115689756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f226015f880 0x7f22600b4ff0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f22600b1c10 tx=0x7f225001c400 comp rx=0 tx=0).stop 2024-03-29T04:20:10.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.527+0000 7f2265523640 1 -- 172.21.15.22:0/4115689756 shutdown_connections 2024-03-29T04:20:10.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.527+0000 7f2265523640 1 --2- 172.21.15.22:0/4115689756 >> v2:172.21.15.151:3300/0 conn(0x7f226006f090 0x7f22600ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.527+0000 7f2265523640 1 --2- 172.21.15.22:0/4115689756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f226015f880 0x7f22600b4ff0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.527+0000 7f2265523640 1 -- 172.21.15.22:0/4115689756 >> 172.21.15.22:0/4115689756 conn(0x7f22600b3cf0 msgr2=0x7f22600b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:10.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.527+0000 7f2265523640 1 -- 172.21.15.22:0/4115689756 shutdown_connections 2024-03-29T04:20:10.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.527+0000 7f2265523640 1 -- 172.21.15.22:0/4115689756 wait complete. 2024-03-29T04:20:10.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f2265523640 1 Processor -- start 2024-03-29T04:20:10.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f2265523640 1 -- start start 2024-03-29T04:20:10.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f2265523640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f226006f090 0x7f2260156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:10.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f2265523640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2260070640 0x7f226015a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:10.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f2265523640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f226015f880 0x7f22600c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:10.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f2265523640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f22601609d0 con 0x7f226006f090 2024-03-29T04:20:10.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f2265523640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2260160850 con 0x7f2260070640 2024-03-29T04:20:10.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f2265523640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f22601691e0 con 0x7f226015f880 2024-03-29T04:20:10.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f225effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f226006f090 0x7f2260156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:10.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f225effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f226006f090 0x7f2260156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38728/0 (socket says 172.21.15.22:38728) 2024-03-29T04:20:10.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f225effd640 1 -- 172.21.15.22:0/2476223435 learned_addr learned my addr 172.21.15.22:0/2476223435 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:10.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f225e7fc640 1 --2- 172.21.15.22:0/2476223435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2260070640 0x7f226015a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:10.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f225f7fe640 1 --2- 172.21.15.22:0/2476223435 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f226015f880 0x7f22600c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:10.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f225effd640 1 -- 172.21.15.22:0/2476223435 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f226015f880 msgr2=0x7f22600c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:10.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f225effd640 1 --2- 172.21.15.22:0/2476223435 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f226015f880 0x7f22600c5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f225effd640 1 -- 172.21.15.22:0/2476223435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2260070640 msgr2=0x7f226015a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:10.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f225effd640 1 --2- 172.21.15.22:0/2476223435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2260070640 0x7f226015a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f225effd640 1 -- 172.21.15.22:0/2476223435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22600c5a10 con 0x7f226006f090 2024-03-29T04:20:10.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f225effd640 1 --2- 172.21.15.22:0/2476223435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f226006f090 0x7f2260156520 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f225001a030 tx=0x7f225001ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:10.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f22477fe640 1 -- 172.21.15.22:0/2476223435 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22500bb070 con 0x7f226006f090 2024-03-29T04:20:10.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f22477fe640 1 -- 172.21.15.22:0/2476223435 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2250004020 con 0x7f226006f090 2024-03-29T04:20:10.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f2265523640 1 -- 172.21.15.22:0/2476223435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f226015aad0 con 0x7f226006f090 2024-03-29T04:20:10.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f22477fe640 1 -- 172.21.15.22:0/2476223435 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22500b1040 con 0x7f226006f090 2024-03-29T04:20:10.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.531+0000 7f2265523640 1 -- 172.21.15.22:0/2476223435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f226015b090 con 0x7f226006f090 2024-03-29T04:20:10.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.535+0000 7f2265523640 1 -- 172.21.15.22:0/2476223435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2260068900 con 0x7f226006f090 2024-03-29T04:20:10.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.535+0000 7f22477fe640 1 -- 172.21.15.22:0/2476223435 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f2250003620 con 0x7f226006f090 2024-03-29T04:20:10.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.535+0000 7f22477fe640 1 --2- 172.21.15.22:0/2476223435 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f224c067ff0 0x7f224c06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:10.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.535+0000 7f22477fe640 1 -- 172.21.15.22:0/2476223435 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f22501207d0 con 0x7f226006f090 2024-03-29T04:20:10.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.535+0000 7f225e7fc640 1 --2- 172.21.15.22:0/2476223435 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f224c067ff0 0x7f224c06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:10.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.535+0000 7f225e7fc640 1 --2- 172.21.15.22:0/2476223435 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f224c067ff0 0x7f224c06a4b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f22600b47b0 tx=0x7f2254007c30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:10.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.539+0000 7f22477fe640 1 -- 172.21.15.22:0/2476223435 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f2250016110 con 0x7f226006f090 2024-03-29T04:20:10.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.703+0000 7f2265523640 1 -- 172.21.15.22:0/2476223435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f22600d1790 con 0x7f226006f090 2024-03-29T04:20:10.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.703+0000 7f22477fe640 1 -- 172.21.15.22:0/2476223435 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) ==== 100+0+11 (secure 0 0 0) 0x7f22500f0e60 con 0x7f226006f090 2024-03-29T04:20:10.686 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-29T04:20:10.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.707+0000 7f2265523640 1 -- 172.21.15.22:0/2476223435 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f224c067ff0 msgr2=0x7f224c06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:10.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.707+0000 7f2265523640 1 --2- 172.21.15.22:0/2476223435 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f224c067ff0 0x7f224c06a4b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f22600b47b0 tx=0x7f2254007c30 comp rx=0 tx=0).stop 2024-03-29T04:20:10.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.707+0000 7f2265523640 1 -- 172.21.15.22:0/2476223435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f226006f090 msgr2=0x7f2260156520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:10.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.707+0000 7f2265523640 1 --2- 172.21.15.22:0/2476223435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f226006f090 0x7f2260156520 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f225001a030 tx=0x7f225001ac40 comp rx=0 tx=0).stop 2024-03-29T04:20:10.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.707+0000 7f2265523640 1 -- 172.21.15.22:0/2476223435 shutdown_connections 2024-03-29T04:20:10.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.707+0000 7f2265523640 1 --2- 172.21.15.22:0/2476223435 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f226015f880 0x7f22600c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.707+0000 7f2265523640 1 --2- 172.21.15.22:0/2476223435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2260070640 0x7f226015a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.707+0000 7f2265523640 1 --2- 172.21.15.22:0/2476223435 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f224c067ff0 0x7f224c06a4b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.707+0000 7f2265523640 1 --2- 172.21.15.22:0/2476223435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f226006f090 0x7f2260156520 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.707+0000 7f2265523640 1 -- 172.21.15.22:0/2476223435 >> 172.21.15.22:0/2476223435 conn(0x7f22600b3cf0 msgr2=0x7f2260158e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:10.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.707+0000 7f2265523640 1 -- 172.21.15.22:0/2476223435 shutdown_connections 2024-03-29T04:20:10.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.707+0000 7f2265523640 1 -- 172.21.15.22:0/2476223435 wait complete. 2024-03-29T04:20:10.706 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:20:10.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe016cea640 1 Processor -- start 2024-03-29T04:20:10.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe016cea640 1 -- start start 2024-03-29T04:20:10.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe016cea640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe01015b0f0 0x7fe0100b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:10.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe016cea640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fe0100cf380 0x7fe0100b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:10.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe016cea640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0100b5760 con 0x7fe0100afc20 2024-03-29T04:20:10.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe016cea640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fe01006e660 con 0x7fe0100cf380 2024-03-29T04:20:10.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe016cea640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe01006e940 con 0x7fe01015b0f0 2024-03-29T04:20:10.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe00ffff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe0100afc20 0x7fe0100b3680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:45588/0 (socket says 172.21.15.22:45588) 2024-03-29T04:20:10.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe00ffff640 1 -- 172.21.15.22:0/873220461 learned_addr learned my addr 172.21.15.22:0/873220461 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:10.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe015287640 1 --2- 172.21.15.22:0/873220461 >> v2:172.21.15.151:3300/0 conn(0x7fe0100cf380 0x7fe0100b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:10.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe014a86640 1 --2- 172.21.15.22:0/873220461 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe01015b0f0 0x7fe0100b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:10.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe00f7fe640 1 -- 172.21.15.22:0/873220461 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3298789101 0 0) 0x7fe0100b5760 con 0x7fe0100afc20 2024-03-29T04:20:10.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe00f7fe640 1 -- 172.21.15.22:0/873220461 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe000005740 con 0x7fe0100afc20 2024-03-29T04:20:10.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe00f7fe640 1 -- 172.21.15.22:0/873220461 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2257907224 0 0) 0x7fe000005740 con 0x7fe0100afc20 2024-03-29T04:20:10.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe00f7fe640 1 -- 172.21.15.22:0/873220461 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe01015b0f0 msgr2=0x7fe0100b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:10.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe00f7fe640 1 --2- 172.21.15.22:0/873220461 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe01015b0f0 0x7fe0100b49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe00f7fe640 1 -- 172.21.15.22:0/873220461 >> v2:172.21.15.151:3300/0 conn(0x7fe0100cf380 msgr2=0x7fe0100b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:10.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe00f7fe640 1 --2- 172.21.15.22:0/873220461 >> v2:172.21.15.151:3300/0 conn(0x7fe0100cf380 0x7fe0100b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe00f7fe640 1 -- 172.21.15.22:0/873220461 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01006ec20 con 0x7fe0100afc20 2024-03-29T04:20:10.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe00f7fe640 1 -- 172.21.15.22:0/873220461 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7fdffc001680 con 0x7fe0100afc20 2024-03-29T04:20:10.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe00f7fe640 1 -- 172.21.15.22:0/873220461 >> v1:172.21.15.22:6789/0 conn(0x7fe0100afc20 legacy=0x7fe0100b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:10.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe00f7fe640 1 --2- 172.21.15.22:0/873220461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe000005bd0 0x7fe000005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:10.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe00f7fe640 1 --2- 172.21.15.22:0/873220461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe000006a20 0x7fe000006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:10.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe00f7fe640 1 --2- 172.21.15.22:0/873220461 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe01015b0f0 0x7fe0000073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:10.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe00f7fe640 1 -- 172.21.15.22:0/873220461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe01006e660 con 0x7fe000005bd0 2024-03-29T04:20:10.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe00f7fe640 1 -- 172.21.15.22:0/873220461 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe01006e940 con 0x7fe000006a20 2024-03-29T04:20:10.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe00f7fe640 1 -- 172.21.15.22:0/873220461 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe00000dbe0 con 0x7fe01015b0f0 2024-03-29T04:20:10.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe014a86640 1 --2- 172.21.15.22:0/873220461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe000005bd0 0x7fe000005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:10.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe00ffff640 1 --2- 172.21.15.22:0/873220461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe000006a20 0x7fe000006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:10.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe015287640 1 --2- 172.21.15.22:0/873220461 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe01015b0f0 0x7fe0000073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:10.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe014a86640 1 -- 172.21.15.22:0/873220461 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe01015b0f0 msgr2=0x7fe0000073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:10.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe014a86640 1 --2- 172.21.15.22:0/873220461 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe01015b0f0 0x7fe0000073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe014a86640 1 -- 172.21.15.22:0/873220461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe000006a20 msgr2=0x7fe000006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:10.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe014a86640 1 --2- 172.21.15.22:0/873220461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe000006a20 0x7fe000006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.875+0000 7fe014a86640 1 -- 172.21.15.22:0/873220461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe000005740 con 0x7fe000005bd0 2024-03-29T04:20:10.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.879+0000 7fe014a86640 1 --2- 172.21.15.22:0/873220461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe000005bd0 0x7fe000005ff0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe004009ae0 tx=0x7fe00401ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5293b5c474b7739a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:10.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.879+0000 7fe00f7fe640 1 -- 172.21.15.22:0/873220461 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0040096b0 con 0x7fe000005bd0 2024-03-29T04:20:10.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.879+0000 7fe00f7fe640 1 -- 172.21.15.22:0/873220461 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0040a6450 con 0x7fe000005bd0 2024-03-29T04:20:10.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.879+0000 7fe00f7fe640 1 -- 172.21.15.22:0/873220461 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0040a6840 con 0x7fe000005bd0 2024-03-29T04:20:10.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.879+0000 7fe016cea640 1 -- 172.21.15.22:0/873220461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe000005bd0 msgr2=0x7fe000005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:10.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.879+0000 7fe016cea640 1 --2- 172.21.15.22:0/873220461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe000005bd0 0x7fe000005ff0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe004009ae0 tx=0x7fe00401ca40 comp rx=0 tx=0).stop 2024-03-29T04:20:10.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.879+0000 7fe015287640 1 -- 172.21.15.22:0/873220461 reap_dead start 2024-03-29T04:20:10.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.879+0000 7fe016cea640 1 -- 172.21.15.22:0/873220461 shutdown_connections 2024-03-29T04:20:10.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.879+0000 7fe016cea640 1 -- 172.21.15.22:0/873220461 >> 172.21.15.22:0/873220461 conn(0x7fe0100b2980 msgr2=0x7fe0100d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:10.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.879+0000 7fe016cea640 1 -- 172.21.15.22:0/873220461 shutdown_connections 2024-03-29T04:20:10.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.879+0000 7fe016cea640 1 -- 172.21.15.22:0/873220461 wait complete. 2024-03-29T04:20:10.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.879+0000 7fe016cea640 1 Processor -- start 2024-03-29T04:20:10.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe016cea640 1 -- start start 2024-03-29T04:20:10.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe016cea640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe000005bd0 0x7fe0101f29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:10.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe016cea640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe010154540 0x7fe010152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:10.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe016cea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe010152420 0x7fe010155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:10.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe016cea640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe01006b660 con 0x7fe010152420 2024-03-29T04:20:10.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe016cea640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe01015b4d0 con 0x7fe010154540 2024-03-29T04:20:10.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe016cea640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe01006b7e0 con 0x7fe000005bd0 2024-03-29T04:20:10.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe015287640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe010152420 0x7fe010155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:10.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe015287640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe010152420 0x7fe010155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38748/0 (socket says 172.21.15.22:38748) 2024-03-29T04:20:10.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe015287640 1 -- 172.21.15.22:0/579798687 learned_addr learned my addr 172.21.15.22:0/579798687 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:10.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe014a86640 1 --2- 172.21.15.22:0/579798687 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe000005bd0 0x7fe0101f29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:10.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe00ffff640 1 --2- 172.21.15.22:0/579798687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe010154540 0x7fe010152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:10.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe015287640 1 -- 172.21.15.22:0/579798687 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe000005bd0 msgr2=0x7fe0101f29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:10.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe015287640 1 --2- 172.21.15.22:0/579798687 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe000005bd0 0x7fe0101f29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe015287640 1 -- 172.21.15.22:0/579798687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe010154540 msgr2=0x7fe010152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:10.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe015287640 1 --2- 172.21.15.22:0/579798687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe010154540 0x7fe010152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:10.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe015287640 1 -- 172.21.15.22:0/579798687 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0101534a0 con 0x7fe010152420 2024-03-29T04:20:10.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe015287640 1 --2- 172.21.15.22:0/579798687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe010152420 0x7fe010155050 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe0100d11a0 tx=0x7fe0080115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:10.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe00d7fa640 1 -- 172.21.15.22:0/579798687 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0080039a0 con 0x7fe010152420 2024-03-29T04:20:10.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe00d7fa640 1 -- 172.21.15.22:0/579798687 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe00801b070 con 0x7fe010152420 2024-03-29T04:20:10.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe016cea640 1 -- 172.21.15.22:0/579798687 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe010153780 con 0x7fe010152420 2024-03-29T04:20:10.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe00d7fa640 1 -- 172.21.15.22:0/579798687 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008003b80 con 0x7fe010152420 2024-03-29T04:20:10.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe016cea640 1 -- 172.21.15.22:0/579798687 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe010153d40 con 0x7fe010152420 2024-03-29T04:20:10.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe016cea640 1 -- 172.21.15.22:0/579798687 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe010068890 con 0x7fe010152420 2024-03-29T04:20:10.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe00d7fa640 1 -- 172.21.15.22:0/579798687 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe008004030 con 0x7fe010152420 2024-03-29T04:20:10.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe00d7fa640 1 --2- 172.21.15.22:0/579798687 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdff4067fc0 0x7fdff406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:10.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.883+0000 7fe00d7fa640 1 -- 172.21.15.22:0/579798687 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fe008086eb0 con 0x7fe010152420 2024-03-29T04:20:10.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.887+0000 7fe014a86640 1 --2- 172.21.15.22:0/579798687 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdff4067fc0 0x7fdff406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:10.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.887+0000 7fe014a86640 1 --2- 172.21.15.22:0/579798687 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdff4067fc0 0x7fdff406a480 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe0040ad000 tx=0x7fe0040046d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:10.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:10.891+0000 7fe00d7fa640 1 -- 172.21.15.22:0/579798687 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fe008057540 con 0x7fe010152420 2024-03-29T04:20:11.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.091+0000 7fe016cea640 1 -- 172.21.15.22:0/579798687 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe010153190 con 0x7fe010152420 2024-03-29T04:20:11.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.095+0000 7fe00d7fa640 1 -- 172.21.15.22:0/579798687 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5181 (secure 0 0 0) 0x7fe008056e40 con 0x7fe010152420 2024-03-29T04:20:11.075 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:20:11.075 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/3112156479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":3112156479},{"type":"v1","addr":"172.21.15.151:6835","nonce":3112156479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5308,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/491273202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":491273202},{"type":"v1","addr":"172.21.15.151:6837","nonce":491273202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/434199693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":434199693},{"type":"v1","addr":"172.21.15.22:6833","nonce":434199693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:18:57.990202+0000","modified":"2024-03-29T04:19:24.799606+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6835/2966923505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2966923505},{"type":"v1","addr":"172.21.15.22:6835","nonce":2966923505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529]},"id":2}]} 2024-03-29T04:20:11.076 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-03-29T04:20:11.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.095+0000 7fe016cea640 1 -- 172.21.15.22:0/579798687 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdff4067fc0 msgr2=0x7fdff406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:11.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.095+0000 7fe016cea640 1 --2- 172.21.15.22:0/579798687 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdff4067fc0 0x7fdff406a480 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe0040ad000 tx=0x7fe0040046d0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.095+0000 7fe016cea640 1 -- 172.21.15.22:0/579798687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe010152420 msgr2=0x7fe010155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:11.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.095+0000 7fe016cea640 1 --2- 172.21.15.22:0/579798687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe010152420 0x7fe010155050 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe0100d11a0 tx=0x7fe0080115a0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.099+0000 7fe016cea640 1 -- 172.21.15.22:0/579798687 shutdown_connections 2024-03-29T04:20:11.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.099+0000 7fe016cea640 1 --2- 172.21.15.22:0/579798687 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fdff4067fc0 0x7fdff406a480 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.099+0000 7fe016cea640 1 --2- 172.21.15.22:0/579798687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe010152420 0x7fe010155050 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.099+0000 7fe016cea640 1 --2- 172.21.15.22:0/579798687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe010154540 0x7fe010152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.099+0000 7fe016cea640 1 --2- 172.21.15.22:0/579798687 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe000005bd0 0x7fe0101f29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.099+0000 7fe016cea640 1 -- 172.21.15.22:0/579798687 >> 172.21.15.22:0/579798687 conn(0x7fe0100b2980 msgr2=0x7fe0100d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:11.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.099+0000 7fe016cea640 1 -- 172.21.15.22:0/579798687 shutdown_connections 2024-03-29T04:20:11.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.099+0000 7fe016cea640 1 -- 172.21.15.22:0/579798687 wait complete. 2024-03-29T04:20:11.096 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:20:11.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f7648cd0640 1 Processor -- start 2024-03-29T04:20:11.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f7648cd0640 1 -- start start 2024-03-29T04:20:11.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f7648cd0640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f76440afd50 0x7f76440b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f7648cd0640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f76440b4870 0x7f764415b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f7648cd0640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76440b3cd0 con 0x7f764415f030 2024-03-29T04:20:11.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f7648cd0640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f76440b0130 con 0x7f76440afd50 2024-03-29T04:20:11.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f7648cd0640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f764415e550 con 0x7f76440b4870 2024-03-29T04:20:11.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f7641d9b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f764415f030 0x7f76440b4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:45594/0 (socket says 172.21.15.22:45594) 2024-03-29T04:20:11.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f7641d9b640 1 -- 172.21.15.22:0/2546440960 learned_addr learned my addr 172.21.15.22:0/2546440960 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:11.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f7642d9d640 1 --2- 172.21.15.22:0/2546440960 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f76440b4870 0x7f764415b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f764259c640 1 --2- 172.21.15.22:0/2546440960 >> v2:172.21.15.151:3300/0 conn(0x7f76440afd50 0x7f76440b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f764159a640 1 -- 172.21.15.22:0/2546440960 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2443821781 0 0) 0x7f76440b3cd0 con 0x7f764415f030 2024-03-29T04:20:11.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f764159a640 1 -- 172.21.15.22:0/2546440960 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7630003530 con 0x7f764415f030 2024-03-29T04:20:11.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f764159a640 1 -- 172.21.15.22:0/2546440960 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1622011720 0 0) 0x7f7630003530 con 0x7f764415f030 2024-03-29T04:20:11.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f764159a640 1 -- 172.21.15.22:0/2546440960 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f76440b4870 msgr2=0x7f764415b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:11.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f764159a640 1 --2- 172.21.15.22:0/2546440960 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f76440b4870 0x7f764415b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f764159a640 1 -- 172.21.15.22:0/2546440960 >> v2:172.21.15.151:3300/0 conn(0x7f76440afd50 msgr2=0x7f76440b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:11.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f764159a640 1 --2- 172.21.15.22:0/2546440960 >> v2:172.21.15.151:3300/0 conn(0x7f76440afd50 0x7f76440b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f764159a640 1 -- 172.21.15.22:0/2546440960 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764415e830 con 0x7f764415f030 2024-03-29T04:20:11.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f764159a640 1 -- 172.21.15.22:0/2546440960 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f762c001680 con 0x7f764415f030 2024-03-29T04:20:11.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f764159a640 1 -- 172.21.15.22:0/2546440960 >> v1:172.21.15.22:6789/0 conn(0x7f764415f030 legacy=0x7f76440b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:11.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f764159a640 1 --2- 172.21.15.22:0/2546440960 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f76440b4870 0x7f76300039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f764159a640 1 --2- 172.21.15.22:0/2546440960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7630003f40 0x7f7630004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f764159a640 1 --2- 172.21.15.22:0/2546440960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7630004d50 0x7f76300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f764159a640 1 -- 172.21.15.22:0/2546440960 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f76440b0130 con 0x7f7630003f40 2024-03-29T04:20:11.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f764159a640 1 -- 172.21.15.22:0/2546440960 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f764415e550 con 0x7f7630004d50 2024-03-29T04:20:11.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f764159a640 1 -- 172.21.15.22:0/2546440960 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f763000fde0 con 0x7f76440b4870 2024-03-29T04:20:11.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f7641d9b640 1 --2- 172.21.15.22:0/2546440960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7630003f40 0x7f7630004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f764259c640 1 --2- 172.21.15.22:0/2546440960 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f76440b4870 0x7f76300039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f7642d9d640 1 --2- 172.21.15.22:0/2546440960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7630004d50 0x7f76300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f7641d9b640 1 -- 172.21.15.22:0/2546440960 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f76440b4870 msgr2=0x7f76300039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:11.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f7641d9b640 1 --2- 172.21.15.22:0/2546440960 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f76440b4870 0x7f76300039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f7641d9b640 1 -- 172.21.15.22:0/2546440960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7630004d50 msgr2=0x7f76300051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:11.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f7641d9b640 1 --2- 172.21.15.22:0/2546440960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7630004d50 0x7f76300051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.243+0000 7f7641d9b640 1 -- 172.21.15.22:0/2546440960 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7630003530 con 0x7f7630003f40 2024-03-29T04:20:11.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.247+0000 7f7641d9b640 1 --2- 172.21.15.22:0/2546440960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7630003f40 0x7f7630004810 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f762c00c210 tx=0x7f762c01f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c674ce5fa361e229 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:11.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.247+0000 7f764159a640 1 -- 172.21.15.22:0/2546440960 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c007d40 con 0x7f7630003f40 2024-03-29T04:20:11.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.247+0000 7f764159a640 1 -- 172.21.15.22:0/2546440960 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f762c005b00 con 0x7f7630003f40 2024-03-29T04:20:11.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.247+0000 7f764159a640 1 -- 172.21.15.22:0/2546440960 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c007070 con 0x7f7630003f40 2024-03-29T04:20:11.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.247+0000 7f7648cd0640 1 -- 172.21.15.22:0/2546440960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7630003f40 msgr2=0x7f7630004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:11.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.247+0000 7f7648cd0640 1 --2- 172.21.15.22:0/2546440960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7630003f40 0x7f7630004810 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f762c00c210 tx=0x7f762c01f370 comp rx=0 tx=0).stop 2024-03-29T04:20:11.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.247+0000 7f7642d9d640 1 -- 172.21.15.22:0/2546440960 reap_dead start 2024-03-29T04:20:11.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.247+0000 7f7648cd0640 1 -- 172.21.15.22:0/2546440960 shutdown_connections 2024-03-29T04:20:11.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.247+0000 7f7648cd0640 1 -- 172.21.15.22:0/2546440960 >> 172.21.15.22:0/2546440960 conn(0x7f76440b2980 msgr2=0x7f76440d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:11.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.247+0000 7f7648cd0640 1 -- 172.21.15.22:0/2546440960 shutdown_connections 2024-03-29T04:20:11.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.247+0000 7f7648cd0640 1 -- 172.21.15.22:0/2546440960 wait complete. 2024-03-29T04:20:11.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f7648cd0640 1 Processor -- start 2024-03-29T04:20:11.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f7648cd0640 1 -- start start 2024-03-29T04:20:11.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f7648cd0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7630003f40 0x7f7644151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f7648cd0640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f764414e8a0 0x7f76441f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f7648cd0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76441f0600 0x7f764414fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f7648cd0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f764415b9e0 con 0x7f7630003f40 2024-03-29T04:20:11.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f7648cd0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f764415b860 con 0x7f76441f0600 2024-03-29T04:20:11.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f7648cd0640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f764415bb60 con 0x7f764414e8a0 2024-03-29T04:20:11.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f764259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7630003f40 0x7f7644151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f764259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7630003f40 0x7f7644151db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38774/0 (socket says 172.21.15.22:38774) 2024-03-29T04:20:11.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f764259c640 1 -- 172.21.15.22:0/635201370 learned_addr learned my addr 172.21.15.22:0/635201370 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:11.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f7641d9b640 1 --2- 172.21.15.22:0/635201370 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f764414e8a0 0x7f76441f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f7642d9d640 1 --2- 172.21.15.22:0/635201370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76441f0600 0x7f764414fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f764259c640 1 -- 172.21.15.22:0/635201370 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f764414e8a0 msgr2=0x7f76441f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:11.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f764259c640 1 --2- 172.21.15.22:0/635201370 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f764414e8a0 0x7f76441f33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f764259c640 1 -- 172.21.15.22:0/635201370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76441f0600 msgr2=0x7f764414fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:11.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f764259c640 1 --2- 172.21.15.22:0/635201370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76441f0600 0x7f764414fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f764259c640 1 -- 172.21.15.22:0/635201370 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76441f3c10 con 0x7f7630003f40 2024-03-29T04:20:11.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f764259c640 1 --2- 172.21.15.22:0/635201370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7630003f40 0x7f7644151db0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f763c0043a0 tx=0x7f763c004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:11.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f763affd640 1 -- 172.21.15.22:0/635201370 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c019020 con 0x7f7630003f40 2024-03-29T04:20:11.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f763affd640 1 -- 172.21.15.22:0/635201370 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f763c008d50 con 0x7f7630003f40 2024-03-29T04:20:11.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f7648cd0640 1 -- 172.21.15.22:0/635201370 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76441f3ef0 con 0x7f7630003f40 2024-03-29T04:20:11.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f763affd640 1 -- 172.21.15.22:0/635201370 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c015670 con 0x7f7630003f40 2024-03-29T04:20:11.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.251+0000 7f7648cd0640 1 -- 172.21.15.22:0/635201370 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76441f00e0 con 0x7f7630003f40 2024-03-29T04:20:11.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.255+0000 7f7648cd0640 1 -- 172.21.15.22:0/635201370 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7644068890 con 0x7f7630003f40 2024-03-29T04:20:11.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.255+0000 7f763affd640 1 -- 172.21.15.22:0/635201370 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f763c01c020 con 0x7f7630003f40 2024-03-29T04:20:11.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.255+0000 7f763affd640 1 --2- 172.21.15.22:0/635201370 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7624067ff0 0x7f762406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.255+0000 7f763affd640 1 -- 172.21.15.22:0/635201370 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f763c0872a0 con 0x7f7630003f40 2024-03-29T04:20:11.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.255+0000 7f7641d9b640 1 --2- 172.21.15.22:0/635201370 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7624067ff0 0x7f762406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.255+0000 7f7641d9b640 1 --2- 172.21.15.22:0/635201370 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7624067ff0 0x7f762406a4b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f762c0ae000 tx=0x7f76440b3ea0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:11.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.263+0000 7f763affd640 1 -- 172.21.15.22:0/635201370 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f763c023030 con 0x7f7630003f40 2024-03-29T04:20:11.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.427+0000 7f7648cd0640 1 -- 172.21.15.22:0/635201370 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7644067ef0 con 0x7f7630003f40 2024-03-29T04:20:11.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.427+0000 7f763affd640 1 -- 172.21.15.22:0/635201370 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16991 (secure 0 0 0) 0x7f763c011040 con 0x7f7630003f40 2024-03-29T04:20:11.411 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:20:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":42,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:19:17.809849+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:18:56.336876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:18:56.950670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:20:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.431+0000 7f7648cd0640 1 -- 172.21.15.22:0/635201370 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7624067ff0 msgr2=0x7f762406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.431+0000 7f7648cd0640 1 --2- 172.21.15.22:0/635201370 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7624067ff0 0x7f762406a4b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f762c0ae000 tx=0x7f76440b3ea0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.431+0000 7f7648cd0640 1 -- 172.21.15.22:0/635201370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7630003f40 msgr2=0x7f7644151db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.431+0000 7f7648cd0640 1 --2- 172.21.15.22:0/635201370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7630003f40 0x7f7644151db0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f763c0043a0 tx=0x7f763c004860 comp rx=0 tx=0).stop 2024-03-29T04:20:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.431+0000 7f7648cd0640 1 -- 172.21.15.22:0/635201370 shutdown_connections 2024-03-29T04:20:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.431+0000 7f7648cd0640 1 --2- 172.21.15.22:0/635201370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76441f0600 0x7f764414fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.431+0000 7f7648cd0640 1 --2- 172.21.15.22:0/635201370 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f764414e8a0 0x7f76441f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.431+0000 7f7648cd0640 1 --2- 172.21.15.22:0/635201370 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7624067ff0 0x7f762406a4b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.431+0000 7f7648cd0640 1 --2- 172.21.15.22:0/635201370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7630003f40 0x7f7644151db0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.431+0000 7f7648cd0640 1 -- 172.21.15.22:0/635201370 >> 172.21.15.22:0/635201370 conn(0x7f76440b2980 msgr2=0x7f764406c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.431+0000 7f7648cd0640 1 -- 172.21.15.22:0/635201370 shutdown_connections 2024-03-29T04:20:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.431+0000 7f7648cd0640 1 -- 172.21.15.22:0/635201370 wait complete. 2024-03-29T04:20:11.430 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T04:20:11.430 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:20:11.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13de213640 1 Processor -- start 2024-03-29T04:20:11.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13de213640 1 -- start start 2024-03-29T04:20:11.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13de213640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f13d813f0a0 0x7f13d8160470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13de213640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13d8141970 con 0x7f13d806de90 2024-03-29T04:20:11.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13de213640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13d8141c50 con 0x7f13d81609b0 2024-03-29T04:20:11.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13de213640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f13d8141f30 con 0x7f13d813f0a0 2024-03-29T04:20:11.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d7fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f13d806de90 0x7f13d80cfa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:45608/0 (socket says 172.21.15.22:45608) 2024-03-29T04:20:11.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d7fff640 1 -- 172.21.15.22:0/142303846 learned_addr learned my addr 172.21.15.22:0/142303846 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:11.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d77fe640 1 --2- 172.21.15.22:0/142303846 >> v2:172.21.15.151:3300/0 conn(0x7f13d813f0a0 0x7f13d8160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d67fc640 1 -- 172.21.15.22:0/142303846 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2991629179 0 0) 0x7f13d8141970 con 0x7f13d806de90 2024-03-29T04:20:11.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d67fc640 1 -- 172.21.15.22:0/142303846 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13c0003530 con 0x7f13d806de90 2024-03-29T04:20:11.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d67fc640 1 -- 172.21.15.22:0/142303846 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1465309618 0 0) 0x7f13c0003530 con 0x7f13d806de90 2024-03-29T04:20:11.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d67fc640 1 -- 172.21.15.22:0/142303846 >> v2:172.21.15.151:3300/0 conn(0x7f13d813f0a0 msgr2=0x7f13d8160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:11.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d67fc640 1 --2- 172.21.15.22:0/142303846 >> v2:172.21.15.151:3300/0 conn(0x7f13d813f0a0 0x7f13d8160470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d67fc640 1 -- 172.21.15.22:0/142303846 >> v1:172.21.15.151:6789/0 conn(0x7f13d81609b0 legacy=0x7f13d806d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:11.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d67fc640 1 -- 172.21.15.22:0/142303846 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d8142210 con 0x7f13d806de90 2024-03-29T04:20:11.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d67fc640 1 -- 172.21.15.22:0/142303846 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f13d0001a00 con 0x7f13d806de90 2024-03-29T04:20:11.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d67fc640 1 -- 172.21.15.22:0/142303846 >> v1:172.21.15.22:6789/0 conn(0x7f13d806de90 legacy=0x7f13d80cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:11.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d67fc640 1 --2- 172.21.15.22:0/142303846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13c00039c0 0x7f13c0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d67fc640 1 --2- 172.21.15.22:0/142303846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13c0004810 0x7f13c0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d67fc640 1 --2- 172.21.15.22:0/142303846 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f13c00051d0 0x7f13c000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d67fc640 1 -- 172.21.15.22:0/142303846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f13d8141f30 con 0x7f13c00039c0 2024-03-29T04:20:11.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d67fc640 1 -- 172.21.15.22:0/142303846 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13c000c070 con 0x7f13c0004810 2024-03-29T04:20:11.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d67fc640 1 -- 172.21.15.22:0/142303846 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f13c000c270 con 0x7f13c00051d0 2024-03-29T04:20:11.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d77fe640 1 --2- 172.21.15.22:0/142303846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13c00039c0 0x7f13c0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d6ffd640 1 --2- 172.21.15.22:0/142303846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13c0004810 0x7f13c0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d77fe640 1 -- 172.21.15.22:0/142303846 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f13c00051d0 msgr2=0x7f13c000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:11.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d7fff640 1 --2- 172.21.15.22:0/142303846 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f13c00051d0 0x7f13c000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d77fe640 1 --2- 172.21.15.22:0/142303846 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f13c00051d0 0x7f13c000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d77fe640 1 -- 172.21.15.22:0/142303846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13c0004810 msgr2=0x7f13c0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:11.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d77fe640 1 --2- 172.21.15.22:0/142303846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13c0004810 0x7f13c0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d77fe640 1 -- 172.21.15.22:0/142303846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c0003530 con 0x7f13c00039c0 2024-03-29T04:20:11.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.567+0000 7f13d7fff640 1 -- 172.21.15.22:0/142303846 reap_dead start 2024-03-29T04:20:11.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.571+0000 7f13d77fe640 1 --2- 172.21.15.22:0/142303846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13c00039c0 0x7f13c0003de0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009ae0 tx=0x7f13cc01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7359acb73b4f921c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:11.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.571+0000 7f13d67fc640 1 -- 172.21.15.22:0/142303846 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc0b2070 con 0x7f13c00039c0 2024-03-29T04:20:11.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.571+0000 7f13d67fc640 1 -- 172.21.15.22:0/142303846 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13cc0a55b0 con 0x7f13c00039c0 2024-03-29T04:20:11.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.571+0000 7f13d67fc640 1 -- 172.21.15.22:0/142303846 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc0096b0 con 0x7f13c00039c0 2024-03-29T04:20:11.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.571+0000 7f13de213640 1 -- 172.21.15.22:0/142303846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13c00039c0 msgr2=0x7f13c0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:11.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.571+0000 7f13de213640 1 --2- 172.21.15.22:0/142303846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13c00039c0 0x7f13c0003de0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009ae0 tx=0x7f13cc01ca40 comp rx=0 tx=0).stop 2024-03-29T04:20:11.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.571+0000 7f13de213640 1 -- 172.21.15.22:0/142303846 shutdown_connections 2024-03-29T04:20:11.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.571+0000 7f13de213640 1 --2- 172.21.15.22:0/142303846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13c00039c0 0x7f13c0003de0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.571+0000 7f13de213640 1 -- 172.21.15.22:0/142303846 >> 172.21.15.22:0/142303846 conn(0x7f13d8072500 msgr2=0x7f13d8072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:11.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.571+0000 7f13de213640 1 -- 172.21.15.22:0/142303846 shutdown_connections 2024-03-29T04:20:11.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.571+0000 7f13de213640 1 -- 172.21.15.22:0/142303846 wait complete. 2024-03-29T04:20:11.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.571+0000 7f13de213640 1 Processor -- start 2024-03-29T04:20:11.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.571+0000 7f13de213640 1 -- start start 2024-03-29T04:20:11.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.571+0000 7f13de213640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13c00039c0 0x7f13d8138820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.571+0000 7f13de213640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f13d806f7d0 0x7f13d806fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.571+0000 7f13de213640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d81e5e20 0x7f13d81e62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.571+0000 7f13de213640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f13d815ff20 con 0x7f13d81e5e20 2024-03-29T04:20:11.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.571+0000 7f13de213640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13d80b5fa0 con 0x7f13c00039c0 2024-03-29T04:20:11.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.571+0000 7f13de213640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f13d81600a0 con 0x7f13d806f7d0 2024-03-29T04:20:11.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.575+0000 7f13d7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d81e5e20 0x7f13d81e62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.575+0000 7f13d7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d81e5e20 0x7f13d81e62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38788/0 (socket says 172.21.15.22:38788) 2024-03-29T04:20:11.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.575+0000 7f13d7fff640 1 -- 172.21.15.22:0/1778004542 learned_addr learned my addr 172.21.15.22:0/1778004542 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:11.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.575+0000 7f13d77fe640 1 --2- 172.21.15.22:0/1778004542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13c00039c0 0x7f13d8138820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.575+0000 7f13d6ffd640 1 --2- 172.21.15.22:0/1778004542 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f13d806f7d0 0x7f13d806fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.575+0000 7f13d7fff640 1 -- 172.21.15.22:0/1778004542 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f13d806f7d0 msgr2=0x7f13d806fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:11.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.575+0000 7f13d7fff640 1 --2- 172.21.15.22:0/1778004542 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f13d806f7d0 0x7f13d806fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.575+0000 7f13d7fff640 1 -- 172.21.15.22:0/1778004542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13c00039c0 msgr2=0x7f13d8138820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:11.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.575+0000 7f13d7fff640 1 --2- 172.21.15.22:0/1778004542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13c00039c0 0x7f13d8138820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.575+0000 7f13d7fff640 1 -- 172.21.15.22:0/1778004542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d8138d60 con 0x7f13d81e5e20 2024-03-29T04:20:11.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.575+0000 7f13d7fff640 1 --2- 172.21.15.22:0/1778004542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d81e5e20 0x7f13d81e62d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f13d80710b0 tx=0x7f13d000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:11.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.575+0000 7f13bffff640 1 -- 172.21.15.22:0/1778004542 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0004d70 con 0x7f13d81e5e20 2024-03-29T04:20:11.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.575+0000 7f13bffff640 1 -- 172.21.15.22:0/1778004542 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d000bb20 con 0x7f13d81e5e20 2024-03-29T04:20:11.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.575+0000 7f13de213640 1 -- 172.21.15.22:0/1778004542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d81e6810 con 0x7f13d81e5e20 2024-03-29T04:20:11.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.575+0000 7f13bffff640 1 -- 172.21.15.22:0/1778004542 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0007110 con 0x7f13d81e5e20 2024-03-29T04:20:11.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.575+0000 7f13de213640 1 -- 172.21.15.22:0/1778004542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d81f47a0 con 0x7f13d81e5e20 2024-03-29T04:20:11.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.575+0000 7f13de213640 1 -- 172.21.15.22:0/1778004542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13d8068890 con 0x7f13d81e5e20 2024-03-29T04:20:11.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.575+0000 7f13bffff640 1 -- 172.21.15.22:0/1778004542 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f13d001b020 con 0x7f13d81e5e20 2024-03-29T04:20:11.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.575+0000 7f13bffff640 1 --2- 172.21.15.22:0/1778004542 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f13c4067ff0 0x7f13c406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.575+0000 7f13bffff640 1 -- 172.21.15.22:0/1778004542 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f13d0086670 con 0x7f13d81e5e20 2024-03-29T04:20:11.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.575+0000 7f13d77fe640 1 --2- 172.21.15.22:0/1778004542 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f13c4067ff0 0x7f13c406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.579+0000 7f13d77fe640 1 --2- 172.21.15.22:0/1778004542 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f13c4067ff0 0x7f13c406a4b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0af000 tx=0x7f13cc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:11.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.579+0000 7f13bffff640 1 -- 172.21.15.22:0/1778004542 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f13d0056c60 con 0x7f13d81e5e20 2024-03-29T04:20:11.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.775+0000 7f13de213640 1 -- 172.21.15.22:0/1778004542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13d806c010 con 0x7f13d81e5e20 2024-03-29T04:20:11.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.775+0000 7f13bffff640 1 -- 172.21.15.22:0/1778004542 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5181 (secure 0 0 0) 0x7f13d0056560 con 0x7f13d81e5e20 2024-03-29T04:20:11.757 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:20:11.757 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/3112156479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":3112156479},{"type":"v1","addr":"172.21.15.151:6835","nonce":3112156479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5308,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/491273202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":491273202},{"type":"v1","addr":"172.21.15.151:6837","nonce":491273202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/434199693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":434199693},{"type":"v1","addr":"172.21.15.22:6833","nonce":434199693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:18:57.990202+0000","modified":"2024-03-29T04:19:24.799606+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6835/2966923505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2966923505},{"type":"v1","addr":"172.21.15.22:6835","nonce":2966923505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529]},"id":2}]} 2024-03-29T04:20:11.758 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-03-29T04:20:11.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.779+0000 7f13de213640 1 -- 172.21.15.22:0/1778004542 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f13c4067ff0 msgr2=0x7f13c406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:11.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.779+0000 7f13de213640 1 --2- 172.21.15.22:0/1778004542 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f13c4067ff0 0x7f13c406a4b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0af000 tx=0x7f13cc008040 comp rx=0 tx=0).stop 2024-03-29T04:20:11.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.779+0000 7f13de213640 1 -- 172.21.15.22:0/1778004542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d81e5e20 msgr2=0x7f13d81e62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:11.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.779+0000 7f13de213640 1 --2- 172.21.15.22:0/1778004542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d81e5e20 0x7f13d81e62d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f13d80710b0 tx=0x7f13d000b4a0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.779+0000 7f13de213640 1 -- 172.21.15.22:0/1778004542 shutdown_connections 2024-03-29T04:20:11.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.779+0000 7f13de213640 1 --2- 172.21.15.22:0/1778004542 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f13c4067ff0 0x7f13c406a4b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.779+0000 7f13de213640 1 --2- 172.21.15.22:0/1778004542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d81e5e20 0x7f13d81e62d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.779+0000 7f13de213640 1 --2- 172.21.15.22:0/1778004542 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f13d806f7d0 0x7f13d806fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.779+0000 7f13de213640 1 --2- 172.21.15.22:0/1778004542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13c00039c0 0x7f13d8138820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.779+0000 7f13de213640 1 -- 172.21.15.22:0/1778004542 >> 172.21.15.22:0/1778004542 conn(0x7f13d8072500 msgr2=0x7f13d80b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:11.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.779+0000 7f13de213640 1 -- 172.21.15.22:0/1778004542 shutdown_connections 2024-03-29T04:20:11.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.779+0000 7f13de213640 1 -- 172.21.15.22:0/1778004542 wait complete. 2024-03-29T04:20:11.778 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:20:11.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.923+0000 7f7486037640 1 Processor -- start 2024-03-29T04:20:11.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.923+0000 7f7486037640 1 -- start start 2024-03-29T04:20:11.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.927+0000 7f7486037640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f748015b0f0 0x7f74800b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.927+0000 7f7486037640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f74800afc20 0x7f74800b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.927+0000 7f7486037640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74800b5760 con 0x7f748006c770 2024-03-29T04:20:11.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.927+0000 7f7486037640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f748006e660 con 0x7f74800afc20 2024-03-29T04:20:11.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.927+0000 7f7486037640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f748006e940 con 0x7f748015b0f0 2024-03-29T04:20:11.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.927+0000 7f747effd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f74800afc20 0x7f74800b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.927+0000 7f747effd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f74800afc20 0x7f74800b4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38802/0 (socket says 172.21.15.22:38802) 2024-03-29T04:20:11.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.927+0000 7f747effd640 1 -- 172.21.15.22:0/556835644 learned_addr learned my addr 172.21.15.22:0/556835644 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:11.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.927+0000 7f747f7fe640 1 --2- 172.21.15.22:0/556835644 >> v2:172.21.15.151:3300/0 conn(0x7f748015b0f0 0x7f74800b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.927+0000 7f747effd640 1 -- 172.21.15.22:0/556835644 >> v2:172.21.15.151:3300/0 conn(0x7f748015b0f0 msgr2=0x7f74800b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:11.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.927+0000 7f747effd640 1 --2- 172.21.15.22:0/556835644 >> v2:172.21.15.151:3300/0 conn(0x7f748015b0f0 0x7f74800b49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.927+0000 7f747effd640 1 -- 172.21.15.22:0/556835644 >> v1:172.21.15.151:6789/0 conn(0x7f748006c770 legacy=0x7f74800b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:11.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.927+0000 7f747effd640 1 -- 172.21.15.22:0/556835644 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748006ec20 con 0x7f74800afc20 2024-03-29T04:20:11.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.927+0000 7f747f7fe640 1 --2- 172.21.15.22:0/556835644 >> v2:172.21.15.151:3300/0 conn(0x7f748015b0f0 0x7f74800b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:11.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.927+0000 7f747effd640 1 --2- 172.21.15.22:0/556835644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74800afc20 0x7f74800b4ee0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f74800b2f70 tx=0x7f747401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e918da9b19abec server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:11.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.927+0000 7f747e7fc640 1 -- 172.21.15.22:0/556835644 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74740aa610 con 0x7f74800afc20 2024-03-29T04:20:11.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.927+0000 7f747e7fc640 1 -- 172.21.15.22:0/556835644 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74740af070 con 0x7f74800afc20 2024-03-29T04:20:11.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.927+0000 7f747e7fc640 1 -- 172.21.15.22:0/556835644 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74740aa9d0 con 0x7f74800afc20 2024-03-29T04:20:11.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.931+0000 7f7486037640 1 -- 172.21.15.22:0/556835644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74800afc20 msgr2=0x7f74800b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:11.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.931+0000 7f7486037640 1 --2- 172.21.15.22:0/556835644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74800afc20 0x7f74800b4ee0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f74800b2f70 tx=0x7f747401c4b0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.931+0000 7f7486037640 1 -- 172.21.15.22:0/556835644 shutdown_connections 2024-03-29T04:20:11.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.931+0000 7f7486037640 1 --2- 172.21.15.22:0/556835644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74800afc20 0x7f74800b4ee0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.931+0000 7f7486037640 1 --2- 172.21.15.22:0/556835644 >> v2:172.21.15.151:3300/0 conn(0x7f748015b0f0 0x7f74800b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.931+0000 7f7486037640 1 -- 172.21.15.22:0/556835644 >> 172.21.15.22:0/556835644 conn(0x7f74800b2980 msgr2=0x7f74800d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:11.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.931+0000 7f7486037640 1 -- 172.21.15.22:0/556835644 shutdown_connections 2024-03-29T04:20:11.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.931+0000 7f7486037640 1 -- 172.21.15.22:0/556835644 wait complete. 2024-03-29T04:20:11.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.931+0000 7f7486037640 1 Processor -- start 2024-03-29T04:20:11.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.931+0000 7f7486037640 1 -- start start 2024-03-29T04:20:11.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f7486037640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f748006c770 0x7f7480158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f7486037640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74800afc20 0x7f748013df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f7486037640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f748015b0f0 0x7f74800cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f7486037640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7480167d40 con 0x7f748006c770 2024-03-29T04:20:11.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f7486037640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f748015b4d0 con 0x7f74800afc20 2024-03-29T04:20:11.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f7486037640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7480167ec0 con 0x7f748015b0f0 2024-03-29T04:20:11.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f747f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f748006c770 0x7f7480158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f747f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f748006c770 0x7f7480158550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38808/0 (socket says 172.21.15.22:38808) 2024-03-29T04:20:11.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f747f7fe640 1 -- 172.21.15.22:0/1466920689 learned_addr learned my addr 172.21.15.22:0/1466920689 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:11.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f747effd640 1 --2- 172.21.15.22:0/1466920689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74800afc20 0x7f748013df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f747ffff640 1 --2- 172.21.15.22:0/1466920689 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f748015b0f0 0x7f74800cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f747f7fe640 1 -- 172.21.15.22:0/1466920689 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f748015b0f0 msgr2=0x7f74800cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:11.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f747f7fe640 1 --2- 172.21.15.22:0/1466920689 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f748015b0f0 0x7f74800cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f747f7fe640 1 -- 172.21.15.22:0/1466920689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74800afc20 msgr2=0x7f748013df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:11.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f747f7fe640 1 --2- 172.21.15.22:0/1466920689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74800afc20 0x7f748013df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:11.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f747f7fe640 1 -- 172.21.15.22:0/1466920689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74800ca9a0 con 0x7f748006c770 2024-03-29T04:20:11.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f747ffff640 1 --2- 172.21.15.22:0/1466920689 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f748015b0f0 0x7f74800cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:11.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f747f7fe640 1 --2- 172.21.15.22:0/1466920689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f748006c770 0x7f7480158550 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f747800c7f0 tx=0x7f747800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f7467fff640 1 -- 172.21.15.22:0/1466920689 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478015070 con 0x7f748006c770 2024-03-29T04:20:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f7486037640 1 -- 172.21.15.22:0/1466920689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74800cac80 con 0x7f748006c770 2024-03-29T04:20:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f7467fff640 1 -- 172.21.15.22:0/1466920689 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7478007820 con 0x7f748006c770 2024-03-29T04:20:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f7467fff640 1 -- 172.21.15.22:0/1466920689 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478007c10 con 0x7f748006c770 2024-03-29T04:20:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f7486037640 1 -- 172.21.15.22:0/1466920689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74800cb240 con 0x7f748006c770 2024-03-29T04:20:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f7486037640 1 -- 172.21.15.22:0/1466920689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7480068890 con 0x7f748006c770 2024-03-29T04:20:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f7467fff640 1 -- 172.21.15.22:0/1466920689 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f7478018070 con 0x7f748006c770 2024-03-29T04:20:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.935+0000 7f7467fff640 1 --2- 172.21.15.22:0/1466920689 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f746c067fc0 0x7f746c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.939+0000 7f7467fff640 1 -- 172.21.15.22:0/1466920689 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f74780868a0 con 0x7f748006c770 2024-03-29T04:20:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.939+0000 7f747effd640 1 --2- 172.21.15.22:0/1466920689 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f746c067fc0 0x7f746c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.939+0000 7f747effd640 1 --2- 172.21.15.22:0/1466920689 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f746c067fc0 0x7f746c06a480 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f74740ae000 tx=0x7f74740a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:11.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:11.943+0000 7f7467fff640 1 -- 172.21.15.22:0/1466920689 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f7478056e20 con 0x7f748006c770 2024-03-29T04:20:12.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.143+0000 7f7486037640 1 -- 172.21.15.22:0/1466920689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74800ccdb0 con 0x7f748006c770 2024-03-29T04:20:12.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.143+0000 7f7467fff640 1 -- 172.21.15.22:0/1466920689 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5181 (secure 0 0 0) 0x7f7478056720 con 0x7f748006c770 2024-03-29T04:20:12.124 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:20:12.124 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/3112156479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":3112156479},{"type":"v1","addr":"172.21.15.151:6835","nonce":3112156479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5308,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/491273202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":491273202},{"type":"v1","addr":"172.21.15.151:6837","nonce":491273202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/434199693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":434199693},{"type":"v1","addr":"172.21.15.22:6833","nonce":434199693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:18:57.990202+0000","modified":"2024-03-29T04:19:24.799606+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6835/2966923505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2966923505},{"type":"v1","addr":"172.21.15.22:6835","nonce":2966923505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529]},"id":2}]} 2024-03-29T04:20:12.124 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-03-29T04:20:12.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.143+0000 7f7486037640 1 -- 172.21.15.22:0/1466920689 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f746c067fc0 msgr2=0x7f746c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:12.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.143+0000 7f7486037640 1 --2- 172.21.15.22:0/1466920689 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f746c067fc0 0x7f746c06a480 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f74740ae000 tx=0x7f74740a7000 comp rx=0 tx=0).stop 2024-03-29T04:20:12.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.143+0000 7f7486037640 1 -- 172.21.15.22:0/1466920689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f748006c770 msgr2=0x7f7480158550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:12.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.143+0000 7f7486037640 1 --2- 172.21.15.22:0/1466920689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f748006c770 0x7f7480158550 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f747800c7f0 tx=0x7f747800ccb0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.143+0000 7f7486037640 1 -- 172.21.15.22:0/1466920689 shutdown_connections 2024-03-29T04:20:12.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.143+0000 7f7486037640 1 --2- 172.21.15.22:0/1466920689 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f748015b0f0 0x7f74800cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.143+0000 7f7486037640 1 --2- 172.21.15.22:0/1466920689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74800afc20 0x7f748013df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.143+0000 7f7486037640 1 --2- 172.21.15.22:0/1466920689 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f746c067fc0 0x7f746c06a480 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.143+0000 7f7486037640 1 --2- 172.21.15.22:0/1466920689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f748006c770 0x7f7480158550 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.143+0000 7f7486037640 1 -- 172.21.15.22:0/1466920689 >> 172.21.15.22:0/1466920689 conn(0x7f74800b2980 msgr2=0x7f74800ce930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:12.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.143+0000 7f7486037640 1 -- 172.21.15.22:0/1466920689 shutdown_connections 2024-03-29T04:20:12.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.143+0000 7f7486037640 1 -- 172.21.15.22:0/1466920689 wait complete. 2024-03-29T04:20:12.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:20:12.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.295+0000 7f3e7adba640 1 Processor -- start 2024-03-29T04:20:12.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.295+0000 7f3e7adba640 1 -- start start 2024-03-29T04:20:12.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.295+0000 7f3e7adba640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3e740afe30 0x7f3e7415ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:12.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.295+0000 7f3e7adba640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3e740af8e0 0x7f3e7406b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:12.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.295+0000 7f3e7adba640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e7415f130 con 0x7f3e7415a670 2024-03-29T04:20:12.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.295+0000 7f3e7adba640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f3e740b4a60 con 0x7f3e740af8e0 2024-03-29T04:20:12.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.295+0000 7f3e7adba640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f3e740b4130 con 0x7f3e740afe30 2024-03-29T04:20:12.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.295+0000 7f3e78b56640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f3e7415a670 0x7f3e7415e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:45622/0 (socket says 172.21.15.22:45622) 2024-03-29T04:20:12.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.295+0000 7f3e78b56640 1 -- 172.21.15.22:0/54681776 learned_addr learned my addr 172.21.15.22:0/54681776 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:12.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.295+0000 7f3e79357640 1 --2- 172.21.15.22:0/54681776 >> v2:172.21.15.22:3300/0 conn(0x7f3e740af8e0 0x7f3e7406b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:12.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.295+0000 7f3e79357640 1 -- 172.21.15.22:0/54681776 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3e740afe30 msgr2=0x7f3e7415ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:12.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.295+0000 7f3e73fff640 1 --2- 172.21.15.22:0/54681776 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3e740afe30 0x7f3e7415ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:12.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.295+0000 7f3e79357640 1 --2- 172.21.15.22:0/54681776 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3e740afe30 0x7f3e7415ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.295+0000 7f3e737fe640 1 -- 172.21.15.22:0/54681776 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4220642243 0 0) 0x7f3e7415f130 con 0x7f3e7415a670 2024-03-29T04:20:12.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.295+0000 7f3e79357640 1 -- 172.21.15.22:0/54681776 >> v1:172.21.15.22:6789/0 conn(0x7f3e7415a670 legacy=0x7f3e7415e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:12.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.295+0000 7f3e79357640 1 -- 172.21.15.22:0/54681776 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e740b4410 con 0x7f3e740af8e0 2024-03-29T04:20:12.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.295+0000 7f3e73fff640 1 --2- 172.21.15.22:0/54681776 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3e740afe30 0x7f3e7415ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:12.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.299+0000 7f3e79357640 1 --2- 172.21.15.22:0/54681776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e740af8e0 0x7f3e7406b660 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009d90 tx=0x7f3e6c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174eb8c4854e4fdc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:12.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.299+0000 7f3e737fe640 1 -- 172.21.15.22:0/54681776 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c0b2070 con 0x7f3e740af8e0 2024-03-29T04:20:12.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.299+0000 7f3e737fe640 1 -- 172.21.15.22:0/54681776 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e6c0a59d0 con 0x7f3e740af8e0 2024-03-29T04:20:12.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.299+0000 7f3e737fe640 1 -- 172.21.15.22:0/54681776 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c0a5da0 con 0x7f3e740af8e0 2024-03-29T04:20:12.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.299+0000 7f3e7adba640 1 -- 172.21.15.22:0/54681776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e740af8e0 msgr2=0x7f3e7406b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:12.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.299+0000 7f3e7adba640 1 --2- 172.21.15.22:0/54681776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e740af8e0 0x7f3e7406b660 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009d90 tx=0x7f3e6c01c970 comp rx=0 tx=0).stop 2024-03-29T04:20:12.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.299+0000 7f3e7adba640 1 -- 172.21.15.22:0/54681776 shutdown_connections 2024-03-29T04:20:12.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.299+0000 7f3e7adba640 1 --2- 172.21.15.22:0/54681776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e740af8e0 0x7f3e7406b660 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.299+0000 7f3e7adba640 1 --2- 172.21.15.22:0/54681776 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3e740afe30 0x7f3e7415ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.299+0000 7f3e7adba640 1 -- 172.21.15.22:0/54681776 >> 172.21.15.22:0/54681776 conn(0x7f3e740b2b30 msgr2=0x7f3e740d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:12.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.299+0000 7f3e7adba640 1 -- 172.21.15.22:0/54681776 shutdown_connections 2024-03-29T04:20:12.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.299+0000 7f3e7adba640 1 -- 172.21.15.22:0/54681776 wait complete. 2024-03-29T04:20:12.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.303+0000 7f3e7adba640 1 Processor -- start 2024-03-29T04:20:12.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.303+0000 7f3e7adba640 1 -- start start 2024-03-29T04:20:12.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.303+0000 7f3e7adba640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e740af8e0 0x7f3e74142360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:12.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.303+0000 7f3e7adba640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3e740afe30 0x7f3e7415b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:12.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.303+0000 7f3e7adba640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e7415a670 0x7f3e74152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:12.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.303+0000 7f3e7adba640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7406bf30 con 0x7f3e740af8e0 2024-03-29T04:20:12.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.303+0000 7f3e7adba640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7406bdb0 con 0x7f3e7415a670 2024-03-29T04:20:12.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.303+0000 7f3e7adba640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f3e7406c0b0 con 0x7f3e740afe30 2024-03-29T04:20:12.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.303+0000 7f3e78b56640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e740af8e0 0x7f3e74142360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:12.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.303+0000 7f3e78b56640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e740af8e0 0x7f3e74142360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38816/0 (socket says 172.21.15.22:38816) 2024-03-29T04:20:12.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.303+0000 7f3e78b56640 1 -- 172.21.15.22:0/2613640756 learned_addr learned my addr 172.21.15.22:0/2613640756 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:12.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.303+0000 7f3e79357640 1 --2- 172.21.15.22:0/2613640756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e7415a670 0x7f3e74152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:12.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.303+0000 7f3e73fff640 1 --2- 172.21.15.22:0/2613640756 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3e740afe30 0x7f3e7415b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:12.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.303+0000 7f3e78b56640 1 -- 172.21.15.22:0/2613640756 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3e740afe30 msgr2=0x7f3e7415b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:12.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.303+0000 7f3e78b56640 1 --2- 172.21.15.22:0/2613640756 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3e740afe30 0x7f3e7415b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.303+0000 7f3e78b56640 1 -- 172.21.15.22:0/2613640756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e7415a670 msgr2=0x7f3e74152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:12.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.303+0000 7f3e78b56640 1 --2- 172.21.15.22:0/2613640756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e7415a670 0x7f3e74152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.303+0000 7f3e78b56640 1 -- 172.21.15.22:0/2613640756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e741428a0 con 0x7f3e740af8e0 2024-03-29T04:20:12.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.307+0000 7f3e73fff640 1 --2- 172.21.15.22:0/2613640756 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3e740afe30 0x7f3e7415b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:20:12.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.307+0000 7f3e78b56640 1 --2- 172.21.15.22:0/2613640756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e740af8e0 0x7f3e74142360 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400c480 tx=0x7f3e6400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:12.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.307+0000 7f3e717fa640 1 -- 172.21.15.22:0/2613640756 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e64018070 con 0x7f3e740af8e0 2024-03-29T04:20:12.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.307+0000 7f3e717fa640 1 -- 172.21.15.22:0/2613640756 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e64007500 con 0x7f3e740af8e0 2024-03-29T04:20:12.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.307+0000 7f3e7adba640 1 -- 172.21.15.22:0/2613640756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e74151060 con 0x7f3e740af8e0 2024-03-29T04:20:12.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.307+0000 7f3e717fa640 1 -- 172.21.15.22:0/2613640756 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e64004a30 con 0x7f3e740af8e0 2024-03-29T04:20:12.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.307+0000 7f3e7adba640 1 -- 172.21.15.22:0/2613640756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e74151620 con 0x7f3e740af8e0 2024-03-29T04:20:12.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.307+0000 7f3e7adba640 1 -- 172.21.15.22:0/2613640756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e74068890 con 0x7f3e740af8e0 2024-03-29T04:20:12.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.307+0000 7f3e717fa640 1 -- 172.21.15.22:0/2613640756 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f3e6401b070 con 0x7f3e740af8e0 2024-03-29T04:20:12.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.307+0000 7f3e717fa640 1 --2- 172.21.15.22:0/2613640756 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3e60067fc0 0x7f3e6006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:12.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.307+0000 7f3e717fa640 1 -- 172.21.15.22:0/2613640756 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f3e64085870 con 0x7f3e740af8e0 2024-03-29T04:20:12.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.307+0000 7f3e73fff640 1 --2- 172.21.15.22:0/2613640756 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3e60067fc0 0x7f3e6006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:12.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.307+0000 7f3e73fff640 1 --2- 172.21.15.22:0/2613640756 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3e60067fc0 0x7f3e6006a480 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3e68006fd0 tx=0x7f3e68008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.311+0000 7f3e717fa640 1 -- 172.21.15.22:0/2613640756 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f3e640558b0 con 0x7f3e740af8e0 2024-03-29T04:20:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.475+0000 7f3e7adba640 1 -- 172.21.15.22:0/2613640756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e7406fb70 con 0x7f3e740af8e0 2024-03-29T04:20:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.475+0000 7f3e717fa640 1 -- 172.21.15.22:0/2613640756 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16991 (secure 0 0 0) 0x7f3e64059bb0 con 0x7f3e740af8e0 2024-03-29T04:20:12.458 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:20:12.459 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":42,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:19:17.809849+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:18:56.336876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:18:56.950670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:20:12.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.479+0000 7f3e7adba640 1 -- 172.21.15.22:0/2613640756 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3e60067fc0 msgr2=0x7f3e6006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.479+0000 7f3e7adba640 1 --2- 172.21.15.22:0/2613640756 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3e60067fc0 0x7f3e6006a480 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3e68006fd0 tx=0x7f3e68008040 comp rx=0 tx=0).stop 2024-03-29T04:20:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.479+0000 7f3e7adba640 1 -- 172.21.15.22:0/2613640756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e740af8e0 msgr2=0x7f3e74142360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.479+0000 7f3e7adba640 1 --2- 172.21.15.22:0/2613640756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e740af8e0 0x7f3e74142360 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400c480 tx=0x7f3e6400c940 comp rx=0 tx=0).stop 2024-03-29T04:20:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.479+0000 7f3e7adba640 1 -- 172.21.15.22:0/2613640756 shutdown_connections 2024-03-29T04:20:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.479+0000 7f3e7adba640 1 --2- 172.21.15.22:0/2613640756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e7415a670 0x7f3e74152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.479+0000 7f3e7adba640 1 --2- 172.21.15.22:0/2613640756 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3e740afe30 0x7f3e7415b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.479+0000 7f3e7adba640 1 --2- 172.21.15.22:0/2613640756 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3e60067fc0 0x7f3e6006a480 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.479+0000 7f3e7adba640 1 --2- 172.21.15.22:0/2613640756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e740af8e0 0x7f3e74142360 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.479+0000 7f3e7adba640 1 -- 172.21.15.22:0/2613640756 >> 172.21.15.22:0/2613640756 conn(0x7f3e740b2b30 msgr2=0x7f3e740b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.479+0000 7f3e7adba640 1 -- 172.21.15.22:0/2613640756 shutdown_connections 2024-03-29T04:20:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.479+0000 7f3e7adba640 1 -- 172.21.15.22:0/2613640756 wait complete. 2024-03-29T04:20:12.478 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T04:20:12.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.619+0000 7fb700de1640 1 Processor -- start 2024-03-29T04:20:12.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.619+0000 7fb700de1640 1 -- start start 2024-03-29T04:20:12.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.619+0000 7fb700de1640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb6fc145670 0x7fb6fc15fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:12.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.619+0000 7fb700de1640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6fc0cfce0 con 0x7fb6fc0d18e0 2024-03-29T04:20:12.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.619+0000 7fb700de1640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6fc0d1e30 con 0x7fb6fc0c6ef0 2024-03-29T04:20:12.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.619+0000 7fb700de1640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fb6fc06f430 con 0x7fb6fc145670 2024-03-29T04:20:12.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.619+0000 7fb6fa59c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb6fc145670 0x7fb6fc15fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:12.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.619+0000 7fb6f3fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb6fc0d18e0 0x7fb6fc1601b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:45630/0 (socket says 172.21.15.22:45630) 2024-03-29T04:20:12.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.619+0000 7fb6f3fff640 1 -- 172.21.15.22:0/2389067706 learned_addr learned my addr 172.21.15.22:0/2389067706 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:12.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.619+0000 7fb6fa59c640 1 -- 172.21.15.22:0/2389067706 >> v1:172.21.15.151:6789/0 conn(0x7fb6fc0c6ef0 legacy=0x7fb6fc0c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:12.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.619+0000 7fb6fa59c640 1 -- 172.21.15.22:0/2389067706 >> v1:172.21.15.22:6789/0 conn(0x7fb6fc0d18e0 legacy=0x7fb6fc1601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:12.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.619+0000 7fb6fa59c640 1 -- 172.21.15.22:0/2389067706 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6fc06f710 con 0x7fb6fc145670 2024-03-29T04:20:12.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.619+0000 7fb6f9d9b640 1 -- 172.21.15.22:0/2389067706 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3381265819 0 0) 0x7fb6fc0cfce0 con 0x7fb6fc0d18e0 2024-03-29T04:20:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb6fa59c640 1 --2- 172.21.15.22:0/2389067706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6fc145670 0x7fb6fc15fc70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0097c0 tx=0x7fb6ec01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b003786eac396 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb6f9d9b640 1 -- 172.21.15.22:0/2389067706 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0b0070 con 0x7fb6fc145670 2024-03-29T04:20:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb6f9d9b640 1 -- 172.21.15.22:0/2389067706 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6ec0ab600 con 0x7fb6fc145670 2024-03-29T04:20:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb6f9d9b640 1 -- 172.21.15.22:0/2389067706 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0ab9d0 con 0x7fb6fc145670 2024-03-29T04:20:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb700de1640 1 -- 172.21.15.22:0/2389067706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6fc145670 msgr2=0x7fb6fc15fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb700de1640 1 --2- 172.21.15.22:0/2389067706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6fc145670 0x7fb6fc15fc70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0097c0 tx=0x7fb6ec01c4a0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb700de1640 1 -- 172.21.15.22:0/2389067706 shutdown_connections 2024-03-29T04:20:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb700de1640 1 --2- 172.21.15.22:0/2389067706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6fc145670 0x7fb6fc15fc70 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb700de1640 1 -- 172.21.15.22:0/2389067706 >> 172.21.15.22:0/2389067706 conn(0x7fb6fc0b3c80 msgr2=0x7fb6fc0b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb700de1640 1 -- 172.21.15.22:0/2389067706 shutdown_connections 2024-03-29T04:20:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb700de1640 1 -- 172.21.15.22:0/2389067706 wait complete. 2024-03-29T04:20:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb700de1640 1 Processor -- start 2024-03-29T04:20:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb700de1640 1 -- start start 2024-03-29T04:20:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb700de1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6fc0c6ef0 0x7fb6fc13bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb700de1640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6fc0d18e0 0x7fb6fc1f8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb700de1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6fc145670 0x7fb6fc1f8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb700de1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc160a40 con 0x7fb6fc0c6ef0 2024-03-29T04:20:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb700de1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc1608c0 con 0x7fb6fc145670 2024-03-29T04:20:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb700de1640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb6fc16a830 con 0x7fb6fc0d18e0 2024-03-29T04:20:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb6fa59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6fc0c6ef0 0x7fb6fc13bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb6fa59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6fc0c6ef0 0x7fb6fc13bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38840/0 (socket says 172.21.15.22:38840) 2024-03-29T04:20:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.623+0000 7fb6fa59c640 1 -- 172.21.15.22:0/3062784937 learned_addr learned my addr 172.21.15.22:0/3062784937 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.627+0000 7fb6fad9d640 1 --2- 172.21.15.22:0/3062784937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6fc145670 0x7fb6fc1f8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.627+0000 7fb6f3fff640 1 --2- 172.21.15.22:0/3062784937 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6fc0d18e0 0x7fb6fc1f8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.627+0000 7fb6fa59c640 1 -- 172.21.15.22:0/3062784937 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6fc0d18e0 msgr2=0x7fb6fc1f8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.627+0000 7fb6fa59c640 1 --2- 172.21.15.22:0/3062784937 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6fc0d18e0 0x7fb6fc1f8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.627+0000 7fb6fa59c640 1 -- 172.21.15.22:0/3062784937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6fc145670 msgr2=0x7fb6fc1f8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.627+0000 7fb6fa59c640 1 --2- 172.21.15.22:0/3062784937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6fc145670 0x7fb6fc1f8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.627+0000 7fb6fa59c640 1 -- 172.21.15.22:0/3062784937 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6fc0cb250 con 0x7fb6fc0c6ef0 2024-03-29T04:20:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.627+0000 7fb6fa59c640 1 --2- 172.21.15.22:0/3062784937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6fc0c6ef0 0x7fb6fc13bb50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0097c0 tx=0x7fb6ec01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.627+0000 7fb6f2ffd640 1 -- 172.21.15.22:0/3062784937 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0b0050 con 0x7fb6fc0c6ef0 2024-03-29T04:20:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.627+0000 7fb6f2ffd640 1 -- 172.21.15.22:0/3062784937 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6ec0b1040 con 0x7fb6fc0c6ef0 2024-03-29T04:20:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.627+0000 7fb700de1640 1 -- 172.21.15.22:0/3062784937 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6fc154580 con 0x7fb6fc0c6ef0 2024-03-29T04:20:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.627+0000 7fb6f2ffd640 1 -- 172.21.15.22:0/3062784937 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec004260 con 0x7fb6fc0c6ef0 2024-03-29T04:20:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.627+0000 7fb700de1640 1 -- 172.21.15.22:0/3062784937 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6fc0cb7d0 con 0x7fb6fc0c6ef0 2024-03-29T04:20:12.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.627+0000 7fb700de1640 1 -- 172.21.15.22:0/3062784937 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6fc068890 con 0x7fb6fc0c6ef0 2024-03-29T04:20:12.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.627+0000 7fb6f2ffd640 1 -- 172.21.15.22:0/3062784937 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb6ec0be060 con 0x7fb6fc0c6ef0 2024-03-29T04:20:12.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.627+0000 7fb6f2ffd640 1 --2- 172.21.15.22:0/3062784937 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb6dc067fc0 0x7fb6dc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:12.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.627+0000 7fb6f2ffd640 1 -- 172.21.15.22:0/3062784937 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fb6ec1200f0 con 0x7fb6fc0c6ef0 2024-03-29T04:20:12.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.627+0000 7fb6f3fff640 1 --2- 172.21.15.22:0/3062784937 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb6dc067fc0 0x7fb6dc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:12.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.631+0000 7fb6f3fff640 1 --2- 172.21.15.22:0/3062784937 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb6dc067fc0 0x7fb6dc06a480 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4007fd0 tx=0x7fb6e4007ba0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:12.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.631+0000 7fb6f2ffd640 1 -- 172.21.15.22:0/3062784937 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fb6ec016090 con 0x7fb6fc0c6ef0 2024-03-29T04:20:12.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.831+0000 7fb700de1640 1 -- 172.21.15.22:0/3062784937 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb6fc0c8730 con 0x7fb6fc0c6ef0 2024-03-29T04:20:12.813 INFO:tasks.ceph.mon.a.smithi022.stderr:2024-03-29T04:20:12.831+0000 7fd3d21cd640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T04:20:12.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.835+0000 7fb6f2ffd640 1 -- 172.21.15.22:0/3062784937 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) ==== 161+0+0 (secure 0 0 0) 0x7fb6ec0f06f0 con 0x7fb6fc0c6ef0 2024-03-29T04:20:12.816 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T04:20:12.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.839+0000 7fb700de1640 1 -- 172.21.15.22:0/3062784937 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb6dc067fc0 msgr2=0x7fb6dc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:12.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.839+0000 7fb700de1640 1 --2- 172.21.15.22:0/3062784937 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb6dc067fc0 0x7fb6dc06a480 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4007fd0 tx=0x7fb6e4007ba0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.839+0000 7fb700de1640 1 -- 172.21.15.22:0/3062784937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6fc0c6ef0 msgr2=0x7fb6fc13bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:12.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.839+0000 7fb700de1640 1 --2- 172.21.15.22:0/3062784937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6fc0c6ef0 0x7fb6fc13bb50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0097c0 tx=0x7fb6ec01ace0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.839+0000 7fb700de1640 1 -- 172.21.15.22:0/3062784937 shutdown_connections 2024-03-29T04:20:12.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.839+0000 7fb700de1640 1 --2- 172.21.15.22:0/3062784937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6fc145670 0x7fb6fc1f8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.839+0000 7fb700de1640 1 --2- 172.21.15.22:0/3062784937 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6fc0d18e0 0x7fb6fc1f8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.839+0000 7fb700de1640 1 --2- 172.21.15.22:0/3062784937 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb6dc067fc0 0x7fb6dc06a480 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.839+0000 7fb700de1640 1 --2- 172.21.15.22:0/3062784937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6fc0c6ef0 0x7fb6fc13bb50 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.839+0000 7fb700de1640 1 -- 172.21.15.22:0/3062784937 >> 172.21.15.22:0/3062784937 conn(0x7fb6fc0b3c80 msgr2=0x7fb6fc0ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:12.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.839+0000 7fb700de1640 1 -- 172.21.15.22:0/3062784937 shutdown_connections 2024-03-29T04:20:12.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.839+0000 7fb700de1640 1 -- 172.21.15.22:0/3062784937 wait complete. 2024-03-29T04:20:12.843 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-29T04:20:12.865 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2024-03-29T04:20:12.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.991+0000 7fbc9e3ab640 1 Processor -- start 2024-03-29T04:20:12.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.991+0000 7fbc9e3ab640 1 -- start start 2024-03-29T04:20:12.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.995+0000 7fbc9e3ab640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fbc9815c160 0x7fbc9806b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.995+0000 7fbc9e3ab640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc9815fe20 0x7fbc98160290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.995+0000 7fbc9e3ab640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc98160c80 con 0x7fbc980ccaa0 2024-03-29T04:20:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.995+0000 7fbc9e3ab640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fbc981607d0 con 0x7fbc9815c160 2024-03-29T04:20:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.995+0000 7fbc9e3ab640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fbc980cd680 con 0x7fbc9815fe20 2024-03-29T04:20:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.995+0000 7fbc97fff640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fbc9815c160 0x7fbc9806b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.995+0000 7fbc9c948640 1 --1- >> v1:172.21.15.151:6789/0 conn(0x7fbc980ccaa0 0x7fbc980cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.151:6789/0 says I am v1:172.21.15.22:50094/0 (socket says 172.21.15.22:50094) 2024-03-29T04:20:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.995+0000 7fbc977fe640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc9815fe20 0x7fbc98160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.995+0000 7fbc9c948640 1 -- 172.21.15.22:0/815490298 learned_addr learned my addr 172.21.15.22:0/815490298 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:12.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.995+0000 7fbc97fff640 1 -- 172.21.15.22:0/815490298 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc9815fe20 msgr2=0x7fbc98160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:12.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.995+0000 7fbc97fff640 1 --2- 172.21.15.22:0/815490298 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc9815fe20 0x7fbc98160290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.995+0000 7fbc97fff640 1 -- 172.21.15.22:0/815490298 >> v1:172.21.15.151:6789/0 conn(0x7fbc980ccaa0 legacy=0x7fbc980cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:12.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.995+0000 7fbc97fff640 1 -- 172.21.15.22:0/815490298 --> v2:172.21.15.151:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc980cd960 con 0x7fbc9815c160 2024-03-29T04:20:12.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.999+0000 7fbc97fff640 1 --2- 172.21.15.22:0/815490298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9815c160 0x7fbc9806b660 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbc880097c0 tx=0x7fbc8801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c90b822700f927c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:12.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.999+0000 7fbc96ffd640 1 -- 172.21.15.22:0/815490298 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc880a7070 con 0x7fbc9815c160 2024-03-29T04:20:12.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.999+0000 7fbc96ffd640 1 -- 172.21.15.22:0/815490298 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc880b0070 con 0x7fbc9815c160 2024-03-29T04:20:12.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.999+0000 7fbc96ffd640 1 -- 172.21.15.22:0/815490298 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc880ab890 con 0x7fbc9815c160 2024-03-29T04:20:12.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.999+0000 7fbc9e3ab640 1 -- 172.21.15.22:0/815490298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9815c160 msgr2=0x7fbc9806b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:12.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.999+0000 7fbc9e3ab640 1 --2- 172.21.15.22:0/815490298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9815c160 0x7fbc9806b660 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbc880097c0 tx=0x7fbc8801c470 comp rx=0 tx=0).stop 2024-03-29T04:20:12.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.999+0000 7fbc9e3ab640 1 -- 172.21.15.22:0/815490298 shutdown_connections 2024-03-29T04:20:12.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.999+0000 7fbc9e3ab640 1 --2- 172.21.15.22:0/815490298 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc9815fe20 0x7fbc98160290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.999+0000 7fbc9e3ab640 1 --2- 172.21.15.22:0/815490298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9815c160 0x7fbc9806b660 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.999+0000 7fbc9e3ab640 1 -- 172.21.15.22:0/815490298 >> 172.21.15.22:0/815490298 conn(0x7fbc980d0c90 msgr2=0x7fbc980d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:12.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.999+0000 7fbc9e3ab640 1 -- 172.21.15.22:0/815490298 shutdown_connections 2024-03-29T04:20:12.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.999+0000 7fbc9e3ab640 1 -- 172.21.15.22:0/815490298 wait complete. 2024-03-29T04:20:12.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.999+0000 7fbc9e3ab640 1 Processor -- start 2024-03-29T04:20:12.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:12.999+0000 7fbc9e3ab640 1 -- start start 2024-03-29T04:20:12.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc9e3ab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc980ccaa0 0x7fbc9814f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc9e3ab640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc9815c160 0x7fbc9813bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc9e3ab640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9815fe20 0x7fbc9813c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc9e3ab640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbc980cf420 con 0x7fbc980ccaa0 2024-03-29T04:20:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc9e3ab640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbc980cf2a0 con 0x7fbc9815fe20 2024-03-29T04:20:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc9e3ab640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fbc9815d550 con 0x7fbc9815c160 2024-03-29T04:20:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc97fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc980ccaa0 0x7fbc9814f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc97fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc980ccaa0 0x7fbc9814f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38876/0 (socket says 172.21.15.22:38876) 2024-03-29T04:20:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc97fff640 1 -- 172.21.15.22:0/3316454502 learned_addr learned my addr 172.21.15.22:0/3316454502 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc9c948640 1 --2- 172.21.15.22:0/3316454502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9815fe20 0x7fbc9813c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc977fe640 1 --2- 172.21.15.22:0/3316454502 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc9815c160 0x7fbc9813bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc97fff640 1 -- 172.21.15.22:0/3316454502 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc9815c160 msgr2=0x7fbc9813bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc97fff640 1 --2- 172.21.15.22:0/3316454502 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc9815c160 0x7fbc9813bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc97fff640 1 -- 172.21.15.22:0/3316454502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9815fe20 msgr2=0x7fbc9813c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc97fff640 1 --2- 172.21.15.22:0/3316454502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9815fe20 0x7fbc9813c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:12.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc97fff640 1 -- 172.21.15.22:0/3316454502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9814aef0 con 0x7fbc980ccaa0 2024-03-29T04:20:12.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc9c948640 1 --2- 172.21.15.22:0/3316454502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9815fe20 0x7fbc9813c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:12.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc977fe640 1 --2- 172.21.15.22:0/3316454502 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc9815c160 0x7fbc9813bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:12.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc97fff640 1 --2- 172.21.15.22:0/3316454502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc980ccaa0 0x7fbc9814f900 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbc880ad040 tx=0x7fbc8801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:12.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc94ff9640 1 -- 172.21.15.22:0/3316454502 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc880bf610 con 0x7fbc980ccaa0 2024-03-29T04:20:12.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc94ff9640 1 -- 172.21.15.22:0/3316454502 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc880bf7d0 con 0x7fbc980ccaa0 2024-03-29T04:20:12.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc9e3ab640 1 -- 172.21.15.22:0/3316454502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc9814b1d0 con 0x7fbc980ccaa0 2024-03-29T04:20:12.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc94ff9640 1 -- 172.21.15.22:0/3316454502 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc880bfbc0 con 0x7fbc980ccaa0 2024-03-29T04:20:12.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc9e3ab640 1 -- 172.21.15.22:0/3316454502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc9814fe40 con 0x7fbc980ccaa0 2024-03-29T04:20:12.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc9e3ab640 1 -- 172.21.15.22:0/3316454502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc98068900 con 0x7fbc980ccaa0 2024-03-29T04:20:12.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc94ff9640 1 -- 172.21.15.22:0/3316454502 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fbc880bd020 con 0x7fbc980ccaa0 2024-03-29T04:20:12.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc94ff9640 1 --2- 172.21.15.22:0/3316454502 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbc84067fc0 0x7fbc8406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:12.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc94ff9640 1 -- 172.21.15.22:0/3316454502 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6216+0+0 (secure 0 0 0) 0x7fbc88120dd0 con 0x7fbc980ccaa0 2024-03-29T04:20:12.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.003+0000 7fbc977fe640 1 --2- 172.21.15.22:0/3316454502 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbc84067fc0 0x7fbc8406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:12.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.007+0000 7fbc977fe640 1 --2- 172.21.15.22:0/3316454502 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbc84067fc0 0x7fbc8406a480 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbc981f8790 tx=0x7fbc8c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:12.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.007+0000 7fbc94ff9640 1 -- 172.21.15.22:0/3316454502 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fbc88016090 con 0x7fbc980ccaa0 2024-03-29T04:20:13.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.199+0000 7fbc9e3ab640 1 -- 172.21.15.22:0/3316454502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fbc9814b3a0 con 0x7fbc980ccaa0 2024-03-29T04:20:13.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.839+0000 7fbc94ff9640 1 -- 172.21.15.22:0/3316454502 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) ==== 104+0+0 (secure 0 0 0) 0x7fbc881220a0 con 0x7fbc980ccaa0 2024-03-29T04:20:13.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.843+0000 7fbc9e3ab640 1 -- 172.21.15.22:0/3316454502 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbc84067fc0 msgr2=0x7fbc8406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:13.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.843+0000 7fbc9e3ab640 1 --2- 172.21.15.22:0/3316454502 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbc84067fc0 0x7fbc8406a480 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbc981f8790 tx=0x7fbc8c008040 comp rx=0 tx=0).stop 2024-03-29T04:20:13.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.843+0000 7fbc9e3ab640 1 -- 172.21.15.22:0/3316454502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc980ccaa0 msgr2=0x7fbc9814f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:13.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.843+0000 7fbc9e3ab640 1 --2- 172.21.15.22:0/3316454502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc980ccaa0 0x7fbc9814f900 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbc880ad040 tx=0x7fbc8801acb0 comp rx=0 tx=0).stop 2024-03-29T04:20:13.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.843+0000 7fbc9e3ab640 1 -- 172.21.15.22:0/3316454502 shutdown_connections 2024-03-29T04:20:13.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.843+0000 7fbc9e3ab640 1 --2- 172.21.15.22:0/3316454502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9815fe20 0x7fbc9813c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:13.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.843+0000 7fbc9e3ab640 1 --2- 172.21.15.22:0/3316454502 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fbc9815c160 0x7fbc9813bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:13.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.843+0000 7fbc9e3ab640 1 --2- 172.21.15.22:0/3316454502 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fbc84067fc0 0x7fbc8406a480 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:13.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.843+0000 7fbc9e3ab640 1 --2- 172.21.15.22:0/3316454502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc980ccaa0 0x7fbc9814f900 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:13.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.843+0000 7fbc9e3ab640 1 -- 172.21.15.22:0/3316454502 >> 172.21.15.22:0/3316454502 conn(0x7fbc980d0c90 msgr2=0x7fbc981522b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:13.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.843+0000 7fbc9e3ab640 1 -- 172.21.15.22:0/3316454502 shutdown_connections 2024-03-29T04:20:13.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:13.843+0000 7fbc9e3ab640 1 -- 172.21.15.22:0/3316454502 wait complete. 2024-03-29T04:20:13.844 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-29T04:20:14.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.035+0000 7f4210790640 1 Processor -- start 2024-03-29T04:20:14.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.035+0000 7f4210790640 1 -- start start 2024-03-29T04:20:14.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.035+0000 7f4210790640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f42080b2870 0x7f42080b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:14.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.035+0000 7f4210790640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42081620c0 con 0x7f42080b2c50 2024-03-29T04:20:14.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.035+0000 7f4210790640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4208162380 con 0x7f42080ceaf0 2024-03-29T04:20:14.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.035+0000 7f4210790640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f4208162f90 con 0x7f42080b2870 2024-03-29T04:20:14.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.039+0000 7f420e52c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f42080b2870 0x7f42080b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:14.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.039+0000 7f420dd2b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f42080b2c50 0x7f42080d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:45654/0 (socket says 172.21.15.22:45654) 2024-03-29T04:20:14.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.039+0000 7f420dd2b640 1 -- 172.21.15.22:0/2619544752 learned_addr learned my addr 172.21.15.22:0/2619544752 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:14.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.039+0000 7f420e52c640 1 -- 172.21.15.22:0/2619544752 >> v1:172.21.15.151:6789/0 conn(0x7f42080ceaf0 legacy=0x7f420815ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:14.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.039+0000 7f420e52c640 1 -- 172.21.15.22:0/2619544752 >> v1:172.21.15.22:6789/0 conn(0x7f42080b2c50 legacy=0x7f42080d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:14.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.039+0000 7f420e52c640 1 -- 172.21.15.22:0/2619544752 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4208162660 con 0x7f42080b2870 2024-03-29T04:20:14.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.039+0000 7f420d52a640 1 -- 172.21.15.22:0/2619544752 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2594530419 0 0) 0x7f42081620c0 con 0x7f42080b2c50 2024-03-29T04:20:14.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.039+0000 7f420e52c640 1 --2- 172.21.15.22:0/2619544752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42080b2870 0x7f42080b1b40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4208163110 tx=0x7f420401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58ea323d8ba105d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:14.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.039+0000 7f420d52a640 1 -- 172.21.15.22:0/2619544752 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42040b0070 con 0x7f42080b2870 2024-03-29T04:20:14.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.039+0000 7f420d52a640 1 -- 172.21.15.22:0/2619544752 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4204004230 con 0x7f42080b2870 2024-03-29T04:20:14.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.039+0000 7f420d52a640 1 -- 172.21.15.22:0/2619544752 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204003620 con 0x7f42080b2870 2024-03-29T04:20:14.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.039+0000 7f4210790640 1 -- 172.21.15.22:0/2619544752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42080b2870 msgr2=0x7f42080b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:14.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.039+0000 7f4210790640 1 --2- 172.21.15.22:0/2619544752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42080b2870 0x7f42080b1b40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4208163110 tx=0x7f420401c460 comp rx=0 tx=0).stop 2024-03-29T04:20:14.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.039+0000 7f4210790640 1 -- 172.21.15.22:0/2619544752 shutdown_connections 2024-03-29T04:20:14.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.039+0000 7f4210790640 1 --2- 172.21.15.22:0/2619544752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42080b2870 0x7f42080b1b40 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:14.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.039+0000 7f4210790640 1 -- 172.21.15.22:0/2619544752 >> 172.21.15.22:0/2619544752 conn(0x7f42080b3cf0 msgr2=0x7f42080b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:14.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.039+0000 7f4210790640 1 -- 172.21.15.22:0/2619544752 shutdown_connections 2024-03-29T04:20:14.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.039+0000 7f4210790640 1 -- 172.21.15.22:0/2619544752 wait complete. 2024-03-29T04:20:14.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f4210790640 1 Processor -- start 2024-03-29T04:20:14.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f4210790640 1 -- start start 2024-03-29T04:20:14.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f4210790640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f42080b2870 0x7f42080c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:14.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f4210790640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42080ceaf0 0x7f42080cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:14.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f4210790640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42080cbb20 0x7f42080b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:14.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f4210790640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f42080d1d80 con 0x7f42080cbb20 2024-03-29T04:20:14.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f4210790640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f420815f920 con 0x7f42080ceaf0 2024-03-29T04:20:14.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f4210790640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f42080d1f00 con 0x7f42080b2870 2024-03-29T04:20:14.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f420ed2d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42080cbb20 0x7f42080b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:14.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f420ed2d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42080cbb20 0x7f42080b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38900/0 (socket says 172.21.15.22:38900) 2024-03-29T04:20:14.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f420ed2d640 1 -- 172.21.15.22:0/1778240514 learned_addr learned my addr 172.21.15.22:0/1778240514 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:14.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f420e52c640 1 --2- 172.21.15.22:0/1778240514 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f42080b2870 0x7f42080c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:14.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f420dd2b640 1 --2- 172.21.15.22:0/1778240514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42080ceaf0 0x7f42080cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:14.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f420ed2d640 1 -- 172.21.15.22:0/1778240514 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f42080b2870 msgr2=0x7f42080c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:14.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f420ed2d640 1 --2- 172.21.15.22:0/1778240514 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f42080b2870 0x7f42080c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:14.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f420ed2d640 1 -- 172.21.15.22:0/1778240514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42080ceaf0 msgr2=0x7f42080cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:14.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f420ed2d640 1 --2- 172.21.15.22:0/1778240514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42080ceaf0 0x7f42080cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:14.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f420ed2d640 1 -- 172.21.15.22:0/1778240514 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42080cd860 con 0x7f42080cbb20 2024-03-29T04:20:14.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f420ed2d640 1 --2- 172.21.15.22:0/1778240514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42080cbb20 0x7f42080b4cf0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f420000c780 tx=0x7f420000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:14.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f41feffd640 1 -- 172.21.15.22:0/1778240514 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200004b70 con 0x7f42080cbb20 2024-03-29T04:20:14.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f41feffd640 1 -- 172.21.15.22:0/1778240514 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4200004d10 con 0x7f42080cbb20 2024-03-29T04:20:14.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f4210790640 1 -- 172.21.15.22:0/1778240514 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42080cdb40 con 0x7f42080cbb20 2024-03-29T04:20:14.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f41feffd640 1 -- 172.21.15.22:0/1778240514 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420000aa90 con 0x7f42080cbb20 2024-03-29T04:20:14.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.043+0000 7f4210790640 1 -- 172.21.15.22:0/1778240514 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42080ce100 con 0x7f42080cbb20 2024-03-29T04:20:14.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.047+0000 7f4210790640 1 -- 172.21.15.22:0/1778240514 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42080687f0 con 0x7f42080cbb20 2024-03-29T04:20:14.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.047+0000 7f41feffd640 1 -- 172.21.15.22:0/1778240514 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f420000ac30 con 0x7f42080cbb20 2024-03-29T04:20:14.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.047+0000 7f41feffd640 1 --2- 172.21.15.22:0/1778240514 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f41ec067fc0 0x7f41ec06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:14.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.047+0000 7f41feffd640 1 -- 172.21.15.22:0/1778240514 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6216+0+0 (secure 0 0 0) 0x7f4200086ca0 con 0x7f42080cbb20 2024-03-29T04:20:14.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.047+0000 7f420e52c640 1 --2- 172.21.15.22:0/1778240514 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f41ec067fc0 0x7f41ec06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:14.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.047+0000 7f420e52c640 1 --2- 172.21.15.22:0/1778240514 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f41ec067fc0 0x7f41ec06a480 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f42040ad000 tx=0x7f4204008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:14.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.051+0000 7f41feffd640 1 -- 172.21.15.22:0/1778240514 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f4200057250 con 0x7f42080cbb20 2024-03-29T04:20:14.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:14.215+0000 7f4210790640 1 -- 172.21.15.22:0/1778240514 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f420806fa90 con 0x7f42080cbb20 2024-03-29T04:20:14.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.003+0000 7f41feffd640 1 -- 172.21.15.22:0/1778240514 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v44) ==== 181+0+0 (secure 0 0 0) 0x7f4200056b50 con 0x7f42080cbb20 2024-03-29T04:20:14.983 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' removed 2024-03-29T04:20:14.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.007+0000 7f4210790640 1 -- 172.21.15.22:0/1778240514 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f41ec067fc0 msgr2=0x7f41ec06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:14.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.007+0000 7f4210790640 1 --2- 172.21.15.22:0/1778240514 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f41ec067fc0 0x7f41ec06a480 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f42040ad000 tx=0x7f4204008040 comp rx=0 tx=0).stop 2024-03-29T04:20:14.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.007+0000 7f4210790640 1 -- 172.21.15.22:0/1778240514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42080cbb20 msgr2=0x7f42080b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:14.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.007+0000 7f4210790640 1 --2- 172.21.15.22:0/1778240514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42080cbb20 0x7f42080b4cf0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f420000c780 tx=0x7f420000cc40 comp rx=0 tx=0).stop 2024-03-29T04:20:14.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.007+0000 7f4210790640 1 -- 172.21.15.22:0/1778240514 shutdown_connections 2024-03-29T04:20:14.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.007+0000 7f4210790640 1 --2- 172.21.15.22:0/1778240514 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f41ec067fc0 0x7f41ec06a480 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:14.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.007+0000 7f4210790640 1 --2- 172.21.15.22:0/1778240514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42080cbb20 0x7f42080b4cf0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:14.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.007+0000 7f4210790640 1 --2- 172.21.15.22:0/1778240514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42080ceaf0 0x7f42080cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:14.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.007+0000 7f4210790640 1 --2- 172.21.15.22:0/1778240514 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f42080b2870 0x7f42080c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:14.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.007+0000 7f4210790640 1 -- 172.21.15.22:0/1778240514 >> 172.21.15.22:0/1778240514 conn(0x7f42080b3cf0 msgr2=0x7f42080b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:14.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.011+0000 7f4210790640 1 -- 172.21.15.22:0/1778240514 shutdown_connections 2024-03-29T04:20:14.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.011+0000 7f4210790640 1 -- 172.21.15.22:0/1778240514 wait complete. 2024-03-29T04:20:15.015 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-29T04:20:15.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.163+0000 7fe10f7bc640 1 Processor -- start 2024-03-29T04:20:15.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.163+0000 7fe10f7bc640 1 -- start start 2024-03-29T04:20:15.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe10f7bc640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe1080b5550 0x7fe108160610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:15.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe10f7bc640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fe1080b5930 0x7fe10815c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:15.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe10f7bc640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe10815d090 con 0x7fe108073490 2024-03-29T04:20:15.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe10f7bc640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fe10815d370 con 0x7fe1080b5930 2024-03-29T04:20:15.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe10f7bc640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe10815fe10 con 0x7fe1080b5550 2024-03-29T04:20:15.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe10dd59640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe108073490 0x7fe10806c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:56136/0 (socket says 172.21.15.22:56136) 2024-03-29T04:20:15.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe10dd59640 1 -- 172.21.15.22:0/4037473393 learned_addr learned my addr 172.21.15.22:0/4037473393 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:15.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe10cd57640 1 --2- 172.21.15.22:0/4037473393 >> v2:172.21.15.151:3300/0 conn(0x7fe1080b5930 0x7fe10815c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:15.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe10d558640 1 --2- 172.21.15.22:0/4037473393 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe1080b5550 0x7fe108160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:15.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe0fbfff640 1 -- 172.21.15.22:0/4037473393 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 287832472 0 0) 0x7fe10815d090 con 0x7fe108073490 2024-03-29T04:20:15.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe0fbfff640 1 -- 172.21.15.22:0/4037473393 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0f4003530 con 0x7fe108073490 2024-03-29T04:20:15.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe0fbfff640 1 -- 172.21.15.22:0/4037473393 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3180953863 0 0) 0x7fe0f4003530 con 0x7fe108073490 2024-03-29T04:20:15.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe0fbfff640 1 -- 172.21.15.22:0/4037473393 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe1080b5550 msgr2=0x7fe108160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:15.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe0fbfff640 1 --2- 172.21.15.22:0/4037473393 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe1080b5550 0x7fe108160610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:15.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe0fbfff640 1 -- 172.21.15.22:0/4037473393 >> v2:172.21.15.151:3300/0 conn(0x7fe1080b5930 msgr2=0x7fe10815c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:15.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe0fbfff640 1 --2- 172.21.15.22:0/4037473393 >> v2:172.21.15.151:3300/0 conn(0x7fe1080b5930 0x7fe10815c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:15.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe0fbfff640 1 -- 172.21.15.22:0/4037473393 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1081600f0 con 0x7fe108073490 2024-03-29T04:20:15.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe0fbfff640 1 -- 172.21.15.22:0/4037473393 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7fe100001a00 con 0x7fe108073490 2024-03-29T04:20:15.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe0fbfff640 1 -- 172.21.15.22:0/4037473393 >> v1:172.21.15.22:6789/0 conn(0x7fe108073490 legacy=0x7fe10806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:15.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe0fbfff640 1 --2- 172.21.15.22:0/4037473393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0f40039c0 0x7fe0f4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:15.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe0fbfff640 1 --2- 172.21.15.22:0/4037473393 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe1080b5550 0x7fe0f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:15.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe0fbfff640 1 --2- 172.21.15.22:0/4037473393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0f4004d50 0x7fe0f40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:15.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe0fbfff640 1 -- 172.21.15.22:0/4037473393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe10815d370 con 0x7fe0f40039c0 2024-03-29T04:20:15.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe0fbfff640 1 -- 172.21.15.22:0/4037473393 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe10815fe10 con 0x7fe0f4004d50 2024-03-29T04:20:15.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe0fbfff640 1 -- 172.21.15.22:0/4037473393 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe0f400b9d0 con 0x7fe1080b5550 2024-03-29T04:20:15.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe10d558640 1 --2- 172.21.15.22:0/4037473393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0f40039c0 0x7fe0f4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:15.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe10dd59640 1 --2- 172.21.15.22:0/4037473393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0f4004d50 0x7fe0f40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:15.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe10cd57640 1 --2- 172.21.15.22:0/4037473393 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe1080b5550 0x7fe0f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:15.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe10d558640 1 -- 172.21.15.22:0/4037473393 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe1080b5550 msgr2=0x7fe0f4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:15.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe10d558640 1 --2- 172.21.15.22:0/4037473393 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe1080b5550 0x7fe0f4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:15.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe10d558640 1 -- 172.21.15.22:0/4037473393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0f4004d50 msgr2=0x7fe0f40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:15.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe10d558640 1 --2- 172.21.15.22:0/4037473393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0f4004d50 0x7fe0f40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:15.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.167+0000 7fe10d558640 1 -- 172.21.15.22:0/4037473393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f4003530 con 0x7fe0f40039c0 2024-03-29T04:20:15.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.171+0000 7fe10d558640 1 --2- 172.21.15.22:0/4037473393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0f40039c0 0x7fe0f4003de0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe104009ae0 tx=0x7fe10401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c8e4da9730f2b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:15.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.171+0000 7fe0fbfff640 1 -- 172.21.15.22:0/4037473393 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1040b2070 con 0x7fe0f40039c0 2024-03-29T04:20:15.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.171+0000 7fe0fbfff640 1 -- 172.21.15.22:0/4037473393 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1040a55a0 con 0x7fe0f40039c0 2024-03-29T04:20:15.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.171+0000 7fe0fbfff640 1 -- 172.21.15.22:0/4037473393 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1040096b0 con 0x7fe0f40039c0 2024-03-29T04:20:15.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.171+0000 7fe10f7bc640 1 -- 172.21.15.22:0/4037473393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0f40039c0 msgr2=0x7fe0f4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:15.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.171+0000 7fe10f7bc640 1 --2- 172.21.15.22:0/4037473393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0f40039c0 0x7fe0f4003de0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe104009ae0 tx=0x7fe10401c9b0 comp rx=0 tx=0).stop 2024-03-29T04:20:15.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.171+0000 7fe10dd59640 1 -- 172.21.15.22:0/4037473393 reap_dead start 2024-03-29T04:20:15.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.171+0000 7fe10f7bc640 1 -- 172.21.15.22:0/4037473393 shutdown_connections 2024-03-29T04:20:15.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.171+0000 7fe10f7bc640 1 -- 172.21.15.22:0/4037473393 >> 172.21.15.22:0/4037473393 conn(0x7fe1080b1b80 msgr2=0x7fe1080b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:15.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.171+0000 7fe10f7bc640 1 -- 172.21.15.22:0/4037473393 shutdown_connections 2024-03-29T04:20:15.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.171+0000 7fe10f7bc640 1 -- 172.21.15.22:0/4037473393 wait complete. 2024-03-29T04:20:15.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.171+0000 7fe10f7bc640 1 Processor -- start 2024-03-29T04:20:15.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.171+0000 7fe10f7bc640 1 -- start start 2024-03-29T04:20:15.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.171+0000 7fe10f7bc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe10814dc50 0x7fe1081507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:15.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.171+0000 7fe10f7bc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe108150ce0 0x7fe1080c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:15.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.171+0000 7fe10f7bc640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe1080cdc70 0x7fe1080c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:15.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.171+0000 7fe10f7bc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe1080ca680 con 0x7fe108150ce0 2024-03-29T04:20:15.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.171+0000 7fe10f7bc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe10806cf90 con 0x7fe10814dc50 2024-03-29T04:20:15.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.171+0000 7fe10f7bc640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe1080ca800 con 0x7fe1080cdc70 2024-03-29T04:20:15.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.175+0000 7fe10cd57640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe108150ce0 0x7fe1080c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:15.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.175+0000 7fe10cd57640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe108150ce0 0x7fe1080c9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51988/0 (socket says 172.21.15.22:51988) 2024-03-29T04:20:15.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.175+0000 7fe10cd57640 1 -- 172.21.15.22:0/4089614485 learned_addr learned my addr 172.21.15.22:0/4089614485 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:15.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.175+0000 7fe10d558640 1 --2- 172.21.15.22:0/4089614485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe10814dc50 0x7fe1081507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:15.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.175+0000 7fe10dd59640 1 --2- 172.21.15.22:0/4089614485 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe1080cdc70 0x7fe1080c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:15.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.175+0000 7fe10cd57640 1 -- 172.21.15.22:0/4089614485 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe1080cdc70 msgr2=0x7fe1080c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:15.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.175+0000 7fe10cd57640 1 --2- 172.21.15.22:0/4089614485 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe1080cdc70 0x7fe1080c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:15.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.175+0000 7fe10cd57640 1 -- 172.21.15.22:0/4089614485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe10814dc50 msgr2=0x7fe1081507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:15.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.175+0000 7fe10cd57640 1 --2- 172.21.15.22:0/4089614485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe10814dc50 0x7fe1081507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:15.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.175+0000 7fe10cd57640 1 -- 172.21.15.22:0/4089614485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe108151780 con 0x7fe108150ce0 2024-03-29T04:20:15.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.175+0000 7fe10cd57640 1 --2- 172.21.15.22:0/4089614485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe108150ce0 0x7fe1080c9200 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc011630 tx=0x7fe0fc011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:15.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.175+0000 7fe0f9ffb640 1 -- 172.21.15.22:0/4089614485 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc004220 con 0x7fe108150ce0 2024-03-29T04:20:15.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.175+0000 7fe0f9ffb640 1 -- 172.21.15.22:0/4089614485 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0fc0043c0 con 0x7fe108150ce0 2024-03-29T04:20:15.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.175+0000 7fe0f9ffb640 1 -- 172.21.15.22:0/4089614485 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc00f960 con 0x7fe108150ce0 2024-03-29T04:20:15.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.175+0000 7fe10f7bc640 1 -- 172.21.15.22:0/4089614485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1081511c0 con 0x7fe108150ce0 2024-03-29T04:20:15.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.175+0000 7fe10f7bc640 1 -- 172.21.15.22:0/4089614485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1080ce2b0 con 0x7fe108150ce0 2024-03-29T04:20:15.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.175+0000 7fe10f7bc640 1 -- 172.21.15.22:0/4089614485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1080687f0 con 0x7fe108150ce0 2024-03-29T04:20:15.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.175+0000 7fe0f9ffb640 1 -- 172.21.15.22:0/4089614485 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe0fc00fb00 con 0x7fe108150ce0 2024-03-29T04:20:15.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.175+0000 7fe0f9ffb640 1 --2- 172.21.15.22:0/4089614485 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe0ec067fc0 0x7fe0ec06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:15.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.175+0000 7fe0f9ffb640 1 -- 172.21.15.22:0/4089614485 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5778+0+0 (secure 0 0 0) 0x7fe0fc086640 con 0x7fe108150ce0 2024-03-29T04:20:15.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.175+0000 7fe10d558640 1 --2- 172.21.15.22:0/4089614485 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe0ec067fc0 0x7fe0ec06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:15.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.179+0000 7fe10d558640 1 --2- 172.21.15.22:0/4089614485 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe0ec067fc0 0x7fe0ec06a480 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe1040af000 tx=0x7fe1040a7040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:15.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.179+0000 7fe0f9ffb640 1 -- 172.21.15.22:0/4089614485 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fe0fc019030 con 0x7fe108150ce0 2024-03-29T04:20:15.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:15.343+0000 7fe10f7bc640 1 -- 172.21.15.22:0/4089614485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe1080af8d0 con 0x7fe108150ce0 2024-03-29T04:20:15.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.007+0000 7fe0f9ffb640 1 -- 172.21.15.22:0/4089614485 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v45) ==== 169+0+0 (secure 0 0 0) 0x7fe0fc0567e0 con 0x7fe108150ce0 2024-03-29T04:20:15.988 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' removed 2024-03-29T04:20:15.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.011+0000 7fe10f7bc640 1 -- 172.21.15.22:0/4089614485 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe0ec067fc0 msgr2=0x7fe0ec06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:15.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.011+0000 7fe10f7bc640 1 --2- 172.21.15.22:0/4089614485 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe0ec067fc0 0x7fe0ec06a480 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe1040af000 tx=0x7fe1040a7040 comp rx=0 tx=0).stop 2024-03-29T04:20:15.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.011+0000 7fe10f7bc640 1 -- 172.21.15.22:0/4089614485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe108150ce0 msgr2=0x7fe1080c9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:15.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.011+0000 7fe10f7bc640 1 --2- 172.21.15.22:0/4089614485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe108150ce0 0x7fe1080c9200 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc011630 tx=0x7fe0fc011af0 comp rx=0 tx=0).stop 2024-03-29T04:20:15.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.011+0000 7fe10f7bc640 1 -- 172.21.15.22:0/4089614485 shutdown_connections 2024-03-29T04:20:15.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.011+0000 7fe10f7bc640 1 --2- 172.21.15.22:0/4089614485 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe1080cdc70 0x7fe1080c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:15.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.011+0000 7fe10f7bc640 1 --2- 172.21.15.22:0/4089614485 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe0ec067fc0 0x7fe0ec06a480 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:15.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.011+0000 7fe10f7bc640 1 --2- 172.21.15.22:0/4089614485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe108150ce0 0x7fe1080c9200 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:15.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.011+0000 7fe10f7bc640 1 --2- 172.21.15.22:0/4089614485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe10814dc50 0x7fe1081507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:15.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.011+0000 7fe10f7bc640 1 -- 172.21.15.22:0/4089614485 >> 172.21.15.22:0/4089614485 conn(0x7fe1080b1b80 msgr2=0x7fe1080afaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:15.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.015+0000 7fe10f7bc640 1 -- 172.21.15.22:0/4089614485 shutdown_connections 2024-03-29T04:20:16.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.019+0000 7fe10f7bc640 1 -- 172.21.15.22:0/4089614485 wait complete. 2024-03-29T04:20:16.030 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-29T04:20:16.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.183+0000 7fe2d9ea4640 1 Processor -- start 2024-03-29T04:20:16.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.183+0000 7fe2d9ea4640 1 -- start start 2024-03-29T04:20:16.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.183+0000 7fe2d9ea4640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fe2d415f740 0x7fe2d4073000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:16.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.183+0000 7fe2d9ea4640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe2d4073540 0x7fe2d41620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:16.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.183+0000 7fe2d9ea4640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2d40d0b20 con 0x7fe2d40d0740 2024-03-29T04:20:16.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.183+0000 7fe2d9ea4640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fe2d4073990 con 0x7fe2d4073540 2024-03-29T04:20:16.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.183+0000 7fe2d9ea4640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fe2d406c490 con 0x7fe2d415f740 2024-03-29T04:20:16.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.183+0000 7fe2d3fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe2d4073540 0x7fe2d41620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:16.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.183+0000 7fe2d37fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe2d40d0740 0x7fe2d406b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:56142/0 (socket says 172.21.15.22:56142) 2024-03-29T04:20:16.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.183+0000 7fe2d37fe640 1 -- 172.21.15.22:0/2572190160 learned_addr learned my addr 172.21.15.22:0/2572190160 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:16.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.187+0000 7fe2d3fff640 1 -- 172.21.15.22:0/2572190160 >> v2:172.21.15.151:3300/0 conn(0x7fe2d415f740 msgr2=0x7fe2d4073000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:16.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.187+0000 7fe2d3fff640 1 --2- 172.21.15.22:0/2572190160 >> v2:172.21.15.151:3300/0 conn(0x7fe2d415f740 0x7fe2d4073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:16.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.187+0000 7fe2d3fff640 1 -- 172.21.15.22:0/2572190160 >> v1:172.21.15.22:6789/0 conn(0x7fe2d40d0740 legacy=0x7fe2d406b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:16.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.187+0000 7fe2d3fff640 1 -- 172.21.15.22:0/2572190160 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d40ca850 con 0x7fe2d4073540 2024-03-29T04:20:16.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.187+0000 7fe2d3fff640 1 --2- 172.21.15.22:0/2572190160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2d4073540 0x7fe2d41620b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40cbb00 tx=0x7fe2c401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac3291aba752d82 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:16.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.187+0000 7fe2d27fc640 1 -- 172.21.15.22:0/2572190160 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40b0070 con 0x7fe2d4073540 2024-03-29T04:20:16.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.187+0000 7fe2d27fc640 1 -- 172.21.15.22:0/2572190160 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2c40ab400 con 0x7fe2d4073540 2024-03-29T04:20:16.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.187+0000 7fe2d27fc640 1 -- 172.21.15.22:0/2572190160 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40ab7d0 con 0x7fe2d4073540 2024-03-29T04:20:16.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.187+0000 7fe2d9ea4640 1 -- 172.21.15.22:0/2572190160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2d4073540 msgr2=0x7fe2d41620b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:16.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.187+0000 7fe2d9ea4640 1 --2- 172.21.15.22:0/2572190160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2d4073540 0x7fe2d41620b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40cbb00 tx=0x7fe2c401c3f0 comp rx=0 tx=0).stop 2024-03-29T04:20:16.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.187+0000 7fe2d9ea4640 1 -- 172.21.15.22:0/2572190160 shutdown_connections 2024-03-29T04:20:16.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.187+0000 7fe2d9ea4640 1 --2- 172.21.15.22:0/2572190160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2d4073540 0x7fe2d41620b0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:16.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.187+0000 7fe2d9ea4640 1 --2- 172.21.15.22:0/2572190160 >> v2:172.21.15.151:3300/0 conn(0x7fe2d415f740 0x7fe2d4073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:16.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.187+0000 7fe2d9ea4640 1 -- 172.21.15.22:0/2572190160 >> 172.21.15.22:0/2572190160 conn(0x7fe2d40724e0 msgr2=0x7fe2d40728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:16.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.187+0000 7fe2d9ea4640 1 -- 172.21.15.22:0/2572190160 shutdown_connections 2024-03-29T04:20:16.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.187+0000 7fe2d9ea4640 1 -- 172.21.15.22:0/2572190160 wait complete. 2024-03-29T04:20:16.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2d9ea4640 1 Processor -- start 2024-03-29T04:20:16.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2d9ea4640 1 -- start start 2024-03-29T04:20:16.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2d9ea4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2d4073540 0x7fe2d415bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:16.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2d9ea4640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe2d40d0740 0x7fe2d413aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:16.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2d9ea4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2d415f740 0x7fe2d413e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:16.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2d9ea4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d406bef0 con 0x7fe2d4073540 2024-03-29T04:20:16.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2d9ea4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d406bd70 con 0x7fe2d415f740 2024-03-29T04:20:16.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2d9ea4640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fe2d416b450 con 0x7fe2d40d0740 2024-03-29T04:20:16.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2d37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2d4073540 0x7fe2d415bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:16.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2d37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2d4073540 0x7fe2d415bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52002/0 (socket says 172.21.15.22:52002) 2024-03-29T04:20:16.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2d37fe640 1 -- 172.21.15.22:0/605666678 learned_addr learned my addr 172.21.15.22:0/605666678 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:16.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2d37fe640 1 -- 172.21.15.22:0/605666678 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe2d40d0740 msgr2=0x7fe2d413aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:16.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2d37fe640 1 --2- 172.21.15.22:0/605666678 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe2d40d0740 0x7fe2d413aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:16.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2d37fe640 1 -- 172.21.15.22:0/605666678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2d415f740 msgr2=0x7fe2d413e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:16.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2d37fe640 1 --2- 172.21.15.22:0/605666678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2d415f740 0x7fe2d413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:16.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2d37fe640 1 -- 172.21.15.22:0/605666678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d413afd0 con 0x7fe2d4073540 2024-03-29T04:20:16.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2d37fe640 1 --2- 172.21.15.22:0/605666678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2d4073540 0x7fe2d415bc20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00ca70 tx=0x7fe2cc00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:16.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2cbfff640 1 -- 172.21.15.22:0/605666678 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc017070 con 0x7fe2d4073540 2024-03-29T04:20:16.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2cbfff640 1 -- 172.21.15.22:0/605666678 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2cc00abd0 con 0x7fe2d4073540 2024-03-29T04:20:16.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2d9ea4640 1 -- 172.21.15.22:0/605666678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d4141b50 con 0x7fe2d4073540 2024-03-29T04:20:16.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2cbfff640 1 -- 172.21.15.22:0/605666678 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc012400 con 0x7fe2d4073540 2024-03-29T04:20:16.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2d9ea4640 1 -- 172.21.15.22:0/605666678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d4142110 con 0x7fe2d4073540 2024-03-29T04:20:16.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2d9ea4640 1 -- 172.21.15.22:0/605666678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2d4068900 con 0x7fe2d4073540 2024-03-29T04:20:16.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.191+0000 7fe2cbfff640 1 -- 172.21.15.22:0/605666678 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe2cc01a070 con 0x7fe2d4073540 2024-03-29T04:20:16.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.195+0000 7fe2cbfff640 1 --2- 172.21.15.22:0/605666678 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe2b4067fc0 0x7fe2b406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:16.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.195+0000 7fe2cbfff640 1 -- 172.21.15.22:0/605666678 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fe2cc0860b0 con 0x7fe2d4073540 2024-03-29T04:20:16.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.195+0000 7fe2d2ffd640 1 --2- 172.21.15.22:0/605666678 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe2b4067fc0 0x7fe2b406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:16.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.195+0000 7fe2d2ffd640 1 --2- 172.21.15.22:0/605666678 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe2b4067fc0 0x7fe2b406a480 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe2d415c160 tx=0x7fe2bc006c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:16.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.199+0000 7fe2cbfff640 1 -- 172.21.15.22:0/605666678 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fe2cc056a60 con 0x7fe2d4073540 2024-03-29T04:20:16.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.367+0000 7fe2d9ea4640 1 -- 172.21.15.22:0/605666678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fe2d406e1b0 con 0x7fe2d4073540 2024-03-29T04:20:16.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.371+0000 7fe2cbfff640 1 -- 172.21.15.22:0/605666678 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fe2cc056360 con 0x7fe2d4073540 2024-03-29T04:20:16.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.375+0000 7fe2d9ea4640 1 -- 172.21.15.22:0/605666678 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe2b4067fc0 msgr2=0x7fe2b406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:16.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.375+0000 7fe2d9ea4640 1 --2- 172.21.15.22:0/605666678 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe2b4067fc0 0x7fe2b406a480 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe2d415c160 tx=0x7fe2bc006c00 comp rx=0 tx=0).stop 2024-03-29T04:20:16.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.375+0000 7fe2d9ea4640 1 -- 172.21.15.22:0/605666678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2d4073540 msgr2=0x7fe2d415bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:16.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.375+0000 7fe2d9ea4640 1 --2- 172.21.15.22:0/605666678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2d4073540 0x7fe2d415bc20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00ca70 tx=0x7fe2cc00cf30 comp rx=0 tx=0).stop 2024-03-29T04:20:16.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.375+0000 7fe2d9ea4640 1 -- 172.21.15.22:0/605666678 shutdown_connections 2024-03-29T04:20:16.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.375+0000 7fe2d9ea4640 1 --2- 172.21.15.22:0/605666678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2d415f740 0x7fe2d413e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:16.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.375+0000 7fe2d9ea4640 1 --2- 172.21.15.22:0/605666678 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fe2d40d0740 0x7fe2d413aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:16.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.375+0000 7fe2d9ea4640 1 --2- 172.21.15.22:0/605666678 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fe2b4067fc0 0x7fe2b406a480 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:16.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.375+0000 7fe2d9ea4640 1 --2- 172.21.15.22:0/605666678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2d4073540 0x7fe2d415bc20 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:16.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.375+0000 7fe2d9ea4640 1 -- 172.21.15.22:0/605666678 >> 172.21.15.22:0/605666678 conn(0x7fe2d40724e0 msgr2=0x7fe2d40cebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:16.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.375+0000 7fe2d9ea4640 1 -- 172.21.15.22:0/605666678 shutdown_connections 2024-03-29T04:20:16.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.375+0000 7fe2d9ea4640 1 -- 172.21.15.22:0/605666678 wait complete. 2024-03-29T04:20:16.373 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining' 2024-03-29T04:20:16.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.519+0000 7f5d1997a640 1 Processor -- start 2024-03-29T04:20:16.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.519+0000 7f5d1997a640 1 -- start start 2024-03-29T04:20:16.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.519+0000 7f5d1997a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5d1415e600 0x7f5d140b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:16.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.519+0000 7f5d1997a640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f5d140d1560 0x7f5d140d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:16.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.519+0000 7f5d1997a640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d140d3e60 con 0x7f5d140d1100 2024-03-29T04:20:16.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.519+0000 7f5d1997a640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f5d140b7450 con 0x7f5d1415e600 2024-03-29T04:20:16.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.519+0000 7f5d1997a640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f5d140d4140 con 0x7f5d140d1560 2024-03-29T04:20:16.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.519+0000 7f5d12ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5d1415e600 0x7f5d140b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:16.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.519+0000 7f5d12ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5d1415e600 0x7f5d140b9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52008/0 (socket says 172.21.15.22:52008) 2024-03-29T04:20:16.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.519+0000 7f5d12ffd640 1 -- 172.21.15.22:0/3659891761 learned_addr learned my addr 172.21.15.22:0/3659891761 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:16.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.523+0000 7f5d137fe640 1 --2- 172.21.15.22:0/3659891761 >> v2:172.21.15.151:3300/0 conn(0x7f5d140d1560 0x7f5d140d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:16.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.523+0000 7f5d12ffd640 1 -- 172.21.15.22:0/3659891761 >> v2:172.21.15.151:3300/0 conn(0x7f5d140d1560 msgr2=0x7f5d140d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:16.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.523+0000 7f5d12ffd640 1 --2- 172.21.15.22:0/3659891761 >> v2:172.21.15.151:3300/0 conn(0x7f5d140d1560 0x7f5d140d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:16.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.523+0000 7f5d12ffd640 1 -- 172.21.15.22:0/3659891761 >> v1:172.21.15.151:6789/0 conn(0x7f5d140d1100 legacy=0x7f5d140b8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:16.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.523+0000 7f5d12ffd640 1 -- 172.21.15.22:0/3659891761 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d140b7680 con 0x7f5d1415e600 2024-03-29T04:20:16.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.523+0000 7f5d137fe640 1 --2- 172.21.15.22:0/3659891761 >> v2:172.21.15.151:3300/0 conn(0x7f5d140d1560 0x7f5d140d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:16.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.523+0000 7f5d12ffd640 1 --2- 172.21.15.22:0/3659891761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d1415e600 0x7f5d140b9f00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5d140d1c90 tx=0x7f5cfc01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ed790565948e53 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:16.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.523+0000 7f5d11ffb640 1 -- 172.21.15.22:0/3659891761 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc0aa610 con 0x7f5d1415e600 2024-03-29T04:20:16.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.523+0000 7f5d11ffb640 1 -- 172.21.15.22:0/3659891761 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cfc0af070 con 0x7f5d1415e600 2024-03-29T04:20:16.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.523+0000 7f5d11ffb640 1 -- 172.21.15.22:0/3659891761 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc0aa9d0 con 0x7f5d1415e600 2024-03-29T04:20:16.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.523+0000 7f5d1997a640 1 -- 172.21.15.22:0/3659891761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d1415e600 msgr2=0x7f5d140b9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:16.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.523+0000 7f5d1997a640 1 --2- 172.21.15.22:0/3659891761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d1415e600 0x7f5d140b9f00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5d140d1c90 tx=0x7f5cfc01c4b0 comp rx=0 tx=0).stop 2024-03-29T04:20:16.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.523+0000 7f5d1997a640 1 -- 172.21.15.22:0/3659891761 shutdown_connections 2024-03-29T04:20:16.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.523+0000 7f5d1997a640 1 --2- 172.21.15.22:0/3659891761 >> v2:172.21.15.151:3300/0 conn(0x7f5d140d1560 0x7f5d140d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:16.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.523+0000 7f5d1997a640 1 --2- 172.21.15.22:0/3659891761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d1415e600 0x7f5d140b9f00 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:16.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.523+0000 7f5d1997a640 1 -- 172.21.15.22:0/3659891761 >> 172.21.15.22:0/3659891761 conn(0x7f5d140b6230 msgr2=0x7f5d140b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:16.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.523+0000 7f5d1997a640 1 -- 172.21.15.22:0/3659891761 shutdown_connections 2024-03-29T04:20:16.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.523+0000 7f5d1997a640 1 -- 172.21.15.22:0/3659891761 wait complete. 2024-03-29T04:20:16.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d1997a640 1 Processor -- start 2024-03-29T04:20:16.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d1997a640 1 -- start start 2024-03-29T04:20:16.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d1997a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d140d1100 0x7f5d1415ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:16.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d1997a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d140d1560 0x7f5d14151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:16.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d1997a640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d1415e600 0x7f5d1414bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:16.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d1997a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5d140b9080 con 0x7f5d140d1560 2024-03-29T04:20:16.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d1997a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5d140d1970 con 0x7f5d140d1100 2024-03-29T04:20:16.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d1997a640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5d140b9200 con 0x7f5d1415e600 2024-03-29T04:20:16.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d127fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d140d1560 0x7f5d14151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:16.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d127fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d140d1560 0x7f5d14151c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52014/0 (socket says 172.21.15.22:52014) 2024-03-29T04:20:16.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d127fc640 1 -- 172.21.15.22:0/735131421 learned_addr learned my addr 172.21.15.22:0/735131421 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:16.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d12ffd640 1 --2- 172.21.15.22:0/735131421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d140d1100 0x7f5d1415ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:16.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d127fc640 1 -- 172.21.15.22:0/735131421 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d1415e600 msgr2=0x7f5d1414bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:16.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d127fc640 1 --2- 172.21.15.22:0/735131421 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d1415e600 0x7f5d1414bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:16.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d127fc640 1 -- 172.21.15.22:0/735131421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d140d1100 msgr2=0x7f5d1415ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:16.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d127fc640 1 --2- 172.21.15.22:0/735131421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d140d1100 0x7f5d1415ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:16.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d127fc640 1 -- 172.21.15.22:0/735131421 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d14160390 con 0x7f5d140d1560 2024-03-29T04:20:16.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d127fc640 1 --2- 172.21.15.22:0/735131421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d140d1560 0x7f5d14151c70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400c450 tx=0x7f5d0400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:16.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d0bfff640 1 -- 172.21.15.22:0/735131421 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04012830 con 0x7f5d140d1560 2024-03-29T04:20:16.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d0bfff640 1 -- 172.21.15.22:0/735131421 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d04017070 con 0x7f5d140d1560 2024-03-29T04:20:16.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d1997a640 1 -- 172.21.15.22:0/735131421 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d1414cf30 con 0x7f5d140d1560 2024-03-29T04:20:16.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d0bfff640 1 -- 172.21.15.22:0/735131421 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04012c30 con 0x7f5d140d1560 2024-03-29T04:20:16.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d1997a640 1 -- 172.21.15.22:0/735131421 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d1414d4f0 con 0x7f5d140d1560 2024-03-29T04:20:16.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d1997a640 1 -- 172.21.15.22:0/735131421 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d14068890 con 0x7f5d140d1560 2024-03-29T04:20:16.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.527+0000 7f5d0bfff640 1 -- 172.21.15.22:0/735131421 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f5d0401c460 con 0x7f5d140d1560 2024-03-29T04:20:16.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.531+0000 7f5d0bfff640 1 --2- 172.21.15.22:0/735131421 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5cf4067fc0 0x7f5cf406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:16.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.531+0000 7f5d0bfff640 1 -- 172.21.15.22:0/735131421 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f5d040860a0 con 0x7f5d140d1560 2024-03-29T04:20:16.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.531+0000 7f5d12ffd640 1 --2- 172.21.15.22:0/735131421 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5cf4067fc0 0x7f5cf406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:16.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.531+0000 7f5d12ffd640 1 --2- 172.21.15.22:0/735131421 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5cf4067fc0 0x7f5cf406a480 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc0ae000 tx=0x7f5cfc0a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:16.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.535+0000 7f5d0bfff640 1 -- 172.21.15.22:0/735131421 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f5d14160390 con 0x7f5d140d1560 2024-03-29T04:20:16.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:16.755+0000 7f5d1997a640 1 -- 172.21.15.22:0/735131421 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]} v 0) -- 0x7f5d1414d730 con 0x7f5d140d1560 2024-03-29T04:20:17.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:17.027+0000 7f5d0bfff640 1 -- 172.21.15.22:0/735131421 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]}]=0 v157) ==== 147+0+0 (secure 0 0 0) 0x7f5d04056a50 con 0x7f5d140d1560 2024-03-29T04:20:17.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:17.027+0000 7f5d1997a640 1 -- 172.21.15.22:0/735131421 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5cf4067fc0 msgr2=0x7f5cf406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:17.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:17.027+0000 7f5d1997a640 1 --2- 172.21.15.22:0/735131421 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5cf4067fc0 0x7f5cf406a480 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc0ae000 tx=0x7f5cfc0a7000 comp rx=0 tx=0).stop 2024-03-29T04:20:17.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:17.027+0000 7f5d1997a640 1 -- 172.21.15.22:0/735131421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d140d1560 msgr2=0x7f5d14151c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:17.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:17.027+0000 7f5d1997a640 1 --2- 172.21.15.22:0/735131421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d140d1560 0x7f5d14151c70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400c450 tx=0x7f5d0400c910 comp rx=0 tx=0).stop 2024-03-29T04:20:17.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:17.027+0000 7f5d1997a640 1 -- 172.21.15.22:0/735131421 shutdown_connections 2024-03-29T04:20:17.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:17.027+0000 7f5d1997a640 1 --2- 172.21.15.22:0/735131421 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5d1415e600 0x7f5d1414bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:17.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:17.027+0000 7f5d1997a640 1 --2- 172.21.15.22:0/735131421 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5cf4067fc0 0x7f5cf406a480 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:17.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:17.027+0000 7f5d1997a640 1 --2- 172.21.15.22:0/735131421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d140d1560 0x7f5d14151c70 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:17.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:17.027+0000 7f5d1997a640 1 --2- 172.21.15.22:0/735131421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d140d1100 0x7f5d1415ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:17.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:17.027+0000 7f5d1997a640 1 -- 172.21.15.22:0/735131421 >> 172.21.15.22:0/735131421 conn(0x7f5d140b6230 msgr2=0x7f5d1415bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:17.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:17.031+0000 7f5d1997a640 1 -- 172.21.15.22:0/735131421 shutdown_connections 2024-03-29T04:20:17.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:17.031+0000 7f5d1997a640 1 -- 172.21.15.22:0/735131421 wait complete. 2024-03-29T04:20:17.033 INFO:tasks.cephfs_test_runner:test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) ... ERROR 2024-03-29T04:20:17.034 INFO:tasks.cephfs_test_runner: 2024-03-29T04:20:17.034 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-29T04:20:17.034 INFO:tasks.cephfs_test_runner:ERROR: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-29T04:20:17.034 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-29T04:20:17.034 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-29T04:20:17.034 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 322, in test_data_uninlining 2024-03-29T04:20:17.034 INFO:tasks.cephfs_test_runner: self.run_test_worker(opt_clear_cache, 2024-03-29T04:20:17.034 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 259, in run_test_worker 2024-03-29T04:20:17.034 INFO:tasks.cephfs_test_runner: info = self.get_inline_data_info(inodes, files, dir_pins, num_mds) 2024-03-29T04:20:17.035 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 116, in get_inline_data_info 2024-03-29T04:20:17.035 INFO:tasks.cephfs_test_runner: json_out = get_inode_dump(inode, 2024-03-29T04:20:17.035 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 101, in get_inode_dump 2024-03-29T04:20:17.035 INFO:tasks.cephfs_test_runner: json_out = self.fs.rank_tell(['dump', 'inode', inode], rank) 2024-03-29T04:20:17.035 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/filesystem.py", line 1332, in rank_tell 2024-03-29T04:20:17.035 INFO:tasks.cephfs_test_runner: return self.mds_tell(*args, mds_id=f"{self.id}:{rank}", **kwargs) 2024-03-29T04:20:17.035 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/filesystem.py", line 1319, in mds_tell 2024-03-29T04:20:17.035 INFO:tasks.cephfs_test_runner: out = self.get_ceph_cmd_stdout("tell", f"mds.{mds_id}", *args, **kwargs) 2024-03-29T04:20:17.035 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-03-29T04:20:17.035 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-03-29T04:20:17.035 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-03-29T04:20:17.035 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-03-29T04:20:17.035 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-03-29T04:20:17.035 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-03-29T04:20:17.035 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-29T04:20:17.035 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-29T04:20:17.035 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 447, in run 2024-03-29T04:20:17.035 INFO:tasks.cephfs_test_runner: r = RemoteProcess(client, args, check_status=check_status, hostname=name, 2024-03-29T04:20:17.036 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 65, in __init__ 2024-03-29T04:20:17.036 INFO:tasks.cephfs_test_runner: self.command = quote(args) 2024-03-29T04:20:17.036 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 257, in quote 2024-03-29T04:20:17.036 INFO:tasks.cephfs_test_runner: return ' '.join(_quote(args)) 2024-03-29T04:20:17.036 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 255, in _quote 2024-03-29T04:20:17.036 INFO:tasks.cephfs_test_runner: yield pipes.quote(a) 2024-03-29T04:20:17.036 INFO:tasks.cephfs_test_runner: File "/usr/lib/python3.8/shlex.py", line 325, in quote 2024-03-29T04:20:17.036 INFO:tasks.cephfs_test_runner: if _find_unsafe(s) is None: 2024-03-29T04:20:17.036 INFO:tasks.cephfs_test_runner:TypeError: expected string or bytes-like object 2024-03-29T04:20:17.036 INFO:tasks.cephfs_test_runner: 2024-03-29T04:20:17.036 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-29T04:20:17.036 INFO:tasks.cephfs_test_runner:Ran 1 test in 112.938s 2024-03-29T04:20:17.036 INFO:tasks.cephfs_test_runner: 2024-03-29T04:20:17.036 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-03-29T04:20:17.036 INFO:tasks.cephfs_test_runner: 2024-03-29T04:20:17.036 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-29T04:20:17.037 INFO:tasks.cephfs_test_runner:ERROR: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-29T04:20:17.037 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-29T04:20:17.037 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-29T04:20:17.037 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 322, in test_data_uninlining 2024-03-29T04:20:17.037 INFO:tasks.cephfs_test_runner: self.run_test_worker(opt_clear_cache, 2024-03-29T04:20:17.037 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 259, in run_test_worker 2024-03-29T04:20:17.037 INFO:tasks.cephfs_test_runner: info = self.get_inline_data_info(inodes, files, dir_pins, num_mds) 2024-03-29T04:20:17.037 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 116, in get_inline_data_info 2024-03-29T04:20:17.037 INFO:tasks.cephfs_test_runner: json_out = get_inode_dump(inode, 2024-03-29T04:20:17.037 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 101, in get_inode_dump 2024-03-29T04:20:17.037 INFO:tasks.cephfs_test_runner: json_out = self.fs.rank_tell(['dump', 'inode', inode], rank) 2024-03-29T04:20:17.037 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/filesystem.py", line 1332, in rank_tell 2024-03-29T04:20:17.037 INFO:tasks.cephfs_test_runner: return self.mds_tell(*args, mds_id=f"{self.id}:{rank}", **kwargs) 2024-03-29T04:20:17.037 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/filesystem.py", line 1319, in mds_tell 2024-03-29T04:20:17.037 INFO:tasks.cephfs_test_runner: out = self.get_ceph_cmd_stdout("tell", f"mds.{mds_id}", *args, **kwargs) 2024-03-29T04:20:17.037 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-03-29T04:20:17.038 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-03-29T04:20:17.038 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-03-29T04:20:17.038 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-03-29T04:20:17.038 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-03-29T04:20:17.038 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-03-29T04:20:17.038 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-29T04:20:17.038 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-29T04:20:17.038 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 447, in run 2024-03-29T04:20:17.038 INFO:tasks.cephfs_test_runner: r = RemoteProcess(client, args, check_status=check_status, hostname=name, 2024-03-29T04:20:17.038 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 65, in __init__ 2024-03-29T04:20:17.038 INFO:tasks.cephfs_test_runner: self.command = quote(args) 2024-03-29T04:20:17.038 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 257, in quote 2024-03-29T04:20:17.038 INFO:tasks.cephfs_test_runner: return ' '.join(_quote(args)) 2024-03-29T04:20:17.038 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 255, in _quote 2024-03-29T04:20:17.038 INFO:tasks.cephfs_test_runner: yield pipes.quote(a) 2024-03-29T04:20:17.038 INFO:tasks.cephfs_test_runner: File "/usr/lib/python3.8/shlex.py", line 325, in quote 2024-03-29T04:20:17.039 INFO:tasks.cephfs_test_runner: if _find_unsafe(s) is None: 2024-03-29T04:20:17.039 INFO:tasks.cephfs_test_runner:TypeError: expected string or bytes-like object 2024-03-29T04:20:17.039 INFO:tasks.cephfs_test_runner: 2024-03-29T04:20:17.039 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-29T04:20:17.279 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=2ae18df486394228bbbfb3b1d334fa2c Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-29T04:20:17.280 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-29T04:20:17.289 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-29T04:20:17.308 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-29T04:20:17.308 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:20:17.308 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:20:17.325 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339616k,nr_inodes=4084904,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275316k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16090 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275312k,nr_inodes=818828,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275316k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T04:20:17.325 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-29T04:20:17.325 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:20:17.372 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339616k,nr_inodes=4084904,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275316k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16090 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275312k,nr_inodes=818828,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275316k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T04:20:17.372 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:20:17.373 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:20:17.379 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336516k,nr_inodes=4084129,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T04:20:17.379 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-29T04:20:17.379 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:20:17.426 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336516k,nr_inodes=4084129,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T04:20:17.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:20:17.426 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-03-29T04:20:17.474 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-29T04:20:17.475 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-29T04:20:17.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:20:17.475 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:20:17.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:20:17.531 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:20:17.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:20:17.626 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link delete ceph-brx 2024-03-29T04:20:17.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:20:17.783 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-03-29T04:20:17.790 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T04:20:17.790 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T04:20:17.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:20:17.790 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:20:17.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:20:17.849 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:20:17.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:20:17.928 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2024-03-29T04:20:18.084 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-29T04:20:18.093 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-29T04:20:18.094 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:20:18.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb080a7d640 1 Processor -- start 2024-03-29T04:20:18.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb080a7d640 1 -- start start 2024-03-29T04:20:18.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb080a7d640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb07c06e2d0 0x7fb07c06b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb080a7d640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb07c0b37c0 con 0x7fb07c0aff30 2024-03-29T04:20:18.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb080a7d640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb07c0b3a30 con 0x7fb07c06bba0 2024-03-29T04:20:18.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb080a7d640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb07c0b4080 con 0x7fb07c06e2d0 2024-03-29T04:20:18.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb07ad9d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb07c0aff30 0x7fb07c06e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:56156/0 (socket says 172.21.15.22:56156) 2024-03-29T04:20:18.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb07ad9d640 1 -- 172.21.15.22:0/1641839391 learned_addr learned my addr 172.21.15.22:0/1641839391 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:18.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb07a59c640 1 --2- 172.21.15.22:0/1641839391 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb07c06e2d0 0x7fb07c06b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:18.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb07959a640 1 -- 172.21.15.22:0/1641839391 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 433958992 0 0) 0x7fb07c0b37c0 con 0x7fb07c0aff30 2024-03-29T04:20:18.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb07959a640 1 -- 172.21.15.22:0/1641839391 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb068004640 con 0x7fb07c0aff30 2024-03-29T04:20:18.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb07959a640 1 -- 172.21.15.22:0/1641839391 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2451387279 0 0) 0x7fb068004640 con 0x7fb07c0aff30 2024-03-29T04:20:18.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb07959a640 1 -- 172.21.15.22:0/1641839391 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb07c06e2d0 msgr2=0x7fb07c06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:18.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb07959a640 1 --2- 172.21.15.22:0/1641839391 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb07c06e2d0 0x7fb07c06b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb07959a640 1 -- 172.21.15.22:0/1641839391 >> v1:172.21.15.151:6789/0 conn(0x7fb07c06bba0 legacy=0x7fb07c0b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:18.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb07959a640 1 -- 172.21.15.22:0/1641839391 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07c0b3d10 con 0x7fb07c0aff30 2024-03-29T04:20:18.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb07959a640 1 -- 172.21.15.22:0/1641839391 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7fb074001a00 con 0x7fb07c0aff30 2024-03-29T04:20:18.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb07959a640 1 -- 172.21.15.22:0/1641839391 >> v1:172.21.15.22:6789/0 conn(0x7fb07c0aff30 legacy=0x7fb07c06e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:18.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb07959a640 1 --2- 172.21.15.22:0/1641839391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb068004b00 0x7fb068004f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb07959a640 1 --2- 172.21.15.22:0/1641839391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb068005920 0x7fb068005da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb07959a640 1 --2- 172.21.15.22:0/1641839391 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb07c06e2d0 0x7fb0680062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb07959a640 1 -- 172.21.15.22:0/1641839391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c0b4080 con 0x7fb068005920 2024-03-29T04:20:18.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb07959a640 1 -- 172.21.15.22:0/1641839391 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb06800cb90 con 0x7fb068004b00 2024-03-29T04:20:18.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb07959a640 1 -- 172.21.15.22:0/1641839391 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb0680034d0 con 0x7fb07c06e2d0 2024-03-29T04:20:18.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb079d9b640 1 --2- 172.21.15.22:0/1641839391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb068005920 0x7fb068005da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:18.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb07ad9d640 1 --2- 172.21.15.22:0/1641839391 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb07c06e2d0 0x7fb0680062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:18.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb07a59c640 1 --2- 172.21.15.22:0/1641839391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb068004b00 0x7fb068004f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:18.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb079d9b640 1 -- 172.21.15.22:0/1641839391 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb07c06e2d0 msgr2=0x7fb0680062e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:18.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb079d9b640 1 --2- 172.21.15.22:0/1641839391 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb07c06e2d0 0x7fb0680062e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb079d9b640 1 -- 172.21.15.22:0/1641839391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb068004b00 msgr2=0x7fb068004f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:18.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb079d9b640 1 --2- 172.21.15.22:0/1641839391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb068004b00 0x7fb068004f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.243+0000 7fb079d9b640 1 -- 172.21.15.22:0/1641839391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb068004640 con 0x7fb068005920 2024-03-29T04:20:18.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.247+0000 7fb079d9b640 1 --2- 172.21.15.22:0/1641839391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb068005920 0x7fb068005da0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb07c06bfa0 tx=0x7fb06c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6bb7896a11dee2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:18.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.247+0000 7fb07959a640 1 -- 172.21.15.22:0/1641839391 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c0013a0 con 0x7fb068005920 2024-03-29T04:20:18.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.247+0000 7fb07959a640 1 -- 172.21.15.22:0/1641839391 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb06c0052f0 con 0x7fb068005920 2024-03-29T04:20:18.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.247+0000 7fb07959a640 1 -- 172.21.15.22:0/1641839391 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c0a4e60 con 0x7fb068005920 2024-03-29T04:20:18.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.247+0000 7fb080a7d640 1 -- 172.21.15.22:0/1641839391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb068005920 msgr2=0x7fb068005da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:18.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.247+0000 7fb080a7d640 1 --2- 172.21.15.22:0/1641839391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb068005920 0x7fb068005da0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb07c06bfa0 tx=0x7fb06c01c3a0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.247+0000 7fb07ad9d640 1 -- 172.21.15.22:0/1641839391 reap_dead start 2024-03-29T04:20:18.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.247+0000 7fb080a7d640 1 -- 172.21.15.22:0/1641839391 shutdown_connections 2024-03-29T04:20:18.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.247+0000 7fb080a7d640 1 -- 172.21.15.22:0/1641839391 >> 172.21.15.22:0/1641839391 conn(0x7fb07c0b2b30 msgr2=0x7fb07c0d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:18.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.247+0000 7fb080a7d640 1 -- 172.21.15.22:0/1641839391 shutdown_connections 2024-03-29T04:20:18.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.247+0000 7fb080a7d640 1 -- 172.21.15.22:0/1641839391 wait complete. 2024-03-29T04:20:18.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.247+0000 7fb080a7d640 1 Processor -- start 2024-03-29T04:20:18.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.247+0000 7fb080a7d640 1 -- start start 2024-03-29T04:20:18.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb080a7d640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb068005920 0x7fb07c14c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb080a7d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb07c157ab0 0x7fb07c0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb080a7d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb07c0cf640 0x7fb07c0cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb080a7d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c1587d0 con 0x7fb07c0cf640 2024-03-29T04:20:18.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb080a7d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c158650 con 0x7fb07c157ab0 2024-03-29T04:20:18.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb080a7d640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb07c15b800 con 0x7fb068005920 2024-03-29T04:20:18.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb07ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb07c0cf640 0x7fb07c0cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:18.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb07ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb07c0cf640 0x7fb07c0cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52026/0 (socket says 172.21.15.22:52026) 2024-03-29T04:20:18.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb07ad9d640 1 -- 172.21.15.22:0/3115091741 learned_addr learned my addr 172.21.15.22:0/3115091741 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:18.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb07a59c640 1 --2- 172.21.15.22:0/3115091741 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb068005920 0x7fb07c14c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:18.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb079d9b640 1 --2- 172.21.15.22:0/3115091741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb07c157ab0 0x7fb07c0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:18.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb07ad9d640 1 -- 172.21.15.22:0/3115091741 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb068005920 msgr2=0x7fb07c14c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:18.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb07ad9d640 1 --2- 172.21.15.22:0/3115091741 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb068005920 0x7fb07c14c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb07ad9d640 1 -- 172.21.15.22:0/3115091741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb07c157ab0 msgr2=0x7fb07c0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:18.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb07ad9d640 1 --2- 172.21.15.22:0/3115091741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb07c157ab0 0x7fb07c0ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb07ad9d640 1 -- 172.21.15.22:0/3115091741 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07c0a9670 con 0x7fb07c0cf640 2024-03-29T04:20:18.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb07ad9d640 1 --2- 172.21.15.22:0/3115091741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb07c0cf640 0x7fb07c0cedf0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0b0360 tx=0x7fb074011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:18.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb072ffd640 1 -- 172.21.15.22:0/3115091741 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb074010040 con 0x7fb07c0cf640 2024-03-29T04:20:18.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb072ffd640 1 -- 172.21.15.22:0/3115091741 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb07401a070 con 0x7fb07c0cf640 2024-03-29T04:20:18.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb080a7d640 1 -- 172.21.15.22:0/3115091741 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb07c0cfb40 con 0x7fb07c0cf640 2024-03-29T04:20:18.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb072ffd640 1 -- 172.21.15.22:0/3115091741 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07400fbf0 con 0x7fb07c0cf640 2024-03-29T04:20:18.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb080a7d640 1 -- 172.21.15.22:0/3115091741 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb07c14c820 con 0x7fb07c0cf640 2024-03-29T04:20:18.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb080a7d640 1 -- 172.21.15.22:0/3115091741 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb07c068890 con 0x7fb07c0cf640 2024-03-29T04:20:18.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb072ffd640 1 -- 172.21.15.22:0/3115091741 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb07401f460 con 0x7fb07c0cf640 2024-03-29T04:20:18.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb072ffd640 1 --2- 172.21.15.22:0/3115091741 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb05c067fc0 0x7fb05c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb072ffd640 1 -- 172.21.15.22:0/3115091741 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fb074089260 con 0x7fb07c0cf640 2024-03-29T04:20:18.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.251+0000 7fb07a59c640 1 --2- 172.21.15.22:0/3115091741 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb05c067fc0 0x7fb05c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:18.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.255+0000 7fb07a59c640 1 --2- 172.21.15.22:0/3115091741 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb05c067fc0 0x7fb05c06a480 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb064009a30 tx=0x7fb064008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:18.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.255+0000 7fb072ffd640 1 -- 172.21.15.22:0/3115091741 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fb074059b80 con 0x7fb07c0cf640 2024-03-29T04:20:18.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.415+0000 7fb080a7d640 1 -- 172.21.15.22:0/3115091741 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb07c06e0b0 con 0x7fb05c067fc0 2024-03-29T04:20:18.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.419+0000 7fb072ffd640 1 -- 172.21.15.22:0/3115091741 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13500 (secure 0 0 0) 0x7fb07c06e0b0 con 0x7fb05c067fc0 2024-03-29T04:20:18.399 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:20:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-29T04:20:18.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.419+0000 7fb080a7d640 1 -- 172.21.15.22:0/3115091741 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb05c067fc0 msgr2=0x7fb05c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:18.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.419+0000 7fb080a7d640 1 --2- 172.21.15.22:0/3115091741 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb05c067fc0 0x7fb05c06a480 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb064009a30 tx=0x7fb064008040 comp rx=0 tx=0).stop 2024-03-29T04:20:18.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.419+0000 7fb080a7d640 1 -- 172.21.15.22:0/3115091741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb07c0cf640 msgr2=0x7fb07c0cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.419+0000 7fb080a7d640 1 --2- 172.21.15.22:0/3115091741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb07c0cf640 0x7fb07c0cedf0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0b0360 tx=0x7fb074011c20 comp rx=0 tx=0).stop 2024-03-29T04:20:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.419+0000 7fb080a7d640 1 -- 172.21.15.22:0/3115091741 shutdown_connections 2024-03-29T04:20:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.419+0000 7fb080a7d640 1 --2- 172.21.15.22:0/3115091741 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb05c067fc0 0x7fb05c06a480 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.419+0000 7fb080a7d640 1 --2- 172.21.15.22:0/3115091741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb07c0cf640 0x7fb07c0cedf0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.419+0000 7fb080a7d640 1 --2- 172.21.15.22:0/3115091741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb07c157ab0 0x7fb07c0ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.419+0000 7fb080a7d640 1 --2- 172.21.15.22:0/3115091741 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb068005920 0x7fb07c14c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.419+0000 7fb080a7d640 1 -- 172.21.15.22:0/3115091741 >> 172.21.15.22:0/3115091741 conn(0x7fb07c0b2b30 msgr2=0x7fb07c1e4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.419+0000 7fb080a7d640 1 -- 172.21.15.22:0/3115091741 shutdown_connections 2024-03-29T04:20:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.419+0000 7fb080a7d640 1 -- 172.21.15.22:0/3115091741 wait complete. 2024-03-29T04:20:18.418 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-03-29T04:20:17.749589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":250624,"kb_used_data":5320,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754724096,"statfs":{"total":773094113280,"available":772837474304,"internally_reserved":0,"allocated":5447680,"data_stored":3048200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-174309431,"num_objects":-138,"num_object_clones":0,"num_object_copies":-276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-138,"num_whiteouts":0,"num_read":-43,"num_read_kb":-2,"num_write":-330,"num_write_kb":-170315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001609"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":217,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-29T04:20:16.100438+0000","last_change":"2024-03-29T04:17:25.361826+0000","last_active":"2024-03-29T04:20:16.100438+0000","last_peered":"2024-03-29T04:20:16.100438+0000","last_clean":"2024-03-29T04:20:16.100438+0000","last_became_active":"2024-03-29T04:17:25.361259+0000","last_became_peered":"2024-03-29T04:17:25.361259+0000","last_unstale":"2024-03-29T04:20:16.100438+0000","last_undegraded":"2024-03-29T04:20:16.100438+0000","last_fullsized":"2024-03-29T04:20:16.100438+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_clean_scrub_stamp":"2024-03-29T04:17:24.334639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:40.027873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":823801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35860,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335980,"statfs":{"total":96636764160,"available":96600043520,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35864,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335976,"statfs":{"total":96636764160,"available":96600039424,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31704,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":823801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:20:18.419 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:20:18.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.563+0000 7faaf038e640 1 Processor -- start 2024-03-29T04:20:18.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.563+0000 7faaf038e640 1 -- start start 2024-03-29T04:20:18.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.563+0000 7faaf038e640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7faae80b41f0 0x7faae80cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.563+0000 7faaf038e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7faae806ca50 0x7faae80ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.563+0000 7faaf038e640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faae80ccef0 con 0x7faae806c670 2024-03-29T04:20:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.563+0000 7faaf038e640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7faae80b34c0 con 0x7faae806ca50 2024-03-29T04:20:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.563+0000 7faaf038e640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7faae80b3710 con 0x7faae80b41f0 2024-03-29T04:20:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.563+0000 7faaee92b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7faae806ca50 0x7faae80ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.563+0000 7faaee92b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7faae806ca50 0x7faae80ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52034/0 (socket says 172.21.15.22:52034) 2024-03-29T04:20:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.563+0000 7faaee92b640 1 -- 172.21.15.22:0/3276538678 learned_addr learned my addr 172.21.15.22:0/3276538678 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.563+0000 7faaed929640 1 --2- 172.21.15.22:0/3276538678 >> v2:172.21.15.151:3300/0 conn(0x7faae80b41f0 0x7faae80cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.563+0000 7faaee92b640 1 -- 172.21.15.22:0/3276538678 >> v2:172.21.15.151:3300/0 conn(0x7faae80b41f0 msgr2=0x7faae80cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.563+0000 7faaee92b640 1 --2- 172.21.15.22:0/3276538678 >> v2:172.21.15.151:3300/0 conn(0x7faae80b41f0 0x7faae80cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.563+0000 7faaee92b640 1 -- 172.21.15.22:0/3276538678 >> v1:172.21.15.151:6789/0 conn(0x7faae806c670 legacy=0x7faae806e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.563+0000 7faaee92b640 1 -- 172.21.15.22:0/3276538678 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae80b39f0 con 0x7faae806ca50 2024-03-29T04:20:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaee92b640 1 --2- 172.21.15.22:0/3276538678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae806ca50 0x7faae80ce5b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7faae806ebe0 tx=0x7faae001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eafcf0f95c8d4d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaed128640 1 -- 172.21.15.22:0/3276538678 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae00a5780 con 0x7faae806ca50 2024-03-29T04:20:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaed128640 1 -- 172.21.15.22:0/3276538678 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faae00a5920 con 0x7faae806ca50 2024-03-29T04:20:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaed128640 1 -- 172.21.15.22:0/3276538678 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae00a5cf0 con 0x7faae806ca50 2024-03-29T04:20:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaf038e640 1 -- 172.21.15.22:0/3276538678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae806ca50 msgr2=0x7faae80ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaf038e640 1 --2- 172.21.15.22:0/3276538678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae806ca50 0x7faae80ce5b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7faae806ebe0 tx=0x7faae001c9b0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaf038e640 1 -- 172.21.15.22:0/3276538678 shutdown_connections 2024-03-29T04:20:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaf038e640 1 --2- 172.21.15.22:0/3276538678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae806ca50 0x7faae80ce5b0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaf038e640 1 --2- 172.21.15.22:0/3276538678 >> v2:172.21.15.151:3300/0 conn(0x7faae80b41f0 0x7faae80cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaf038e640 1 -- 172.21.15.22:0/3276538678 >> 172.21.15.22:0/3276538678 conn(0x7faae80cfee0 msgr2=0x7faae80d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaf038e640 1 -- 172.21.15.22:0/3276538678 shutdown_connections 2024-03-29T04:20:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaf038e640 1 -- 172.21.15.22:0/3276538678 wait complete. 2024-03-29T04:20:18.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaf038e640 1 Processor -- start 2024-03-29T04:20:18.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaf038e640 1 -- start start 2024-03-29T04:20:18.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaf038e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae806c670 0x7faae8165280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaf038e640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faae806ca50 0x7faae81666f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaf038e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae80b41f0 0x7faae814e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaf038e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faae8169070 con 0x7faae806c670 2024-03-29T04:20:18.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaf038e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faae8168ef0 con 0x7faae80b41f0 2024-03-29T04:20:18.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaf038e640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7faae81691f0 con 0x7faae806ca50 2024-03-29T04:20:18.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaee12a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae806c670 0x7faae8165280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:18.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaee12a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae806c670 0x7faae8165280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52050/0 (socket says 172.21.15.22:52050) 2024-03-29T04:20:18.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.567+0000 7faaee12a640 1 -- 172.21.15.22:0/3118698044 learned_addr learned my addr 172.21.15.22:0/3118698044 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:18.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.571+0000 7faaee92b640 1 --2- 172.21.15.22:0/3118698044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae80b41f0 0x7faae814e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:18.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.571+0000 7faaed929640 1 --2- 172.21.15.22:0/3118698044 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faae806ca50 0x7faae81666f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:18.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.571+0000 7faaee12a640 1 -- 172.21.15.22:0/3118698044 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faae806ca50 msgr2=0x7faae81666f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:18.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.571+0000 7faaee12a640 1 --2- 172.21.15.22:0/3118698044 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faae806ca50 0x7faae81666f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.571+0000 7faaee12a640 1 -- 172.21.15.22:0/3118698044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae80b41f0 msgr2=0x7faae814e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:18.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.571+0000 7faaee12a640 1 --2- 172.21.15.22:0/3118698044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae80b41f0 0x7faae814e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.571+0000 7faaee12a640 1 -- 172.21.15.22:0/3118698044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae8165860 con 0x7faae806c670 2024-03-29T04:20:18.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.571+0000 7faaee12a640 1 --2- 172.21.15.22:0/3118698044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae806c670 0x7faae8165280 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7faae400b410 tx=0x7faae400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:18.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.571+0000 7faadeffd640 1 -- 172.21.15.22:0/3118698044 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4015030 con 0x7faae806c670 2024-03-29T04:20:18.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.571+0000 7faadeffd640 1 -- 172.21.15.22:0/3118698044 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faae4004020 con 0x7faae806c670 2024-03-29T04:20:18.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.571+0000 7faadeffd640 1 -- 172.21.15.22:0/3118698044 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4004450 con 0x7faae806c670 2024-03-29T04:20:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.571+0000 7faaf038e640 1 -- 172.21.15.22:0/3118698044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae8165b40 con 0x7faae806c670 2024-03-29T04:20:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.571+0000 7faaf038e640 1 -- 172.21.15.22:0/3118698044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae8166100 con 0x7faae806c670 2024-03-29T04:20:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.571+0000 7faaf038e640 1 -- 172.21.15.22:0/3118698044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae8068890 con 0x7faae806c670 2024-03-29T04:20:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.571+0000 7faadeffd640 1 -- 172.21.15.22:0/3118698044 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7faae4018070 con 0x7faae806c670 2024-03-29T04:20:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.571+0000 7faadeffd640 1 --2- 172.21.15.22:0/3118698044 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faacc067ff0 0x7faacc06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.571+0000 7faadeffd640 1 -- 172.21.15.22:0/3118698044 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7faae4085da0 con 0x7faae806c670 2024-03-29T04:20:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.571+0000 7faaed929640 1 --2- 172.21.15.22:0/3118698044 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faacc067ff0 0x7faacc06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.575+0000 7faaed929640 1 --2- 172.21.15.22:0/3118698044 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faacc067ff0 0x7faacc06a4b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faae81442d0 tx=0x7faad8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.579+0000 7faadeffd640 1 -- 172.21.15.22:0/3118698044 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7faae4056750 con 0x7faae806c670 2024-03-29T04:20:18.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.739+0000 7faaf038e640 1 -- 172.21.15.22:0/3118698044 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faae80b32a0 con 0x7faacc067ff0 2024-03-29T04:20:18.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.739+0000 7faadeffd640 1 -- 172.21.15.22:0/3118698044 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13500 (secure 0 0 0) 0x7faae80b32a0 con 0x7faacc067ff0 2024-03-29T04:20:18.721 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:20:18.722 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-29T04:20:18.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.743+0000 7faaf038e640 1 -- 172.21.15.22:0/3118698044 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faacc067ff0 msgr2=0x7faacc06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:18.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.743+0000 7faaf038e640 1 --2- 172.21.15.22:0/3118698044 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faacc067ff0 0x7faacc06a4b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faae81442d0 tx=0x7faad8008040 comp rx=0 tx=0).stop 2024-03-29T04:20:18.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.743+0000 7faaf038e640 1 -- 172.21.15.22:0/3118698044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae806c670 msgr2=0x7faae8165280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:18.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.743+0000 7faaf038e640 1 --2- 172.21.15.22:0/3118698044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae806c670 0x7faae8165280 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7faae400b410 tx=0x7faae400b8d0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.743+0000 7faaf038e640 1 -- 172.21.15.22:0/3118698044 shutdown_connections 2024-03-29T04:20:18.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.743+0000 7faaf038e640 1 --2- 172.21.15.22:0/3118698044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae80b41f0 0x7faae814e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.743+0000 7faaf038e640 1 --2- 172.21.15.22:0/3118698044 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faae806ca50 0x7faae81666f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.743+0000 7faaf038e640 1 --2- 172.21.15.22:0/3118698044 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faacc067ff0 0x7faacc06a4b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.743+0000 7faaf038e640 1 --2- 172.21.15.22:0/3118698044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae806c670 0x7faae8165280 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.743+0000 7faaf038e640 1 -- 172.21.15.22:0/3118698044 >> 172.21.15.22:0/3118698044 conn(0x7faae80cfee0 msgr2=0x7faae8144650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:18.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.743+0000 7faaf038e640 1 -- 172.21.15.22:0/3118698044 shutdown_connections 2024-03-29T04:20:18.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.743+0000 7faaf038e640 1 -- 172.21.15.22:0/3118698044 wait complete. 2024-03-29T04:20:18.741 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-03-29T04:20:17.749589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":250624,"kb_used_data":5320,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754724096,"statfs":{"total":773094113280,"available":772837474304,"internally_reserved":0,"allocated":5447680,"data_stored":3048200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-174309431,"num_objects":-138,"num_object_clones":0,"num_object_copies":-276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-138,"num_whiteouts":0,"num_read":-43,"num_read_kb":-2,"num_write":-330,"num_write_kb":-170315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001609"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":217,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-29T04:20:16.100438+0000","last_change":"2024-03-29T04:17:25.361826+0000","last_active":"2024-03-29T04:20:16.100438+0000","last_peered":"2024-03-29T04:20:16.100438+0000","last_clean":"2024-03-29T04:20:16.100438+0000","last_became_active":"2024-03-29T04:17:25.361259+0000","last_became_peered":"2024-03-29T04:17:25.361259+0000","last_unstale":"2024-03-29T04:20:16.100438+0000","last_undegraded":"2024-03-29T04:20:16.100438+0000","last_fullsized":"2024-03-29T04:20:16.100438+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_clean_scrub_stamp":"2024-03-29T04:17:24.334639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:40.027873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":823801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35860,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335980,"statfs":{"total":96636764160,"available":96600043520,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35864,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335976,"statfs":{"total":96636764160,"available":96600039424,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31704,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":823801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:20:18.741 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-29T04:20:18.741 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:20:18.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bc7258640 1 Processor -- start 2024-03-29T04:20:18.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bc7258640 1 -- start start 2024-03-29T04:20:18.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bc7258640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc006d780 0x7f0bc006dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bc7258640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc0141970 con 0x7f0bc00cfa20 2024-03-29T04:20:18.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bc7258640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc0141c50 con 0x7f0bc013f0a0 2024-03-29T04:20:18.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bc7258640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc006e100 con 0x7f0bc006d780 2024-03-29T04:20:18.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bc57f5640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f0bc00cfa20 0x7f0bc00cfe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:56160/0 (socket says 172.21.15.22:56160) 2024-03-29T04:20:18.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bc57f5640 1 -- 172.21.15.22:0/1017916246 learned_addr learned my addr 172.21.15.22:0/1017916246 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:18.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bbffff640 1 --2- 172.21.15.22:0/1017916246 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc006d780 0x7f0bc006dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:18.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bbf7fe640 1 -- 172.21.15.22:0/1017916246 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3436089423 0 0) 0x7f0bc0141970 con 0x7f0bc00cfa20 2024-03-29T04:20:18.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bbf7fe640 1 -- 172.21.15.22:0/1017916246 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bac003530 con 0x7f0bc00cfa20 2024-03-29T04:20:18.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bbf7fe640 1 -- 172.21.15.22:0/1017916246 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3800010539 0 0) 0x7f0bac003530 con 0x7f0bc00cfa20 2024-03-29T04:20:18.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bbf7fe640 1 -- 172.21.15.22:0/1017916246 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc006d780 msgr2=0x7f0bc006dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:18.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bbf7fe640 1 --2- 172.21.15.22:0/1017916246 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc006d780 0x7f0bc006dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bbf7fe640 1 -- 172.21.15.22:0/1017916246 >> v1:172.21.15.151:6789/0 conn(0x7f0bc013f0a0 legacy=0x7f0bc0160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:18.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bbf7fe640 1 -- 172.21.15.22:0/1017916246 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc0141f30 con 0x7f0bc00cfa20 2024-03-29T04:20:18.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bbf7fe640 1 -- 172.21.15.22:0/1017916246 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f0bb0001a00 con 0x7f0bc00cfa20 2024-03-29T04:20:18.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bbf7fe640 1 -- 172.21.15.22:0/1017916246 >> v1:172.21.15.22:6789/0 conn(0x7f0bc00cfa20 legacy=0x7f0bc00cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:18.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bbf7fe640 1 --2- 172.21.15.22:0/1017916246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bac0039f0 0x7f0bac003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bbf7fe640 1 --2- 172.21.15.22:0/1017916246 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc006d780 0x7f0bac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bbf7fe640 1 --2- 172.21.15.22:0/1017916246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bac004d50 0x7f0bac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bbf7fe640 1 -- 172.21.15.22:0/1017916246 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc006e100 con 0x7f0bac0039f0 2024-03-29T04:20:18.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bbf7fe640 1 -- 172.21.15.22:0/1017916246 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0bac00bab0 con 0x7f0bac004d50 2024-03-29T04:20:18.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bbf7fe640 1 -- 172.21.15.22:0/1017916246 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0bac00bcb0 con 0x7f0bc006d780 2024-03-29T04:20:18.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bc4ff4640 1 --2- 172.21.15.22:0/1017916246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bac0039f0 0x7f0bac003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:18.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bbffff640 1 --2- 172.21.15.22:0/1017916246 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc006d780 0x7f0bac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:18.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bc4ff4640 1 -- 172.21.15.22:0/1017916246 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc006d780 msgr2=0x7f0bac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:18.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bc57f5640 1 --2- 172.21.15.22:0/1017916246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bac004d50 0x7f0bac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:18.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bc4ff4640 1 --2- 172.21.15.22:0/1017916246 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc006d780 0x7f0bac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bc4ff4640 1 -- 172.21.15.22:0/1017916246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bac004d50 msgr2=0x7f0bac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:18.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bc4ff4640 1 --2- 172.21.15.22:0/1017916246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bac004d50 0x7f0bac0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bc4ff4640 1 -- 172.21.15.22:0/1017916246 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bac003530 con 0x7f0bac0039f0 2024-03-29T04:20:18.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.887+0000 7f0bbffff640 1 --2- 172.21.15.22:0/1017916246 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc006d780 0x7f0bac004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:18.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.891+0000 7f0bc4ff4640 1 --2- 172.21.15.22:0/1017916246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bac0039f0 0x7f0bac003e10 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8001750 tx=0x7f0bb801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b1883af609f30b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:18.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.891+0000 7f0bbf7fe640 1 -- 172.21.15.22:0/1017916246 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb80013a0 con 0x7f0bac0039f0 2024-03-29T04:20:18.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.891+0000 7f0bbf7fe640 1 -- 172.21.15.22:0/1017916246 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb80050d0 con 0x7f0bac0039f0 2024-03-29T04:20:18.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.891+0000 7f0bbf7fe640 1 -- 172.21.15.22:0/1017916246 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb80054a0 con 0x7f0bac0039f0 2024-03-29T04:20:18.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.891+0000 7f0bc7258640 1 -- 172.21.15.22:0/1017916246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bac0039f0 msgr2=0x7f0bac003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:18.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.891+0000 7f0bc7258640 1 --2- 172.21.15.22:0/1017916246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bac0039f0 0x7f0bac003e10 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8001750 tx=0x7f0bb801c3b0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.891+0000 7f0bc57f5640 1 -- 172.21.15.22:0/1017916246 reap_dead start 2024-03-29T04:20:18.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.891+0000 7f0bc7258640 1 -- 172.21.15.22:0/1017916246 shutdown_connections 2024-03-29T04:20:18.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.891+0000 7f0bc7258640 1 -- 172.21.15.22:0/1017916246 >> 172.21.15.22:0/1017916246 conn(0x7f0bc0072500 msgr2=0x7f0bc0072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:18.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.891+0000 7f0bc7258640 1 -- 172.21.15.22:0/1017916246 shutdown_connections 2024-03-29T04:20:18.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.891+0000 7f0bc7258640 1 -- 172.21.15.22:0/1017916246 wait complete. 2024-03-29T04:20:18.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.891+0000 7f0bc7258640 1 Processor -- start 2024-03-29T04:20:18.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.891+0000 7f0bc7258640 1 -- start start 2024-03-29T04:20:18.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.891+0000 7f0bc7258640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bc0137500 0x7f0bc00b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.891+0000 7f0bc7258640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc00701e0 0x7f0bc0162400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bc7258640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc015a6e0 0x7f0bc015ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bc7258640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc013f480 con 0x7f0bc0137500 2024-03-29T04:20:18.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bc7258640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc015ff20 con 0x7f0bc015a6e0 2024-03-29T04:20:18.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bc7258640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc01600a0 con 0x7f0bc00701e0 2024-03-29T04:20:18.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bc4ff4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bc0137500 0x7f0bc00b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:18.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bc4ff4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bc0137500 0x7f0bc00b0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52074/0 (socket says 172.21.15.22:52074) 2024-03-29T04:20:18.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bc4ff4640 1 -- 172.21.15.22:0/1583020681 learned_addr learned my addr 172.21.15.22:0/1583020681 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:18.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bc57f5640 1 --2- 172.21.15.22:0/1583020681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc015a6e0 0x7f0bc015ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:18.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bc4ff4640 1 -- 172.21.15.22:0/1583020681 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc00701e0 msgr2=0x7f0bc0162400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:18.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bc4ff4640 1 --2- 172.21.15.22:0/1583020681 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc00701e0 0x7f0bc0162400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bc4ff4640 1 -- 172.21.15.22:0/1583020681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc015a6e0 msgr2=0x7f0bc015ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:18.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bc4ff4640 1 --2- 172.21.15.22:0/1583020681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc015a6e0 0x7f0bc015ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:18.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bc4ff4640 1 -- 172.21.15.22:0/1583020681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc00b10c0 con 0x7f0bc0137500 2024-03-29T04:20:18.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bc4ff4640 1 --2- 172.21.15.22:0/1583020681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bc0137500 0x7f0bc00b0ae0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8005040 tx=0x7f0bac005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:18.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bbd7fa640 1 -- 172.21.15.22:0/1583020681 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb80b8070 con 0x7f0bc0137500 2024-03-29T04:20:18.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bbd7fa640 1 -- 172.21.15.22:0/1583020681 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb8007220 con 0x7f0bc0137500 2024-03-29T04:20:18.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bc7258640 1 -- 172.21.15.22:0/1583020681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc00b13a0 con 0x7f0bc0137500 2024-03-29T04:20:18.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bbd7fa640 1 -- 172.21.15.22:0/1583020681 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb80075f0 con 0x7f0bc0137500 2024-03-29T04:20:18.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bc7258640 1 -- 172.21.15.22:0/1583020681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc0162940 con 0x7f0bc0137500 2024-03-29T04:20:18.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bc7258640 1 -- 172.21.15.22:0/1583020681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc0068890 con 0x7f0bc0137500 2024-03-29T04:20:18.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bbd7fa640 1 -- 172.21.15.22:0/1583020681 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f0bb80067e0 con 0x7f0bc0137500 2024-03-29T04:20:18.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bbd7fa640 1 --2- 172.21.15.22:0/1583020681 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0ba0067ff0 0x7f0ba006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:18.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bbd7fa640 1 -- 172.21.15.22:0/1583020681 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f0bb811ff70 con 0x7f0bc0137500 2024-03-29T04:20:18.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bbffff640 1 --2- 172.21.15.22:0/1583020681 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0ba0067ff0 0x7f0ba006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:18.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.895+0000 7f0bbffff640 1 --2- 172.21.15.22:0/1583020681 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0ba0067ff0 0x7f0ba006a4b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0bc01f89e0 tx=0x7f0ba8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:18.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:18.899+0000 7f0bbd7fa640 1 -- 172.21.15.22:0/1583020681 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f0bb80160f0 con 0x7f0bc0137500 2024-03-29T04:20:19.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.063+0000 7f0bc7258640 1 -- 172.21.15.22:0/1583020681 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0bc01416b0 con 0x7f0ba0067ff0 2024-03-29T04:20:19.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.063+0000 7f0bbd7fa640 1 -- 172.21.15.22:0/1583020681 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13500 (secure 0 0 0) 0x7f0bc01416b0 con 0x7f0ba0067ff0 2024-03-29T04:20:19.045 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:20:19.045 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-29T04:20:19.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.067+0000 7f0bc7258640 1 -- 172.21.15.22:0/1583020681 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0ba0067ff0 msgr2=0x7f0ba006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:19.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.067+0000 7f0bc7258640 1 --2- 172.21.15.22:0/1583020681 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0ba0067ff0 0x7f0ba006a4b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0bc01f89e0 tx=0x7f0ba8008040 comp rx=0 tx=0).stop 2024-03-29T04:20:19.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.067+0000 7f0bc7258640 1 -- 172.21.15.22:0/1583020681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bc0137500 msgr2=0x7f0bc00b0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:19.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.067+0000 7f0bc7258640 1 --2- 172.21.15.22:0/1583020681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bc0137500 0x7f0bc00b0ae0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8005040 tx=0x7f0bac005840 comp rx=0 tx=0).stop 2024-03-29T04:20:19.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.067+0000 7f0bc7258640 1 -- 172.21.15.22:0/1583020681 shutdown_connections 2024-03-29T04:20:19.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.067+0000 7f0bc7258640 1 --2- 172.21.15.22:0/1583020681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc015a6e0 0x7f0bc015ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:19.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.067+0000 7f0bc7258640 1 --2- 172.21.15.22:0/1583020681 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f0bc00701e0 0x7f0bc0162400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:19.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.067+0000 7f0bc7258640 1 --2- 172.21.15.22:0/1583020681 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f0ba0067ff0 0x7f0ba006a4b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:19.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.067+0000 7f0bc7258640 1 --2- 172.21.15.22:0/1583020681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bc0137500 0x7f0bc00b0ae0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:19.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.067+0000 7f0bc7258640 1 -- 172.21.15.22:0/1583020681 >> 172.21.15.22:0/1583020681 conn(0x7f0bc0072500 msgr2=0x7f0bc015f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:19.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.067+0000 7f0bc7258640 1 -- 172.21.15.22:0/1583020681 shutdown_connections 2024-03-29T04:20:19.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.067+0000 7f0bc7258640 1 -- 172.21.15.22:0/1583020681 wait complete. 2024-03-29T04:20:19.065 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-03-29T04:20:17.749589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":250624,"kb_used_data":5320,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754724096,"statfs":{"total":773094113280,"available":772837474304,"internally_reserved":0,"allocated":5447680,"data_stored":3048200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-174309431,"num_objects":-138,"num_object_clones":0,"num_object_copies":-276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-138,"num_whiteouts":0,"num_read":-43,"num_read_kb":-2,"num_write":-330,"num_write_kb":-170315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001609"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":217,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-29T04:20:16.100438+0000","last_change":"2024-03-29T04:17:25.361826+0000","last_active":"2024-03-29T04:20:16.100438+0000","last_peered":"2024-03-29T04:20:16.100438+0000","last_clean":"2024-03-29T04:20:16.100438+0000","last_became_active":"2024-03-29T04:17:25.361259+0000","last_became_peered":"2024-03-29T04:17:25.361259+0000","last_unstale":"2024-03-29T04:20:16.100438+0000","last_undegraded":"2024-03-29T04:20:16.100438+0000","last_fullsized":"2024-03-29T04:20:16.100438+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_clean_scrub_stamp":"2024-03-29T04:17:24.334639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:40.027873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":823801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35860,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335980,"statfs":{"total":96636764160,"available":96600043520,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35864,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335976,"statfs":{"total":96636764160,"available":96600039424,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31704,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":823801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:20:19.066 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:20:19.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.207+0000 7f5b8a7c8640 1 Processor -- start 2024-03-29T04:20:19.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.207+0000 7f5b8a7c8640 1 -- start start 2024-03-29T04:20:19.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.207+0000 7f5b8a7c8640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5b840d1810 0x7f5b840b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:19.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.207+0000 7f5b8a7c8640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f5b840b9170 0x7f5b840d2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:19.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.207+0000 7f5b8a7c8640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b840d3e60 con 0x7f5b8415e600 2024-03-29T04:20:19.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.207+0000 7f5b8a7c8640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f5b840b7450 con 0x7f5b840d1810 2024-03-29T04:20:19.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.207+0000 7f5b8a7c8640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f5b840d4140 con 0x7f5b840b9170 2024-03-29T04:20:19.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.207+0000 7f5b837fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5b840d1810 0x7f5b840b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:19.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.207+0000 7f5b83fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f5b8415e600 0x7f5b840d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:56174/0 (socket says 172.21.15.22:56174) 2024-03-29T04:20:19.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.207+0000 7f5b83fff640 1 -- 172.21.15.22:0/96360583 learned_addr learned my addr 172.21.15.22:0/96360583 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:19.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.207+0000 7f5b88d65640 1 --2- 172.21.15.22:0/96360583 >> v2:172.21.15.151:3300/0 conn(0x7f5b840b9170 0x7f5b840d2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:19.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.207+0000 7f5b837fe640 1 -- 172.21.15.22:0/96360583 >> v2:172.21.15.151:3300/0 conn(0x7f5b840b9170 msgr2=0x7f5b840d2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:19.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.207+0000 7f5b837fe640 1 --2- 172.21.15.22:0/96360583 >> v2:172.21.15.151:3300/0 conn(0x7f5b840b9170 0x7f5b840d2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:19.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.207+0000 7f5b837fe640 1 -- 172.21.15.22:0/96360583 >> v1:172.21.15.22:6789/0 conn(0x7f5b8415e600 legacy=0x7f5b840d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:19.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.207+0000 7f5b837fe640 1 -- 172.21.15.22:0/96360583 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b840b7680 con 0x7f5b840d1810 2024-03-29T04:20:19.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.207+0000 7f5b88d65640 1 --2- 172.21.15.22:0/96360583 >> v2:172.21.15.151:3300/0 conn(0x7f5b840b9170 0x7f5b840d2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:19.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.211+0000 7f5b837fe640 1 --2- 172.21.15.22:0/96360583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b840d1810 0x7f5b840b9f00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f5b840d42c0 tx=0x7f5b7801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88aaddaef10b080 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:19.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.211+0000 7f5b82ffd640 1 -- 172.21.15.22:0/96360583 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b780b1070 con 0x7f5b840d1810 2024-03-29T04:20:19.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.211+0000 7f5b82ffd640 1 -- 172.21.15.22:0/96360583 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b780ac600 con 0x7f5b840d1810 2024-03-29T04:20:19.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.211+0000 7f5b82ffd640 1 -- 172.21.15.22:0/96360583 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b780ac9d0 con 0x7f5b840d1810 2024-03-29T04:20:19.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.211+0000 7f5b8a7c8640 1 -- 172.21.15.22:0/96360583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b840d1810 msgr2=0x7f5b840b9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:19.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.211+0000 7f5b8a7c8640 1 --2- 172.21.15.22:0/96360583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b840d1810 0x7f5b840b9f00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f5b840d42c0 tx=0x7f5b7801c3a0 comp rx=0 tx=0).stop 2024-03-29T04:20:19.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.211+0000 7f5b8a7c8640 1 -- 172.21.15.22:0/96360583 shutdown_connections 2024-03-29T04:20:19.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.211+0000 7f5b8a7c8640 1 --2- 172.21.15.22:0/96360583 >> v2:172.21.15.151:3300/0 conn(0x7f5b840b9170 0x7f5b840d2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:19.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.211+0000 7f5b8a7c8640 1 --2- 172.21.15.22:0/96360583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b840d1810 0x7f5b840b9f00 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:19.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.211+0000 7f5b8a7c8640 1 -- 172.21.15.22:0/96360583 >> 172.21.15.22:0/96360583 conn(0x7f5b840b6230 msgr2=0x7f5b840b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:19.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.211+0000 7f5b8a7c8640 1 -- 172.21.15.22:0/96360583 shutdown_connections 2024-03-29T04:20:19.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.211+0000 7f5b8a7c8640 1 -- 172.21.15.22:0/96360583 wait complete. 2024-03-29T04:20:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.211+0000 7f5b8a7c8640 1 Processor -- start 2024-03-29T04:20:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.211+0000 7f5b8a7c8640 1 -- start start 2024-03-29T04:20:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.211+0000 7f5b8a7c8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b840b9170 0x7f5b8414d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.211+0000 7f5b8a7c8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b840d1810 0x7f5b8414c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:19.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.211+0000 7f5b8a7c8640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5b8415e600 0x7f5b84154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:19.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.211+0000 7f5b8a7c8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5b840d3720 con 0x7f5b840b9170 2024-03-29T04:20:19.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.211+0000 7f5b8a7c8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5b840d35a0 con 0x7f5b840d1810 2024-03-29T04:20:19.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.211+0000 7f5b8a7c8640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5b840d38a0 con 0x7f5b8415e600 2024-03-29T04:20:19.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b83fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b840b9170 0x7f5b8414d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:19.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b83fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b840b9170 0x7f5b8414d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52096/0 (socket says 172.21.15.22:52096) 2024-03-29T04:20:19.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b83fff640 1 -- 172.21.15.22:0/1832674579 learned_addr learned my addr 172.21.15.22:0/1832674579 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:19.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b837fe640 1 --2- 172.21.15.22:0/1832674579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b840d1810 0x7f5b8414c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:19.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b88d65640 1 --2- 172.21.15.22:0/1832674579 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5b8415e600 0x7f5b84154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:19.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b83fff640 1 -- 172.21.15.22:0/1832674579 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5b8415e600 msgr2=0x7f5b84154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:19.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b83fff640 1 --2- 172.21.15.22:0/1832674579 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5b8415e600 0x7f5b84154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:19.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b83fff640 1 -- 172.21.15.22:0/1832674579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b840d1810 msgr2=0x7f5b8414c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b83fff640 1 --2- 172.21.15.22:0/1832674579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b840d1810 0x7f5b8414c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b83fff640 1 -- 172.21.15.22:0/1832674579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b840c8570 con 0x7f5b840b9170 2024-03-29T04:20:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b88d65640 1 --2- 172.21.15.22:0/1832674579 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5b8415e600 0x7f5b84154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:19.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b83fff640 1 --2- 172.21.15.22:0/1832674579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b840b9170 0x7f5b8414d8e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400c370 tx=0x7f5b7400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:19.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b80ff9640 1 -- 172.21.15.22:0/1832674579 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b74017070 con 0x7f5b840b9170 2024-03-29T04:20:19.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b80ff9640 1 -- 172.21.15.22:0/1832674579 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b74005cf0 con 0x7f5b840b9170 2024-03-29T04:20:19.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b80ff9640 1 -- 172.21.15.22:0/1832674579 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b740072f0 con 0x7f5b840b9170 2024-03-29T04:20:19.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b8a7c8640 1 -- 172.21.15.22:0/1832674579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b840c8850 con 0x7f5b840b9170 2024-03-29T04:20:19.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b8a7c8640 1 -- 172.21.15.22:0/1832674579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b840c8e10 con 0x7f5b840b9170 2024-03-29T04:20:19.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b8a7c8640 1 -- 172.21.15.22:0/1832674579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b84068890 con 0x7f5b840b9170 2024-03-29T04:20:19.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b80ff9640 1 -- 172.21.15.22:0/1832674579 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f5b7401a070 con 0x7f5b840b9170 2024-03-29T04:20:19.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b80ff9640 1 --2- 172.21.15.22:0/1832674579 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5b70067fc0 0x7f5b7006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:19.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b80ff9640 1 -- 172.21.15.22:0/1832674579 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f5b74085a00 con 0x7f5b840b9170 2024-03-29T04:20:19.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.215+0000 7f5b837fe640 1 --2- 172.21.15.22:0/1832674579 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5b70067fc0 0x7f5b7006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:19.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.219+0000 7f5b837fe640 1 --2- 172.21.15.22:0/1832674579 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5b70067fc0 0x7f5b7006a480 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5b780ae000 tx=0x7f5b78008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:19.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.219+0000 7f5b80ff9640 1 -- 172.21.15.22:0/1832674579 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f5b74021020 con 0x7f5b840b9170 2024-03-29T04:20:19.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.383+0000 7f5b8a7c8640 1 -- 172.21.15.22:0/1832674579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b8406b9e0 con 0x7f5b840b9170 2024-03-29T04:20:19.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.383+0000 7f5b80ff9640 1 -- 172.21.15.22:0/1832674579 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+13293 (secure 0 0 0) 0x7f5b74056490 con 0x7f5b840b9170 2024-03-29T04:20:19.364 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:20:19.365 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":45,"fsid":"9f2f9779-fe00-4763-9aa1-fd3768e2f135","created":"2024-03-29T04:17:10.867231+0000","modified":"2024-03-29T04:20:16.006868+0000","last_up_change":"2024-03-29T04:17:22.328914+0000","last_in_change":"2024-03-29T04:17:13.882337+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:17:23.735304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0022de92-e2b8-4c00-ae06-21ca89b626e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6810","nonce":2469357214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6815","nonce":2469357214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6827","nonce":2469357214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6817","nonce":2469357214},{"type":"v1","addr":"172.21.15.22:6820","nonce":2469357214}]},"public_addr":"172.21.15.22:6810/2469357214","cluster_addr":"172.21.15.22:6815/2469357214","heartbeat_back_addr":"172.21.15.22:6827/2469357214","heartbeat_front_addr":"172.21.15.22:6820/2469357214","state":["exists","up"]},{"osd":1,"uuid":"75b2b30d-31a8-447c-adba-05194124a6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6801","nonce":3551960158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6803","nonce":3551960158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6807","nonce":3551960158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3551960158},{"type":"v1","addr":"172.21.15.22:6805","nonce":3551960158}]},"public_addr":"172.21.15.22:6801/3551960158","cluster_addr":"172.21.15.22:6803/3551960158","heartbeat_back_addr":"172.21.15.22:6807/3551960158","heartbeat_front_addr":"172.21.15.22:6805/3551960158","state":["exists","up"]},{"osd":2,"uuid":"0de67084-451e-438c-ad00-e722cee6d793","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6809","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6811","nonce":2216395437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6819","nonce":2216395437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6829","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6831","nonce":2216395437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2216395437},{"type":"v1","addr":"172.21.15.22:6826","nonce":2216395437}]},"public_addr":"172.21.15.22:6811/2216395437","cluster_addr":"172.21.15.22:6819/2216395437","heartbeat_back_addr":"172.21.15.22:6831/2216395437","heartbeat_front_addr":"172.21.15.22:6826/2216395437","state":["exists","up"]},{"osd":3,"uuid":"5adf4e26-c554-43ae-9f2a-aa1bcbdbcb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6813","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6816","nonce":583588642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6821","nonce":583588642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6830","nonce":583588642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":583588642},{"type":"v1","addr":"172.21.15.22:6825","nonce":583588642}]},"public_addr":"172.21.15.22:6816/583588642","cluster_addr":"172.21.15.22:6821/583588642","heartbeat_back_addr":"172.21.15.22:6830/583588642","heartbeat_front_addr":"172.21.15.22:6825/583588642","state":["exists","up"]},{"osd":4,"uuid":"085f3376-4fe9-4509-9c38-ac04f9bbcb1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6817","nonce":2643908371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6819","nonce":2643908371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6823","nonce":2643908371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2643908371},{"type":"v1","addr":"172.21.15.151:6821","nonce":2643908371}]},"public_addr":"172.21.15.151:6817/2643908371","cluster_addr":"172.21.15.151:6819/2643908371","heartbeat_back_addr":"172.21.15.151:6823/2643908371","heartbeat_front_addr":"172.21.15.151:6821/2643908371","state":["exists","up"]},{"osd":5,"uuid":"1da43d7c-2d62-49a6-b02e-e67bbaa9dc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6802","nonce":3763105575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6806","nonce":3763105575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6813","nonce":3763105575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3763105575},{"type":"v1","addr":"172.21.15.151:6809","nonce":3763105575}]},"public_addr":"172.21.15.151:6802/3763105575","cluster_addr":"172.21.15.151:6806/3763105575","heartbeat_back_addr":"172.21.15.151:6813/3763105575","heartbeat_front_addr":"172.21.15.151:6809/3763105575","state":["exists","up"]},{"osd":6,"uuid":"3a524e69-571d-4115-b9e6-e130c1f34085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6825","nonce":2519745336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6827","nonce":2519745336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6831","nonce":2519745336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2519745336},{"type":"v1","addr":"172.21.15.151:6829","nonce":2519745336}]},"public_addr":"172.21.15.151:6825/2519745336","cluster_addr":"172.21.15.151:6827/2519745336","heartbeat_back_addr":"172.21.15.151:6831/2519745336","heartbeat_front_addr":"172.21.15.151:6829/2519745336","state":["exists","up"]},{"osd":7,"uuid":"bdf4fc46-ba73-47af-b980-11f11f191d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6803","nonce":2961229122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6807","nonce":2961229122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6815","nonce":2961229122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2961229122},{"type":"v1","addr":"172.21.15.151:6812","nonce":2961229122}]},"public_addr":"172.21.15.151:6803/2961229122","cluster_addr":"172.21.15.151:6807/2961229122","heartbeat_back_addr":"172.21.15.151:6815/2961229122","heartbeat_front_addr":"172.21.15.151:6812/2961229122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.386186+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.484027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.483604+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.339150+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.205501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.427015+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.378242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:17:19.545451+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6835/2966923505":"2024-03-30T04:20:12.837636+0000","172.21.15.22:6834/2966923505":"2024-03-30T04:20:12.837636+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:20:19.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.387+0000 7f5b8a7c8640 1 -- 172.21.15.22:0/1832674579 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5b70067fc0 msgr2=0x7f5b7006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:19.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.387+0000 7f5b8a7c8640 1 --2- 172.21.15.22:0/1832674579 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5b70067fc0 0x7f5b7006a480 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5b780ae000 tx=0x7f5b78008040 comp rx=0 tx=0).stop 2024-03-29T04:20:19.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.387+0000 7f5b8a7c8640 1 -- 172.21.15.22:0/1832674579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b840b9170 msgr2=0x7f5b8414d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:19.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.387+0000 7f5b8a7c8640 1 --2- 172.21.15.22:0/1832674579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b840b9170 0x7f5b8414d8e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400c370 tx=0x7f5b7400c830 comp rx=0 tx=0).stop 2024-03-29T04:20:19.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.387+0000 7f5b8a7c8640 1 -- 172.21.15.22:0/1832674579 shutdown_connections 2024-03-29T04:20:19.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.387+0000 7f5b8a7c8640 1 --2- 172.21.15.22:0/1832674579 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f5b8415e600 0x7f5b84154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:19.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.387+0000 7f5b8a7c8640 1 --2- 172.21.15.22:0/1832674579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b840d1810 0x7f5b8414c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:19.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.387+0000 7f5b8a7c8640 1 --2- 172.21.15.22:0/1832674579 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5b70067fc0 0x7f5b7006a480 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:19.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.387+0000 7f5b8a7c8640 1 --2- 172.21.15.22:0/1832674579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b840b9170 0x7f5b8414d8e0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:19.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.387+0000 7f5b8a7c8640 1 -- 172.21.15.22:0/1832674579 >> 172.21.15.22:0/1832674579 conn(0x7f5b840b6230 msgr2=0x7f5b840ce970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:19.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.387+0000 7f5b8a7c8640 1 -- 172.21.15.22:0/1832674579 shutdown_connections 2024-03-29T04:20:19.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:19.387+0000 7f5b8a7c8640 1 -- 172.21.15.22:0/1832674579 wait complete. 2024-03-29T04:20:20.383 INFO:tasks.ceph:Scrubbing osd.0 2024-03-29T04:20:20.384 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:20:20.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.543+0000 7f8332aae640 1 Processor -- start 2024-03-29T04:20:20.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.543+0000 7f8332aae640 1 -- start start 2024-03-29T04:20:20.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.543+0000 7f8332aae640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f832c159fb0 0x7f832c15c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:20.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.543+0000 7f8332aae640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f832c0b1b60 0x7f832c0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:20.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.543+0000 7f8332aae640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f832c0d0020 con 0x7f832c0b1ff0 2024-03-29T04:20:20.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.543+0000 7f8332aae640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f832c15ce70 con 0x7f832c0b1b60 2024-03-29T04:20:20.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.543+0000 7f8332aae640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f832c15d150 con 0x7f832c159fb0 2024-03-29T04:20:20.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.543+0000 7f832bfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f832c0b1b60 0x7f832c0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:20.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.543+0000 7f833104b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f832c0b1ff0 0x7f832c0ceb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:56190/0 (socket says 172.21.15.22:56190) 2024-03-29T04:20:20.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.543+0000 7f833104b640 1 -- 172.21.15.22:0/4176395316 learned_addr learned my addr 172.21.15.22:0/4176395316 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:20.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.547+0000 7f833084a640 1 --2- 172.21.15.22:0/4176395316 >> v2:172.21.15.151:3300/0 conn(0x7f832c159fb0 0x7f832c15c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:20.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.547+0000 7f832bfff640 1 -- 172.21.15.22:0/4176395316 >> v2:172.21.15.151:3300/0 conn(0x7f832c159fb0 msgr2=0x7f832c15c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:20.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.547+0000 7f832bfff640 1 --2- 172.21.15.22:0/4176395316 >> v2:172.21.15.151:3300/0 conn(0x7f832c159fb0 0x7f832c15c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:20.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.547+0000 7f832bfff640 1 -- 172.21.15.22:0/4176395316 >> v1:172.21.15.22:6789/0 conn(0x7f832c0b1ff0 legacy=0x7f832c0ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:20.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.547+0000 7f832bfff640 1 -- 172.21.15.22:0/4176395316 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832c15d430 con 0x7f832c0b1b60 2024-03-29T04:20:20.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.547+0000 7f832b7fe640 1 -- 172.21.15.22:0/4176395316 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 400183341 0 0) 0x7f832c0d0020 con 0x7f832c0b1ff0 2024-03-29T04:20:20.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.547+0000 7f832bfff640 1 --2- 172.21.15.22:0/4176395316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f832c0b1b60 0x7f832c0cc890 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f83140097c0 tx=0x7f831401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7f58f2f154d19f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:20.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.547+0000 7f832b7fe640 1 -- 172.21.15.22:0/4176395316 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140ab680 con 0x7f832c0b1b60 2024-03-29T04:20:20.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.547+0000 7f832b7fe640 1 -- 172.21.15.22:0/4176395316 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83140b0070 con 0x7f832c0b1b60 2024-03-29T04:20:20.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.547+0000 7f832b7fe640 1 -- 172.21.15.22:0/4176395316 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140aba40 con 0x7f832c0b1b60 2024-03-29T04:20:20.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.547+0000 7f8332aae640 1 -- 172.21.15.22:0/4176395316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f832c0b1b60 msgr2=0x7f832c0cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:20.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.547+0000 7f8332aae640 1 --2- 172.21.15.22:0/4176395316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f832c0b1b60 0x7f832c0cc890 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f83140097c0 tx=0x7f831401c460 comp rx=0 tx=0).stop 2024-03-29T04:20:20.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.547+0000 7f8332aae640 1 -- 172.21.15.22:0/4176395316 shutdown_connections 2024-03-29T04:20:20.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.547+0000 7f8332aae640 1 --2- 172.21.15.22:0/4176395316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f832c0b1b60 0x7f832c0cc890 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:20.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.547+0000 7f8332aae640 1 --2- 172.21.15.22:0/4176395316 >> v2:172.21.15.151:3300/0 conn(0x7f832c159fb0 0x7f832c15c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:20.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.547+0000 7f8332aae640 1 -- 172.21.15.22:0/4176395316 >> 172.21.15.22:0/4176395316 conn(0x7f832c0b3cf0 msgr2=0x7f832c0b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:20.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.547+0000 7f8332aae640 1 -- 172.21.15.22:0/4176395316 shutdown_connections 2024-03-29T04:20:20.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.547+0000 7f8332aae640 1 -- 172.21.15.22:0/4176395316 wait complete. 2024-03-29T04:20:20.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f8332aae640 1 Processor -- start 2024-03-29T04:20:20.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f8332aae640 1 -- start start 2024-03-29T04:20:20.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f8332aae640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f832c0b1b60 0x7f832c14f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:20.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f8332aae640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f832c0b1ff0 0x7f832c15a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:20.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f8332aae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f832c159fb0 0x7f832c06e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:20.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f8332aae640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f832c0d0d10 con 0x7f832c159fb0 2024-03-29T04:20:20.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f8332aae640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f832c0cf450 con 0x7f832c0b1b60 2024-03-29T04:20:20.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f8332aae640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f832c0d0e90 con 0x7f832c0b1ff0 2024-03-29T04:20:20.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f833104b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f832c159fb0 0x7f832c06e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:20.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f833104b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f832c159fb0 0x7f832c06e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52114/0 (socket says 172.21.15.22:52114) 2024-03-29T04:20:20.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f833104b640 1 -- 172.21.15.22:0/2347611185 learned_addr learned my addr 172.21.15.22:0/2347611185 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:20.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f832bfff640 1 --2- 172.21.15.22:0/2347611185 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f832c0b1ff0 0x7f832c15a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:20.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f833104b640 1 -- 172.21.15.22:0/2347611185 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f832c0b1ff0 msgr2=0x7f832c15a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:20.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f833104b640 1 --2- 172.21.15.22:0/2347611185 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f832c0b1ff0 0x7f832c15a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:20.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f833104b640 1 -- 172.21.15.22:0/2347611185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f832c0b1b60 msgr2=0x7f832c14f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:20.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f833084a640 1 --2- 172.21.15.22:0/2347611185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f832c0b1b60 0x7f832c14f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:20.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f833104b640 1 --2- 172.21.15.22:0/2347611185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f832c0b1b60 0x7f832c14f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:20.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f833104b640 1 -- 172.21.15.22:0/2347611185 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832c06e770 con 0x7f832c159fb0 2024-03-29T04:20:20.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f832bfff640 1 --2- 172.21.15.22:0/2347611185 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f832c0b1ff0 0x7f832c15a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:20.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f833084a640 1 --2- 172.21.15.22:0/2347611185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f832c0b1b60 0x7f832c14f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:20.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f833104b640 1 --2- 172.21.15.22:0/2347611185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f832c159fb0 0x7f832c06e230 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f832400cb20 tx=0x7f832400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:20.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f83297fa640 1 -- 172.21.15.22:0/2347611185 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832400aca0 con 0x7f832c159fb0 2024-03-29T04:20:20.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f83297fa640 1 -- 172.21.15.22:0/2347611185 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8324012400 con 0x7f832c159fb0 2024-03-29T04:20:20.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f83297fa640 1 -- 172.21.15.22:0/2347611185 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832400ae60 con 0x7f832c159fb0 2024-03-29T04:20:20.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f8332aae640 1 -- 172.21.15.22:0/2347611185 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832c15aad0 con 0x7f832c159fb0 2024-03-29T04:20:20.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f8332aae640 1 -- 172.21.15.22:0/2347611185 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f832c15b090 con 0x7f832c159fb0 2024-03-29T04:20:20.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f8332aae640 1 -- 172.21.15.22:0/2347611185 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f832c0687f0 con 0x7f832c159fb0 2024-03-29T04:20:20.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.551+0000 7f83297fa640 1 -- 172.21.15.22:0/2347611185 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f83240127f0 con 0x7f832c159fb0 2024-03-29T04:20:20.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.555+0000 7f83297fa640 1 --2- 172.21.15.22:0/2347611185 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f830c067ff0 0x7f830c06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:20.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.555+0000 7f83297fa640 1 -- 172.21.15.22:0/2347611185 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f8324087470 con 0x7f832c159fb0 2024-03-29T04:20:20.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.555+0000 7f83297fa640 1 --2- 172.21.15.22:0/2347611185 >> [v2:172.21.15.22:6808/2469357214,v1:172.21.15.22:6810/2469357214] conn(0x7f830c06e490 0x7f830c0708d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:20.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.555+0000 7f83297fa640 1 -- 172.21.15.22:0/2347611185 --> [v2:172.21.15.22:6808/2469357214,v1:172.21.15.22:6810/2469357214] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f830c070fa0 con 0x7f830c06e490 2024-03-29T04:20:20.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.555+0000 7f83297fa640 1 -- 172.21.15.22:0/2347611185 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f8324087900 con 0x7f832c159fb0 2024-03-29T04:20:20.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.555+0000 7f832bfff640 1 --2- 172.21.15.22:0/2347611185 >> [v2:172.21.15.22:6808/2469357214,v1:172.21.15.22:6810/2469357214] conn(0x7f830c06e490 0x7f830c0708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:20.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.555+0000 7f833084a640 1 --2- 172.21.15.22:0/2347611185 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f830c067ff0 0x7f830c06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:20.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.555+0000 7f832bfff640 1 --2- 172.21.15.22:0/2347611185 >> [v2:172.21.15.22:6808/2469357214,v1:172.21.15.22:6810/2469357214] conn(0x7f830c06e490 0x7f830c0708d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:20.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.555+0000 7f833084a640 1 --2- 172.21.15.22:0/2347611185 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f830c067ff0 0x7f830c06a4b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f831c0097a0 tx=0x7f831c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:20.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.555+0000 7f83297fa640 1 -- 172.21.15.22:0/2347611185 <== osd.0 v2:172.21.15.22:6808/2469357214 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f830c070fa0 con 0x7f830c06e490 2024-03-29T04:20:20.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.567+0000 7f8332aae640 1 -- 172.21.15.22:0/2347611185 --> [v2:172.21.15.22:6808/2469357214,v1:172.21.15.22:6810/2469357214] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f832c15bdc0 con 0x7f830c06e490 2024-03-29T04:20:20.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.567+0000 7f83297fa640 1 -- 172.21.15.22:0/2347611185 <== osd.0 v2:172.21.15.22:6808/2469357214 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f832c15bdc0 con 0x7f830c06e490 2024-03-29T04:20:20.550 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-29T04:20:20.550 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:20:20.550 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-29T04:20:20.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.567+0000 7f8332aae640 1 -- 172.21.15.22:0/2347611185 >> [v2:172.21.15.22:6808/2469357214,v1:172.21.15.22:6810/2469357214] conn(0x7f830c06e490 msgr2=0x7f830c0708d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:20.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.567+0000 7f8332aae640 1 --2- 172.21.15.22:0/2347611185 >> [v2:172.21.15.22:6808/2469357214,v1:172.21.15.22:6810/2469357214] conn(0x7f830c06e490 0x7f830c0708d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:20.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.567+0000 7f8332aae640 1 -- 172.21.15.22:0/2347611185 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f830c067ff0 msgr2=0x7f830c06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:20.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.567+0000 7f8332aae640 1 --2- 172.21.15.22:0/2347611185 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f830c067ff0 0x7f830c06a4b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f831c0097a0 tx=0x7f831c008040 comp rx=0 tx=0).stop 2024-03-29T04:20:20.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.567+0000 7f8332aae640 1 -- 172.21.15.22:0/2347611185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f832c159fb0 msgr2=0x7f832c06e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:20.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.567+0000 7f8332aae640 1 --2- 172.21.15.22:0/2347611185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f832c159fb0 0x7f832c06e230 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f832400cb20 tx=0x7f832400a060 comp rx=0 tx=0).stop 2024-03-29T04:20:20.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.567+0000 7f833104b640 1 -- 172.21.15.22:0/2347611185 reap_dead start 2024-03-29T04:20:20.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.567+0000 7f8332aae640 1 -- 172.21.15.22:0/2347611185 shutdown_connections 2024-03-29T04:20:20.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.567+0000 7f8332aae640 1 -- 172.21.15.22:0/2347611185 >> 172.21.15.22:0/2347611185 conn(0x7f832c0b3cf0 msgr2=0x7f832c0b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:20.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.567+0000 7f8332aae640 1 -- 172.21.15.22:0/2347611185 shutdown_connections 2024-03-29T04:20:20.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.567+0000 7f8332aae640 1 -- 172.21.15.22:0/2347611185 wait complete. 2024-03-29T04:20:20.566 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-29T04:20:20.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.699+0000 7f4e01856640 1 Processor -- start 2024-03-29T04:20:20.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.699+0000 7f4e01856640 1 -- start start 2024-03-29T04:20:20.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.699+0000 7f4e01856640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4dfc144270 0x7f4dfc144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:20.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.699+0000 7f4e01856640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f4dfc0b2ef0 0x7f4dfc15f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:20.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.699+0000 7f4e01856640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dfc0b33f0 con 0x7f4dfc0b4490 2024-03-29T04:20:20.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.699+0000 7f4e01856640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f4dfc144fa0 con 0x7f4dfc0b2ef0 2024-03-29T04:20:20.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.699+0000 7f4e01856640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4dfc15fce0 con 0x7f4dfc144270 2024-03-29T04:20:20.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.699+0000 7f4dfb7fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f4dfc0b4490 0x7f4dfc06c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:56202/0 (socket says 172.21.15.22:56202) 2024-03-29T04:20:20.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.699+0000 7f4dfb7fe640 1 -- 172.21.15.22:0/2402152086 learned_addr learned my addr 172.21.15.22:0/2402152086 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:20.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.699+0000 7f4dfa7fc640 1 --2- 172.21.15.22:0/2402152086 >> v2:172.21.15.151:3300/0 conn(0x7f4dfc0b2ef0 0x7f4dfc15f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:20.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.699+0000 7f4dfaffd640 1 --2- 172.21.15.22:0/2402152086 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4dfc144270 0x7f4dfc144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:20.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.699+0000 7f4df9ffb640 1 -- 172.21.15.22:0/2402152086 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3325094377 0 0) 0x7f4dfc0b33f0 con 0x7f4dfc0b4490 2024-03-29T04:20:20.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.699+0000 7f4df9ffb640 1 -- 172.21.15.22:0/2402152086 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4de4003530 con 0x7f4dfc0b4490 2024-03-29T04:20:20.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.699+0000 7f4df9ffb640 1 -- 172.21.15.22:0/2402152086 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4222137503 0 0) 0x7f4de4003530 con 0x7f4dfc0b4490 2024-03-29T04:20:20.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.699+0000 7f4df9ffb640 1 -- 172.21.15.22:0/2402152086 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4dfc144270 msgr2=0x7f4dfc144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:20.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.699+0000 7f4df9ffb640 1 --2- 172.21.15.22:0/2402152086 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4dfc144270 0x7f4dfc144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:20.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.699+0000 7f4df9ffb640 1 -- 172.21.15.22:0/2402152086 >> v2:172.21.15.151:3300/0 conn(0x7f4dfc0b2ef0 msgr2=0x7f4dfc15f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:20.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.699+0000 7f4df9ffb640 1 --2- 172.21.15.22:0/2402152086 >> v2:172.21.15.151:3300/0 conn(0x7f4dfc0b2ef0 0x7f4dfc15f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:20.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.699+0000 7f4df9ffb640 1 -- 172.21.15.22:0/2402152086 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dfc0b36d0 con 0x7f4dfc0b4490 2024-03-29T04:20:20.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.699+0000 7f4df9ffb640 1 -- 172.21.15.22:0/2402152086 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f4df4001a00 con 0x7f4dfc0b4490 2024-03-29T04:20:20.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.699+0000 7f4dfa7fc640 1 --2- 172.21.15.22:0/2402152086 >> v2:172.21.15.151:3300/0 conn(0x7f4dfc0b2ef0 0x7f4dfc15f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:20:20.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4df9ffb640 1 -- 172.21.15.22:0/2402152086 >> v1:172.21.15.22:6789/0 conn(0x7f4dfc0b4490 legacy=0x7f4dfc06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:20.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4df9ffb640 1 --2- 172.21.15.22:0/2402152086 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4dfc144270 0x7f4de40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:20.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4df9ffb640 1 --2- 172.21.15.22:0/2402152086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4de4003f40 0x7f4de4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:20.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4df9ffb640 1 --2- 172.21.15.22:0/2402152086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4de4004d50 0x7f4de40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:20.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4df9ffb640 1 -- 172.21.15.22:0/2402152086 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc15fce0 con 0x7f4de4004d50 2024-03-29T04:20:20.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4df9ffb640 1 -- 172.21.15.22:0/2402152086 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4de400b9d0 con 0x7f4de4003f40 2024-03-29T04:20:20.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4df9ffb640 1 -- 172.21.15.22:0/2402152086 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4de400bb50 con 0x7f4dfc144270 2024-03-29T04:20:20.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4dfb7fe640 1 --2- 172.21.15.22:0/2402152086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4de4004d50 0x7f4de40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:20.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4dfaffd640 1 --2- 172.21.15.22:0/2402152086 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4dfc144270 0x7f4de40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:20.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4dfa7fc640 1 --2- 172.21.15.22:0/2402152086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4de4003f40 0x7f4de4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:20.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4dfb7fe640 1 -- 172.21.15.22:0/2402152086 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4dfc144270 msgr2=0x7f4de40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:20.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4dfb7fe640 1 --2- 172.21.15.22:0/2402152086 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4dfc144270 0x7f4de40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:20.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4dfb7fe640 1 -- 172.21.15.22:0/2402152086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4de4003f40 msgr2=0x7f4de4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:20.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4dfb7fe640 1 --2- 172.21.15.22:0/2402152086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4de4003f40 0x7f4de4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:20.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4dfb7fe640 1 -- 172.21.15.22:0/2402152086 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de4003530 con 0x7f4de4004d50 2024-03-29T04:20:20.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4dfb7fe640 1 --2- 172.21.15.22:0/2402152086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4de4004d50 0x7f4de40051d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4df400a920 tx=0x7f4df401d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b2468e75a6173b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:20.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4df9ffb640 1 -- 172.21.15.22:0/2402152086 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df400a4c0 con 0x7f4de4004d50 2024-03-29T04:20:20.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4df9ffb640 1 -- 172.21.15.22:0/2402152086 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4df40072f0 con 0x7f4de4004d50 2024-03-29T04:20:20.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4df9ffb640 1 -- 172.21.15.22:0/2402152086 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df4007720 con 0x7f4de4004d50 2024-03-29T04:20:20.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4e01856640 1 -- 172.21.15.22:0/2402152086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4de4004d50 msgr2=0x7f4de40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:20.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4e01856640 1 --2- 172.21.15.22:0/2402152086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4de4004d50 0x7f4de40051d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4df400a920 tx=0x7f4df401d3e0 comp rx=0 tx=0).stop 2024-03-29T04:20:20.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4dfb7fe640 1 -- 172.21.15.22:0/2402152086 reap_dead start 2024-03-29T04:20:20.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4e01856640 1 -- 172.21.15.22:0/2402152086 shutdown_connections 2024-03-29T04:20:20.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4e01856640 1 -- 172.21.15.22:0/2402152086 >> 172.21.15.22:0/2402152086 conn(0x7f4dfc0cfee0 msgr2=0x7f4dfc0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:20.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4e01856640 1 -- 172.21.15.22:0/2402152086 shutdown_connections 2024-03-29T04:20:20.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.703+0000 7f4e01856640 1 -- 172.21.15.22:0/2402152086 wait complete. 2024-03-29T04:20:20.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4e01856640 1 Processor -- start 2024-03-29T04:20:20.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4e01856640 1 -- start start 2024-03-29T04:20:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4e01856640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dfc13b4d0 0x7f4dfc13e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4e01856640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4dfc0c5060 0x7f4dfc13f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4e01856640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4dfc13fe20 0x7f4dfc13b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4e01856640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc06cd80 con 0x7f4dfc13b4d0 2024-03-29T04:20:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4e01856640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc0b3f70 con 0x7f4dfc0c5060 2024-03-29T04:20:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4e01856640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f4dfc06cf00 con 0x7f4dfc13fe20 2024-03-29T04:20:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4dfaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dfc13b4d0 0x7f4dfc13e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4dfaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dfc13b4d0 0x7f4dfc13e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52122/0 (socket says 172.21.15.22:52122) 2024-03-29T04:20:20.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4dfaffd640 1 -- 172.21.15.22:0/2893622434 learned_addr learned my addr 172.21.15.22:0/2893622434 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:20.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4dfb7fe640 1 --2- 172.21.15.22:0/2893622434 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4dfc13fe20 0x7f4dfc13b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:20.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4dfa7fc640 1 --2- 172.21.15.22:0/2893622434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4dfc0c5060 0x7f4dfc13f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:20.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4dfaffd640 1 -- 172.21.15.22:0/2893622434 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4dfc13fe20 msgr2=0x7f4dfc13b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:20.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4dfaffd640 1 --2- 172.21.15.22:0/2893622434 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4dfc13fe20 0x7f4dfc13b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:20.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4dfaffd640 1 -- 172.21.15.22:0/2893622434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4dfc0c5060 msgr2=0x7f4dfc13f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:20.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4dfaffd640 1 --2- 172.21.15.22:0/2893622434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4dfc0c5060 0x7f4dfc13f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:20.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4dfaffd640 1 -- 172.21.15.22:0/2893622434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dfc13e840 con 0x7f4dfc13b4d0 2024-03-29T04:20:20.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4dfb7fe640 1 --2- 172.21.15.22:0/2893622434 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4dfc13fe20 0x7f4dfc13b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:20:20.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4dfaffd640 1 --2- 172.21.15.22:0/2893622434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dfc13b4d0 0x7f4dfc13e110 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4dec0115e0 tx=0x7f4dec011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:20.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4de3fff640 1 -- 172.21.15.22:0/2893622434 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec00f8f0 con 0x7f4dfc13b4d0 2024-03-29T04:20:20.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4de3fff640 1 -- 172.21.15.22:0/2893622434 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dec00fa90 con 0x7f4dfc13b4d0 2024-03-29T04:20:20.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4e01856640 1 -- 172.21.15.22:0/2893622434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dfc13eb20 con 0x7f4dfc13b4d0 2024-03-29T04:20:20.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4de3fff640 1 -- 172.21.15.22:0/2893622434 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec00fe40 con 0x7f4dfc13b4d0 2024-03-29T04:20:20.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4e01856640 1 -- 172.21.15.22:0/2893622434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dfc13f0e0 con 0x7f4dfc13b4d0 2024-03-29T04:20:20.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4e01856640 1 -- 172.21.15.22:0/2893622434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dfc15b150 con 0x7f4dfc13b4d0 2024-03-29T04:20:20.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.707+0000 7f4de3fff640 1 -- 172.21.15.22:0/2893622434 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f4dec015460 con 0x7f4dfc13b4d0 2024-03-29T04:20:20.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.711+0000 7f4de3fff640 1 --2- 172.21.15.22:0/2893622434 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4de8067fc0 0x7f4de806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:20.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.711+0000 7f4de3fff640 1 -- 172.21.15.22:0/2893622434 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f4dec086d10 con 0x7f4dfc13b4d0 2024-03-29T04:20:20.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.711+0000 7f4dfa7fc640 1 --2- 172.21.15.22:0/2893622434 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4de8067fc0 0x7f4de806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:20.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.711+0000 7f4dfa7fc640 1 --2- 172.21.15.22:0/2893622434 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4de8067fc0 0x7f4de806a480 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc15afe0 tx=0x7f4df0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:20.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.715+0000 7f4de3fff640 1 -- 172.21.15.22:0/2893622434 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f4dec019030 con 0x7f4dfc13b4d0 2024-03-29T04:20:20.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.879+0000 7f4e01856640 1 -- 172.21.15.22:0/2893622434 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f4dfc068890 con 0x7f4de8067fc0 2024-03-29T04:20:20.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.879+0000 7f4de3fff640 1 -- 172.21.15.22:0/2893622434 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4dfc068890 con 0x7f4de8067fc0 2024-03-29T04:20:20.859 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 0 to deep-scrub 2024-03-29T04:20:20.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.879+0000 7f4e01856640 1 -- 172.21.15.22:0/2893622434 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4de8067fc0 msgr2=0x7f4de806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:20.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.879+0000 7f4e01856640 1 --2- 172.21.15.22:0/2893622434 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4de8067fc0 0x7f4de806a480 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc15afe0 tx=0x7f4df0008040 comp rx=0 tx=0).stop 2024-03-29T04:20:20.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.879+0000 7f4e01856640 1 -- 172.21.15.22:0/2893622434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dfc13b4d0 msgr2=0x7f4dfc13e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:20.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.879+0000 7f4e01856640 1 --2- 172.21.15.22:0/2893622434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dfc13b4d0 0x7f4dfc13e110 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4dec0115e0 tx=0x7f4dec011aa0 comp rx=0 tx=0).stop 2024-03-29T04:20:20.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.879+0000 7f4e01856640 1 -- 172.21.15.22:0/2893622434 shutdown_connections 2024-03-29T04:20:20.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.879+0000 7f4e01856640 1 --2- 172.21.15.22:0/2893622434 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f4dfc13fe20 0x7f4dfc13b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:20.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.879+0000 7f4e01856640 1 --2- 172.21.15.22:0/2893622434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4dfc0c5060 0x7f4dfc13f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:20.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.879+0000 7f4e01856640 1 --2- 172.21.15.22:0/2893622434 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f4de8067fc0 0x7f4de806a480 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:20.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.879+0000 7f4e01856640 1 --2- 172.21.15.22:0/2893622434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dfc13b4d0 0x7f4dfc13e110 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:20.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.879+0000 7f4e01856640 1 -- 172.21.15.22:0/2893622434 >> 172.21.15.22:0/2893622434 conn(0x7f4dfc0cfee0 msgr2=0x7f4dfc0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:20.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.879+0000 7f4e01856640 1 -- 172.21.15.22:0/2893622434 shutdown_connections 2024-03-29T04:20:20.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:20.879+0000 7f4e01856640 1 -- 172.21.15.22:0/2893622434 wait complete. 2024-03-29T04:20:20.879 INFO:tasks.ceph:Scrubbing osd.1 2024-03-29T04:20:20.879 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:20:21.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.019+0000 7f6264e2e640 1 Processor -- start 2024-03-29T04:20:21.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.019+0000 7f6264e2e640 1 -- start start 2024-03-29T04:20:21.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.019+0000 7f6264e2e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f626015e690 0x7f62600b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.019+0000 7f6264e2e640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f62600b9fd0 0x7f62600d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.019+0000 7f6264e2e640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f62600d1640 0x7f6260161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.019+0000 7f6264e2e640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f62600d1ab0 con 0x7f626015e690 2024-03-29T04:20:21.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.019+0000 7f6264e2e640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f62600d2780 con 0x7f62600b9fd0 2024-03-29T04:20:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.019+0000 7f6264e2e640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f62600b4ba0 con 0x7f62600d1640 2024-03-29T04:20:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.019+0000 7f625e59c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f626015e690 0x7f62600b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.019+0000 7f625e59c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f626015e690 0x7f62600b3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52128/0 (socket says 172.21.15.22:52128) 2024-03-29T04:20:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.019+0000 7f625e59c640 1 -- 172.21.15.22:0/2079516678 learned_addr learned my addr 172.21.15.22:0/2079516678 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.023+0000 7f625dd9b640 1 --2- 172.21.15.22:0/2079516678 >> v2:172.21.15.151:3300/0 conn(0x7f62600b9fd0 0x7f62600d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.023+0000 7f625e59c640 1 -- 172.21.15.22:0/2079516678 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f62600d1640 msgr2=0x7f6260161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.023+0000 7f625e59c640 1 --2- 172.21.15.22:0/2079516678 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f62600d1640 0x7f6260161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.023+0000 7f625e59c640 1 -- 172.21.15.22:0/2079516678 >> v2:172.21.15.151:3300/0 conn(0x7f62600b9fd0 msgr2=0x7f62600d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.023+0000 7f625e59c640 1 --2- 172.21.15.22:0/2079516678 >> v2:172.21.15.151:3300/0 conn(0x7f62600b9fd0 0x7f62600d1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.023+0000 7f625e59c640 1 -- 172.21.15.22:0/2079516678 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62600b4300 con 0x7f626015e690 2024-03-29T04:20:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.023+0000 7f625dd9b640 1 --2- 172.21.15.22:0/2079516678 >> v2:172.21.15.151:3300/0 conn(0x7f62600b9fd0 0x7f62600d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:21.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.023+0000 7f625e59c640 1 --2- 172.21.15.22:0/2079516678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f626015e690 0x7f62600b3a60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f62600d4280 tx=0x7f625001c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5bf7357c160cb2e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:21.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.023+0000 7f625d59a640 1 -- 172.21.15.22:0/2079516678 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62500b0070 con 0x7f626015e690 2024-03-29T04:20:21.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.023+0000 7f625d59a640 1 -- 172.21.15.22:0/2079516678 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62500ab600 con 0x7f626015e690 2024-03-29T04:20:21.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.023+0000 7f625d59a640 1 -- 172.21.15.22:0/2079516678 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62500ab9d0 con 0x7f626015e690 2024-03-29T04:20:21.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.023+0000 7f6264e2e640 1 -- 172.21.15.22:0/2079516678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f626015e690 msgr2=0x7f62600b3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.023+0000 7f6264e2e640 1 --2- 172.21.15.22:0/2079516678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f626015e690 0x7f62600b3a60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f62600d4280 tx=0x7f625001c1a0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.023+0000 7f6264e2e640 1 -- 172.21.15.22:0/2079516678 shutdown_connections 2024-03-29T04:20:21.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.023+0000 7f6264e2e640 1 --2- 172.21.15.22:0/2079516678 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f62600d1640 0x7f6260161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.023+0000 7f6264e2e640 1 --2- 172.21.15.22:0/2079516678 >> v2:172.21.15.151:3300/0 conn(0x7f62600b9fd0 0x7f62600d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.023+0000 7f6264e2e640 1 --2- 172.21.15.22:0/2079516678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f626015e690 0x7f62600b3a60 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.023+0000 7f6264e2e640 1 -- 172.21.15.22:0/2079516678 >> 172.21.15.22:0/2079516678 conn(0x7f62600b6230 msgr2=0x7f62600b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:21.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.023+0000 7f6264e2e640 1 -- 172.21.15.22:0/2079516678 shutdown_connections 2024-03-29T04:20:21.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.023+0000 7f6264e2e640 1 -- 172.21.15.22:0/2079516678 wait complete. 2024-03-29T04:20:21.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f6264e2e640 1 Processor -- start 2024-03-29T04:20:21.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f6264e2e640 1 -- start start 2024-03-29T04:20:21.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f6264e2e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62600b9fd0 0x7f626015bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f6264e2e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62600d1640 0x7f6260167c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f6264e2e640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f62600cd030 0x7f62600c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f6264e2e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f62600d2d90 con 0x7f62600b9fd0 2024-03-29T04:20:21.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f6264e2e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f62600d2c10 con 0x7f62600d1640 2024-03-29T04:20:21.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f6264e2e640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f62600d2f10 con 0x7f62600cd030 2024-03-29T04:20:21.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f625e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62600b9fd0 0x7f626015bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f625e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62600b9fd0 0x7f626015bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52144/0 (socket says 172.21.15.22:52144) 2024-03-29T04:20:21.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f625e59c640 1 -- 172.21.15.22:0/2627057984 learned_addr learned my addr 172.21.15.22:0/2627057984 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:21.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f625dd9b640 1 --2- 172.21.15.22:0/2627057984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62600d1640 0x7f6260167c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f625e59c640 1 -- 172.21.15.22:0/2627057984 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f62600cd030 msgr2=0x7f62600c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f625e59c640 1 --2- 172.21.15.22:0/2627057984 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f62600cd030 0x7f62600c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f625e59c640 1 -- 172.21.15.22:0/2627057984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62600d1640 msgr2=0x7f6260167c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f625e59c640 1 --2- 172.21.15.22:0/2627057984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62600d1640 0x7f6260167c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f625e59c640 1 -- 172.21.15.22:0/2627057984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626015c200 con 0x7f62600b9fd0 2024-03-29T04:20:21.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f625e59c640 1 --2- 172.21.15.22:0/2627057984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62600b9fd0 0x7f626015bb90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f62500ae040 tx=0x7f6250018f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:21.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f624effd640 1 -- 172.21.15.22:0/2627057984 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62500b0040 con 0x7f62600b9fd0 2024-03-29T04:20:21.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f624effd640 1 -- 172.21.15.22:0/2627057984 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62500b9070 con 0x7f62600b9fd0 2024-03-29T04:20:21.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f6264e2e640 1 -- 172.21.15.22:0/2627057984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626015c4e0 con 0x7f62600b9fd0 2024-03-29T04:20:21.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f624effd640 1 -- 172.21.15.22:0/2627057984 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62500b48a0 con 0x7f62600b9fd0 2024-03-29T04:20:21.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f6264e2e640 1 -- 172.21.15.22:0/2627057984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f626015caa0 con 0x7f62600b9fd0 2024-03-29T04:20:21.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f6264e2e640 1 -- 172.21.15.22:0/2627057984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6260168190 con 0x7f62600b9fd0 2024-03-29T04:20:21.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.027+0000 7f624effd640 1 -- 172.21.15.22:0/2627057984 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f62500ab4c0 con 0x7f62600b9fd0 2024-03-29T04:20:21.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.031+0000 7f624effd640 1 --2- 172.21.15.22:0/2627057984 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6240067ff0 0x7f624006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.031+0000 7f624effd640 1 -- 172.21.15.22:0/2627057984 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f6250120b50 con 0x7f62600b9fd0 2024-03-29T04:20:21.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.031+0000 7f624effd640 1 --2- 172.21.15.22:0/2627057984 >> [v2:172.21.15.22:6800/3551960158,v1:172.21.15.22:6801/3551960158] conn(0x7f624006e490 0x7f62400708d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.031+0000 7f624effd640 1 -- 172.21.15.22:0/2627057984 --> [v2:172.21.15.22:6800/3551960158,v1:172.21.15.22:6801/3551960158] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6240070fa0 con 0x7f624006e490 2024-03-29T04:20:21.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.031+0000 7f624effd640 1 -- 172.21.15.22:0/2627057984 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f6250120fe0 con 0x7f62600b9fd0 2024-03-29T04:20:21.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.031+0000 7f625dd9b640 1 --2- 172.21.15.22:0/2627057984 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6240067ff0 0x7f624006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.031+0000 7f625ed9d640 1 --2- 172.21.15.22:0/2627057984 >> [v2:172.21.15.22:6800/3551960158,v1:172.21.15.22:6801/3551960158] conn(0x7f624006e490 0x7f62400708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.031+0000 7f625ed9d640 1 --2- 172.21.15.22:0/2627057984 >> [v2:172.21.15.22:6800/3551960158,v1:172.21.15.22:6801/3551960158] conn(0x7f624006e490 0x7f62400708d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:21.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.031+0000 7f625dd9b640 1 --2- 172.21.15.22:0/2627057984 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6240067ff0 0x7f624006a4b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6254006fd0 tx=0x7f6254008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:21.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.031+0000 7f624effd640 1 -- 172.21.15.22:0/2627057984 <== osd.1 v2:172.21.15.22:6800/3551960158 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6240070fa0 con 0x7f624006e490 2024-03-29T04:20:21.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.043+0000 7f6264e2e640 1 -- 172.21.15.22:0/2627057984 --> [v2:172.21.15.22:6800/3551960158,v1:172.21.15.22:6801/3551960158] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f62600687f0 con 0x7f624006e490 2024-03-29T04:20:21.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.043+0000 7f624effd640 1 -- 172.21.15.22:0/2627057984 <== osd.1 v2:172.21.15.22:6800/3551960158 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f62600687f0 con 0x7f624006e490 2024-03-29T04:20:21.025 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-29T04:20:21.026 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:20:21.026 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-29T04:20:21.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.043+0000 7f6264e2e640 1 -- 172.21.15.22:0/2627057984 >> [v2:172.21.15.22:6800/3551960158,v1:172.21.15.22:6801/3551960158] conn(0x7f624006e490 msgr2=0x7f62400708d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.043+0000 7f6264e2e640 1 --2- 172.21.15.22:0/2627057984 >> [v2:172.21.15.22:6800/3551960158,v1:172.21.15.22:6801/3551960158] conn(0x7f624006e490 0x7f62400708d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.043+0000 7f6264e2e640 1 -- 172.21.15.22:0/2627057984 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6240067ff0 msgr2=0x7f624006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.043+0000 7f6264e2e640 1 --2- 172.21.15.22:0/2627057984 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6240067ff0 0x7f624006a4b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6254006fd0 tx=0x7f6254008040 comp rx=0 tx=0).stop 2024-03-29T04:20:21.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.043+0000 7f6264e2e640 1 -- 172.21.15.22:0/2627057984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62600b9fd0 msgr2=0x7f626015bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.043+0000 7f6264e2e640 1 --2- 172.21.15.22:0/2627057984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62600b9fd0 0x7f626015bb90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f62500ae040 tx=0x7f6250018f40 comp rx=0 tx=0).stop 2024-03-29T04:20:21.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.043+0000 7f625ed9d640 1 -- 172.21.15.22:0/2627057984 reap_dead start 2024-03-29T04:20:21.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.043+0000 7f6264e2e640 1 -- 172.21.15.22:0/2627057984 shutdown_connections 2024-03-29T04:20:21.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.043+0000 7f6264e2e640 1 -- 172.21.15.22:0/2627057984 >> 172.21.15.22:0/2627057984 conn(0x7f62600b6230 msgr2=0x7f626006c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:21.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.043+0000 7f6264e2e640 1 -- 172.21.15.22:0/2627057984 shutdown_connections 2024-03-29T04:20:21.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.043+0000 7f6264e2e640 1 -- 172.21.15.22:0/2627057984 wait complete. 2024-03-29T04:20:21.041 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-29T04:20:21.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.175+0000 7ff2cd31b640 1 Processor -- start 2024-03-29T04:20:21.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.175+0000 7ff2cd31b640 1 -- start start 2024-03-29T04:20:21.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.175+0000 7ff2cd31b640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7ff2c815e620 0x7ff2c815ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.175+0000 7ff2cd31b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff2c80b9c70 0x7ff2c80d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.175+0000 7ff2cd31b640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c80d2b50 0x7ff2c80b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.175+0000 7ff2cd31b640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7ff2c815f940 con 0x7ff2c80b9c70 2024-03-29T04:20:21.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.175+0000 7ff2cd31b640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7ff2c80cea20 con 0x7ff2c815e620 2024-03-29T04:20:21.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.175+0000 7ff2cd31b640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c815f0a0 con 0x7ff2c80d2b50 2024-03-29T04:20:21.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.175+0000 7ff2c67fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff2c80b9c70 0x7ff2c80d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.175+0000 7ff2c67fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff2c80b9c70 0x7ff2c80d2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52150/0 (socket says 172.21.15.22:52150) 2024-03-29T04:20:21.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.175+0000 7ff2c67fc640 1 -- 172.21.15.22:0/663290252 learned_addr learned my addr 172.21.15.22:0/663290252 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:21.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.175+0000 7ff2c6ffd640 1 --2- 172.21.15.22:0/663290252 >> v2:172.21.15.151:3300/0 conn(0x7ff2c815e620 0x7ff2c815ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.175+0000 7ff2c67fc640 1 -- 172.21.15.22:0/663290252 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c80d2b50 msgr2=0x7ff2c80b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:21.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.175+0000 7ff2c77fe640 1 --2- 172.21.15.22:0/663290252 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c80d2b50 0x7ff2c80b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.175+0000 7ff2c67fc640 1 --2- 172.21.15.22:0/663290252 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c80d2b50 0x7ff2c80b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.175+0000 7ff2c67fc640 1 -- 172.21.15.22:0/663290252 >> v2:172.21.15.151:3300/0 conn(0x7ff2c815e620 msgr2=0x7ff2c815ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:21.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.175+0000 7ff2c67fc640 1 --2- 172.21.15.22:0/663290252 >> v2:172.21.15.151:3300/0 conn(0x7ff2c815e620 0x7ff2c815ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.175+0000 7ff2c67fc640 1 -- 172.21.15.22:0/663290252 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c815f380 con 0x7ff2c80b9c70 2024-03-29T04:20:21.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.175+0000 7ff2c6ffd640 1 --2- 172.21.15.22:0/663290252 >> v2:172.21.15.151:3300/0 conn(0x7ff2c815e620 0x7ff2c815ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:21.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2c67fc640 1 --2- 172.21.15.22:0/663290252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2c80b9c70 0x7ff2c80d2610 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8009870 tx=0x7ff2b801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b063bd8b06d5ad4d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:21.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2c5ffb640 1 -- 172.21.15.22:0/663290252 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80b0070 con 0x7ff2c80b9c70 2024-03-29T04:20:21.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2c5ffb640 1 -- 172.21.15.22:0/663290252 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b80ab600 con 0x7ff2c80b9c70 2024-03-29T04:20:21.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2c5ffb640 1 -- 172.21.15.22:0/663290252 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80ab9d0 con 0x7ff2c80b9c70 2024-03-29T04:20:21.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2cd31b640 1 -- 172.21.15.22:0/663290252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2c80b9c70 msgr2=0x7ff2c80d2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2cd31b640 1 --2- 172.21.15.22:0/663290252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2c80b9c70 0x7ff2c80d2610 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8009870 tx=0x7ff2b801c460 comp rx=0 tx=0).stop 2024-03-29T04:20:21.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2cd31b640 1 -- 172.21.15.22:0/663290252 shutdown_connections 2024-03-29T04:20:21.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2cd31b640 1 --2- 172.21.15.22:0/663290252 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c80d2b50 0x7ff2c80b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2cd31b640 1 --2- 172.21.15.22:0/663290252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2c80b9c70 0x7ff2c80d2610 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2cd31b640 1 --2- 172.21.15.22:0/663290252 >> v2:172.21.15.151:3300/0 conn(0x7ff2c815e620 0x7ff2c815ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2cd31b640 1 -- 172.21.15.22:0/663290252 >> 172.21.15.22:0/663290252 conn(0x7ff2c80b62a0 msgr2=0x7ff2c80b66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:21.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2cd31b640 1 -- 172.21.15.22:0/663290252 shutdown_connections 2024-03-29T04:20:21.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2cd31b640 1 -- 172.21.15.22:0/663290252 wait complete. 2024-03-29T04:20:21.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2cd31b640 1 Processor -- start 2024-03-29T04:20:21.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2cd31b640 1 -- start start 2024-03-29T04:20:21.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2cd31b640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c80b9c70 0x7ff2c814c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2cd31b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2c80d2b50 0x7ff2c815d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2cd31b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c815d9a0 0x7ff2c815ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2cd31b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c8162320 con 0x7ff2c80d2b50 2024-03-29T04:20:21.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2cd31b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c81621a0 con 0x7ff2c815d9a0 2024-03-29T04:20:21.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2cd31b640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c81624a0 con 0x7ff2c80b9c70 2024-03-29T04:20:21.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2c67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2c80d2b50 0x7ff2c815d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2c67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2c80d2b50 0x7ff2c815d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52164/0 (socket says 172.21.15.22:52164) 2024-03-29T04:20:21.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2c67fc640 1 -- 172.21.15.22:0/2705026763 learned_addr learned my addr 172.21.15.22:0/2705026763 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:21.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2c67fc640 1 -- 172.21.15.22:0/2705026763 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c80b9c70 msgr2=0x7ff2c814c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2c77fe640 1 --2- 172.21.15.22:0/2705026763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c815d9a0 0x7ff2c815ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2c6ffd640 1 --2- 172.21.15.22:0/2705026763 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c80b9c70 0x7ff2c814c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2c67fc640 1 --2- 172.21.15.22:0/2705026763 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c80b9c70 0x7ff2c814c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2c67fc640 1 -- 172.21.15.22:0/2705026763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c815d9a0 msgr2=0x7ff2c815ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2c67fc640 1 --2- 172.21.15.22:0/2705026763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c815d9a0 0x7ff2c815ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2c67fc640 1 -- 172.21.15.22:0/2705026763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c815b910 con 0x7ff2c80d2b50 2024-03-29T04:20:21.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2c77fe640 1 --2- 172.21.15.22:0/2705026763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c815d9a0 0x7ff2c815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:21.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2c67fc640 1 --2- 172.21.15.22:0/2705026763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2c80d2b50 0x7ff2c815d460 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80ae040 tx=0x7ff2b8019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:21.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2af7fe640 1 -- 172.21.15.22:0/2705026763 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80b0040 con 0x7ff2c80d2b50 2024-03-29T04:20:21.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2af7fe640 1 -- 172.21.15.22:0/2705026763 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b80b4440 con 0x7ff2c80d2b50 2024-03-29T04:20:21.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2cd31b640 1 -- 172.21.15.22:0/2705026763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c815bbf0 con 0x7ff2c80d2b50 2024-03-29T04:20:21.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.179+0000 7ff2af7fe640 1 -- 172.21.15.22:0/2705026763 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80b4850 con 0x7ff2c80d2b50 2024-03-29T04:20:21.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.183+0000 7ff2cd31b640 1 -- 172.21.15.22:0/2705026763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c80c7ff0 con 0x7ff2c80d2b50 2024-03-29T04:20:21.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.183+0000 7ff2cd31b640 1 -- 172.21.15.22:0/2705026763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c8068890 con 0x7ff2c80d2b50 2024-03-29T04:20:21.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.183+0000 7ff2af7fe640 1 -- 172.21.15.22:0/2705026763 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff2b80bb020 con 0x7ff2c80d2b50 2024-03-29T04:20:21.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.183+0000 7ff2af7fe640 1 --2- 172.21.15.22:0/2705026763 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff2b4067ff0 0x7ff2b406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.183+0000 7ff2af7fe640 1 -- 172.21.15.22:0/2705026763 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7ff2b811f9c0 con 0x7ff2c80d2b50 2024-03-29T04:20:21.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.183+0000 7ff2c6ffd640 1 --2- 172.21.15.22:0/2705026763 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff2b4067ff0 0x7ff2b406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.183+0000 7ff2c6ffd640 1 --2- 172.21.15.22:0/2705026763 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff2b4067ff0 0x7ff2b406a4b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc006fd0 tx=0x7ff2bc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:21.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.187+0000 7ff2af7fe640 1 -- 172.21.15.22:0/2705026763 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff2b80c2020 con 0x7ff2c80d2b50 2024-03-29T04:20:21.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.351+0000 7ff2cd31b640 1 -- 172.21.15.22:0/2705026763 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7ff2c80c7730 con 0x7ff2b4067ff0 2024-03-29T04:20:21.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.351+0000 7ff2af7fe640 1 -- 172.21.15.22:0/2705026763 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff2c80c7730 con 0x7ff2b4067ff0 2024-03-29T04:20:21.333 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 1 to deep-scrub 2024-03-29T04:20:21.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.355+0000 7ff2cd31b640 1 -- 172.21.15.22:0/2705026763 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff2b4067ff0 msgr2=0x7ff2b406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.355+0000 7ff2cd31b640 1 --2- 172.21.15.22:0/2705026763 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff2b4067ff0 0x7ff2b406a4b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc006fd0 tx=0x7ff2bc008040 comp rx=0 tx=0).stop 2024-03-29T04:20:21.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.355+0000 7ff2cd31b640 1 -- 172.21.15.22:0/2705026763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2c80d2b50 msgr2=0x7ff2c815d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.355+0000 7ff2cd31b640 1 --2- 172.21.15.22:0/2705026763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2c80d2b50 0x7ff2c815d460 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80ae040 tx=0x7ff2b8019200 comp rx=0 tx=0).stop 2024-03-29T04:20:21.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.355+0000 7ff2cd31b640 1 -- 172.21.15.22:0/2705026763 shutdown_connections 2024-03-29T04:20:21.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.355+0000 7ff2cd31b640 1 --2- 172.21.15.22:0/2705026763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c815d9a0 0x7ff2c815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.355+0000 7ff2cd31b640 1 --2- 172.21.15.22:0/2705026763 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff2b4067ff0 0x7ff2b406a4b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.355+0000 7ff2cd31b640 1 --2- 172.21.15.22:0/2705026763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2c80d2b50 0x7ff2c815d460 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.355+0000 7ff2cd31b640 1 --2- 172.21.15.22:0/2705026763 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff2c80b9c70 0x7ff2c814c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.355+0000 7ff2cd31b640 1 -- 172.21.15.22:0/2705026763 >> 172.21.15.22:0/2705026763 conn(0x7ff2c80b62a0 msgr2=0x7ff2c80b6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:21.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.355+0000 7ff2cd31b640 1 -- 172.21.15.22:0/2705026763 shutdown_connections 2024-03-29T04:20:21.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.355+0000 7ff2cd31b640 1 -- 172.21.15.22:0/2705026763 wait complete. 2024-03-29T04:20:21.353 INFO:tasks.ceph:Scrubbing osd.2 2024-03-29T04:20:21.353 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:20:21.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.503+0000 7f32a48b1640 1 Processor -- start 2024-03-29T04:20:21.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.503+0000 7f32a48b1640 1 -- start start 2024-03-29T04:20:21.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.503+0000 7f32a48b1640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f32a00d3cf0 0x7f32a00d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.503+0000 7f32a48b1640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32a0074030 con 0x7f32a00b7cd0 2024-03-29T04:20:21.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.503+0000 7f32a48b1640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32a00d2c50 con 0x7f32a0161af0 2024-03-29T04:20:21.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.503+0000 7f32a48b1640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f32a00d4150 con 0x7f32a00d3cf0 2024-03-29T04:20:21.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.503+0000 7f329ed9d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f32a00b7cd0 0x7f32a006b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:56206/0 (socket says 172.21.15.22:56206) 2024-03-29T04:20:21.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.503+0000 7f329dd9b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f32a00d3cf0 0x7f32a00d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.503+0000 7f329ed9d640 1 -- 172.21.15.22:0/1010328865 learned_addr learned my addr 172.21.15.22:0/1010328865 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:21.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.503+0000 7f329dd9b640 1 -- 172.21.15.22:0/1010328865 >> v1:172.21.15.151:6789/0 conn(0x7f32a0161af0 legacy=0x7f32a00b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:21.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.503+0000 7f329dd9b640 1 -- 172.21.15.22:0/1010328865 >> v1:172.21.15.22:6789/0 conn(0x7f32a00b7cd0 legacy=0x7f32a006b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.503+0000 7f329dd9b640 1 -- 172.21.15.22:0/1010328865 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a00d2f30 con 0x7f32a00d3cf0 2024-03-29T04:20:21.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.503+0000 7f329d59a640 1 -- 172.21.15.22:0/1010328865 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 559053497 0 0) 0x7f32a0074030 con 0x7f32a00b7cd0 2024-03-29T04:20:21.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.507+0000 7f329dd9b640 1 --2- 172.21.15.22:0/1010328865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32a00d3cf0 0x7f32a00d24e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f32940097c0 tx=0x7f329401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e660350fd54f85 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:21.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.507+0000 7f329d59a640 1 -- 172.21.15.22:0/1010328865 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32940b5070 con 0x7f32a00d3cf0 2024-03-29T04:20:21.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.507+0000 7f329d59a640 1 -- 172.21.15.22:0/1010328865 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32940a8950 con 0x7f32a00d3cf0 2024-03-29T04:20:21.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.507+0000 7f329d59a640 1 -- 172.21.15.22:0/1010328865 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32940a8d20 con 0x7f32a00d3cf0 2024-03-29T04:20:21.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.507+0000 7f32a48b1640 1 -- 172.21.15.22:0/1010328865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32a00d3cf0 msgr2=0x7f32a00d24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.507+0000 7f32a48b1640 1 --2- 172.21.15.22:0/1010328865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32a00d3cf0 0x7f32a00d24e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f32940097c0 tx=0x7f329401ec50 comp rx=0 tx=0).stop 2024-03-29T04:20:21.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.507+0000 7f32a48b1640 1 -- 172.21.15.22:0/1010328865 shutdown_connections 2024-03-29T04:20:21.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.507+0000 7f32a48b1640 1 --2- 172.21.15.22:0/1010328865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32a00d3cf0 0x7f32a00d24e0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.507+0000 7f32a48b1640 1 -- 172.21.15.22:0/1010328865 >> 172.21.15.22:0/1010328865 conn(0x7f32a00d42e0 msgr2=0x7f32a00d46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:21.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.507+0000 7f32a48b1640 1 -- 172.21.15.22:0/1010328865 shutdown_connections 2024-03-29T04:20:21.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.507+0000 7f32a48b1640 1 -- 172.21.15.22:0/1010328865 wait complete. 2024-03-29T04:20:21.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f32a48b1640 1 Processor -- start 2024-03-29T04:20:21.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f32a48b1640 1 -- start start 2024-03-29T04:20:21.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f32a48b1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32a00b7cd0 0x7f32a014f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f32a48b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32a00d3cf0 0x7f32a00c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f32a48b1640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f32a0161af0 0x7f32a00b8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f32a48b1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f32a00d34f0 con 0x7f32a00d3cf0 2024-03-29T04:20:21.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f32a48b1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f32a00d37d0 con 0x7f32a00b7cd0 2024-03-29T04:20:21.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f32a48b1640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f32a0161ed0 con 0x7f32a0161af0 2024-03-29T04:20:21.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f329dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32a00d3cf0 0x7f32a00c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f329dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32a00d3cf0 0x7f32a00c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52186/0 (socket says 172.21.15.22:52186) 2024-03-29T04:20:21.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f329dd9b640 1 -- 172.21.15.22:0/2635364873 learned_addr learned my addr 172.21.15.22:0/2635364873 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:21.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f329e59c640 1 --2- 172.21.15.22:0/2635364873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32a00b7cd0 0x7f32a014f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f329ed9d640 1 --2- 172.21.15.22:0/2635364873 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f32a0161af0 0x7f32a00b8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f329dd9b640 1 -- 172.21.15.22:0/2635364873 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f32a0161af0 msgr2=0x7f32a00b8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f329dd9b640 1 --2- 172.21.15.22:0/2635364873 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f32a0161af0 0x7f32a00b8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f329dd9b640 1 -- 172.21.15.22:0/2635364873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32a00b7cd0 msgr2=0x7f32a014f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f329dd9b640 1 --2- 172.21.15.22:0/2635364873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32a00b7cd0 0x7f32a014f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f329dd9b640 1 -- 172.21.15.22:0/2635364873 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a00c9880 con 0x7f32a00d3cf0 2024-03-29T04:20:21.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f329ed9d640 1 --2- 172.21.15.22:0/2635364873 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f32a0161af0 0x7f32a00b8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:21.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f329dd9b640 1 --2- 172.21.15.22:0/2635364873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32a00d3cf0 0x7f32a00c8e00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f32940b3040 tx=0x7f329401b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:21.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f328effd640 1 -- 172.21.15.22:0/2635364873 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32940b5040 con 0x7f32a00d3cf0 2024-03-29T04:20:21.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f328effd640 1 -- 172.21.15.22:0/2635364873 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32940c0070 con 0x7f32a00d3cf0 2024-03-29T04:20:21.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f328effd640 1 -- 172.21.15.22:0/2635364873 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32940b0aa0 con 0x7f32a00d3cf0 2024-03-29T04:20:21.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f32a48b1640 1 -- 172.21.15.22:0/2635364873 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a00c9b60 con 0x7f32a00d3cf0 2024-03-29T04:20:21.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f32a48b1640 1 -- 172.21.15.22:0/2635364873 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a015a770 con 0x7f32a00d3cf0 2024-03-29T04:20:21.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f32a48b1640 1 -- 172.21.15.22:0/2635364873 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f32a00687f0 con 0x7f32a00d3cf0 2024-03-29T04:20:21.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.511+0000 7f328effd640 1 -- 172.21.15.22:0/2635364873 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f32940b0c40 con 0x7f32a00d3cf0 2024-03-29T04:20:21.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.515+0000 7f328effd640 1 --2- 172.21.15.22:0/2635364873 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3280067fc0 0x7f328006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.515+0000 7f328effd640 1 -- 172.21.15.22:0/2635364873 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f3294123a30 con 0x7f32a00d3cf0 2024-03-29T04:20:21.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.515+0000 7f328effd640 1 --2- 172.21.15.22:0/2635364873 >> [v2:172.21.15.22:6809/2216395437,v1:172.21.15.22:6811/2216395437] conn(0x7f328006e460 0x7f32800708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.515+0000 7f328effd640 1 -- 172.21.15.22:0/2635364873 --> [v2:172.21.15.22:6809/2216395437,v1:172.21.15.22:6811/2216395437] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3294007e50 con 0x7f328006e460 2024-03-29T04:20:21.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.515+0000 7f328effd640 1 -- 172.21.15.22:0/2635364873 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f32941282a0 con 0x7f32a00d3cf0 2024-03-29T04:20:21.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.515+0000 7f329ed9d640 1 --2- 172.21.15.22:0/2635364873 >> [v2:172.21.15.22:6809/2216395437,v1:172.21.15.22:6811/2216395437] conn(0x7f328006e460 0x7f32800708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.515+0000 7f329e59c640 1 --2- 172.21.15.22:0/2635364873 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3280067fc0 0x7f328006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.515+0000 7f329ed9d640 1 --2- 172.21.15.22:0/2635364873 >> [v2:172.21.15.22:6809/2216395437,v1:172.21.15.22:6811/2216395437] conn(0x7f328006e460 0x7f32800708a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:21.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.515+0000 7f329e59c640 1 --2- 172.21.15.22:0/2635364873 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3280067fc0 0x7f328006a480 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f32a0160390 tx=0x7f32900094d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:21.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.515+0000 7f328effd640 1 -- 172.21.15.22:0/2635364873 <== osd.2 v2:172.21.15.22:6809/2216395437 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3294007e50 con 0x7f328006e460 2024-03-29T04:20:21.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.527+0000 7f32a48b1640 1 -- 172.21.15.22:0/2635364873 --> [v2:172.21.15.22:6809/2216395437,v1:172.21.15.22:6811/2216395437] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f32a015a260 con 0x7f328006e460 2024-03-29T04:20:21.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.527+0000 7f328effd640 1 -- 172.21.15.22:0/2635364873 <== osd.2 v2:172.21.15.22:6809/2216395437 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f32a015a260 con 0x7f328006e460 2024-03-29T04:20:21.511 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-29T04:20:21.511 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:20:21.511 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-29T04:20:21.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.527+0000 7f32a48b1640 1 -- 172.21.15.22:0/2635364873 >> [v2:172.21.15.22:6809/2216395437,v1:172.21.15.22:6811/2216395437] conn(0x7f328006e460 msgr2=0x7f32800708a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.527+0000 7f32a48b1640 1 --2- 172.21.15.22:0/2635364873 >> [v2:172.21.15.22:6809/2216395437,v1:172.21.15.22:6811/2216395437] conn(0x7f328006e460 0x7f32800708a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.527+0000 7f32a48b1640 1 -- 172.21.15.22:0/2635364873 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3280067fc0 msgr2=0x7f328006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.527+0000 7f32a48b1640 1 --2- 172.21.15.22:0/2635364873 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3280067fc0 0x7f328006a480 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f32a0160390 tx=0x7f32900094d0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.527+0000 7f32a48b1640 1 -- 172.21.15.22:0/2635364873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32a00d3cf0 msgr2=0x7f32a00c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.527+0000 7f32a48b1640 1 --2- 172.21.15.22:0/2635364873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32a00d3cf0 0x7f32a00c8e00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f32940b3040 tx=0x7f329401b9f0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.527+0000 7f329ed9d640 1 -- 172.21.15.22:0/2635364873 reap_dead start 2024-03-29T04:20:21.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.527+0000 7f32a48b1640 1 -- 172.21.15.22:0/2635364873 shutdown_connections 2024-03-29T04:20:21.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.527+0000 7f32a48b1640 1 -- 172.21.15.22:0/2635364873 >> 172.21.15.22:0/2635364873 conn(0x7f32a00d42e0 msgr2=0x7f32a00d46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:21.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.527+0000 7f32a48b1640 1 -- 172.21.15.22:0/2635364873 shutdown_connections 2024-03-29T04:20:21.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.527+0000 7f32a48b1640 1 -- 172.21.15.22:0/2635364873 wait complete. 2024-03-29T04:20:21.527 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-29T04:20:21.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.663+0000 7f3b5a724640 1 Processor -- start 2024-03-29T04:20:21.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.663+0000 7f3b5a724640 1 -- start start 2024-03-29T04:20:21.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.667+0000 7f3b5a724640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f3b540cef80 0x7f3b540b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.667+0000 7f3b5a724640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3b540cf360 0x7f3b540cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.667+0000 7f3b5a724640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b540b4500 con 0x7f3b540b3840 2024-03-29T04:20:21.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.667+0000 7f3b5a724640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f3b540b47e0 con 0x7f3b540cf360 2024-03-29T04:20:21.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.667+0000 7f3b5a724640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f3b540b4ac0 con 0x7f3b540cef80 2024-03-29T04:20:21.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.667+0000 7f3b537fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3b540cf360 0x7f3b540cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.667+0000 7f3b537fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3b540cf360 0x7f3b540cf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52188/0 (socket says 172.21.15.22:52188) 2024-03-29T04:20:21.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.667+0000 7f3b537fe640 1 -- 172.21.15.22:0/1538664206 learned_addr learned my addr 172.21.15.22:0/1538664206 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:21.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.667+0000 7f3b53fff640 1 --2- 172.21.15.22:0/1538664206 >> v2:172.21.15.151:3300/0 conn(0x7f3b540cef80 0x7f3b540b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.667+0000 7f3b537fe640 1 -- 172.21.15.22:0/1538664206 >> v2:172.21.15.151:3300/0 conn(0x7f3b540cef80 msgr2=0x7f3b540b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:21.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.667+0000 7f3b537fe640 1 --2- 172.21.15.22:0/1538664206 >> v2:172.21.15.151:3300/0 conn(0x7f3b540cef80 0x7f3b540b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.667+0000 7f3b537fe640 1 -- 172.21.15.22:0/1538664206 >> v1:172.21.15.151:6789/0 conn(0x7f3b540b3840 legacy=0x7f3b540b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:21.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.667+0000 7f3b537fe640 1 -- 172.21.15.22:0/1538664206 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b540b4da0 con 0x7f3b540cf360 2024-03-29T04:20:21.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.667+0000 7f3b537fe640 1 --2- 172.21.15.22:0/1538664206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b540cf360 0x7f3b540cf7f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3b540d53f0 tx=0x7f3b4801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0cc30fc3820176 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:21.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.667+0000 7f3b52ffd640 1 -- 172.21.15.22:0/1538664206 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b480b0070 con 0x7f3b540cf360 2024-03-29T04:20:21.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.667+0000 7f3b52ffd640 1 -- 172.21.15.22:0/1538664206 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b480ab600 con 0x7f3b540cf360 2024-03-29T04:20:21.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.667+0000 7f3b52ffd640 1 -- 172.21.15.22:0/1538664206 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b480ab9d0 con 0x7f3b540cf360 2024-03-29T04:20:21.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b5a724640 1 -- 172.21.15.22:0/1538664206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b540cf360 msgr2=0x7f3b540cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b5a724640 1 --2- 172.21.15.22:0/1538664206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b540cf360 0x7f3b540cf7f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3b540d53f0 tx=0x7f3b4801c4e0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b5a724640 1 -- 172.21.15.22:0/1538664206 shutdown_connections 2024-03-29T04:20:21.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b5a724640 1 --2- 172.21.15.22:0/1538664206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b540cf360 0x7f3b540cf7f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b5a724640 1 --2- 172.21.15.22:0/1538664206 >> v2:172.21.15.151:3300/0 conn(0x7f3b540cef80 0x7f3b540b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b5a724640 1 -- 172.21.15.22:0/1538664206 >> 172.21.15.22:0/1538664206 conn(0x7f3b540d4310 msgr2=0x7f3b540d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:21.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b5a724640 1 -- 172.21.15.22:0/1538664206 shutdown_connections 2024-03-29T04:20:21.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b5a724640 1 -- 172.21.15.22:0/1538664206 wait complete. 2024-03-29T04:20:21.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b5a724640 1 Processor -- start 2024-03-29T04:20:21.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b5a724640 1 -- start start 2024-03-29T04:20:21.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b5a724640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b540b3840 0x7f3b54152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b5a724640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b540cef80 0x7f3b541503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b5a724640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b540cf360 0x7f3b5415c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b5a724640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5416aeb0 con 0x7f3b540cef80 2024-03-29T04:20:21.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b5a724640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5416ad30 con 0x7f3b540b3840 2024-03-29T04:20:21.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b5a724640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f3b5416b030 con 0x7f3b540cf360 2024-03-29T04:20:21.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b540cef80 0x7f3b541503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b540cef80 0x7f3b541503b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52192/0 (socket says 172.21.15.22:52192) 2024-03-29T04:20:21.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b537fe640 1 -- 172.21.15.22:0/1745767699 learned_addr learned my addr 172.21.15.22:0/1745767699 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:21.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b53fff640 1 --2- 172.21.15.22:0/1745767699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b540b3840 0x7f3b54152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b58cc1640 1 --2- 172.21.15.22:0/1745767699 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b540cf360 0x7f3b5415c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b537fe640 1 -- 172.21.15.22:0/1745767699 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b540cf360 msgr2=0x7f3b5415c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b537fe640 1 --2- 172.21.15.22:0/1745767699 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b540cf360 0x7f3b5415c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b537fe640 1 -- 172.21.15.22:0/1745767699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b540b3840 msgr2=0x7f3b54152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b537fe640 1 --2- 172.21.15.22:0/1745767699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b540b3840 0x7f3b54152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b537fe640 1 -- 172.21.15.22:0/1745767699 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5414f910 con 0x7f3b540cef80 2024-03-29T04:20:21.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b58cc1640 1 --2- 172.21.15.22:0/1745767699 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b540cf360 0x7f3b5415c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:21.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b537fe640 1 --2- 172.21.15.22:0/1745767699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b540cef80 0x7f3b541503b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3b4801a110 tx=0x7f3b48019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:21.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b50ff9640 1 -- 172.21.15.22:0/1745767699 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b480b9070 con 0x7f3b540cef80 2024-03-29T04:20:21.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b50ff9640 1 -- 172.21.15.22:0/1745767699 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b480b4600 con 0x7f3b540cef80 2024-03-29T04:20:21.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b5a724640 1 -- 172.21.15.22:0/1745767699 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b541618c0 con 0x7f3b540cef80 2024-03-29T04:20:21.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b50ff9640 1 -- 172.21.15.22:0/1745767699 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b480b0040 con 0x7f3b540cef80 2024-03-29T04:20:21.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.671+0000 7f3b5a724640 1 -- 172.21.15.22:0/1745767699 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b541531c0 con 0x7f3b540cef80 2024-03-29T04:20:21.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.675+0000 7f3b5a724640 1 -- 172.21.15.22:0/1745767699 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b54068890 con 0x7f3b540cef80 2024-03-29T04:20:21.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.675+0000 7f3b50ff9640 1 -- 172.21.15.22:0/1745767699 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f3b480bc070 con 0x7f3b540cef80 2024-03-29T04:20:21.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.675+0000 7f3b50ff9640 1 --2- 172.21.15.22:0/1745767699 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b40067fc0 0x7f3b4006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.675+0000 7f3b50ff9640 1 -- 172.21.15.22:0/1745767699 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f3b4811f510 con 0x7f3b540cef80 2024-03-29T04:20:21.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.675+0000 7f3b53fff640 1 --2- 172.21.15.22:0/1745767699 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b40067fc0 0x7f3b4006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.675+0000 7f3b53fff640 1 --2- 172.21.15.22:0/1745767699 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b40067fc0 0x7f3b4006a480 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400b9d0 tx=0x7f3b44008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:21.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.679+0000 7f3b50ff9640 1 -- 172.21.15.22:0/1745767699 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f3b480160d0 con 0x7f3b540cef80 2024-03-29T04:20:21.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.839+0000 7f3b5a724640 1 -- 172.21.15.22:0/1745767699 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f3b540d2610 con 0x7f3b40067fc0 2024-03-29T04:20:21.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.839+0000 7f3b50ff9640 1 -- 172.21.15.22:0/1745767699 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3b540d2610 con 0x7f3b40067fc0 2024-03-29T04:20:21.820 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 2 to deep-scrub 2024-03-29T04:20:21.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.839+0000 7f3b5a724640 1 -- 172.21.15.22:0/1745767699 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b40067fc0 msgr2=0x7f3b4006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.839+0000 7f3b5a724640 1 --2- 172.21.15.22:0/1745767699 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b40067fc0 0x7f3b4006a480 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400b9d0 tx=0x7f3b44008040 comp rx=0 tx=0).stop 2024-03-29T04:20:21.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.839+0000 7f3b5a724640 1 -- 172.21.15.22:0/1745767699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b540cef80 msgr2=0x7f3b541503b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.839+0000 7f3b5a724640 1 --2- 172.21.15.22:0/1745767699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b540cef80 0x7f3b541503b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3b4801a110 tx=0x7f3b48019280 comp rx=0 tx=0).stop 2024-03-29T04:20:21.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.839+0000 7f3b5a724640 1 -- 172.21.15.22:0/1745767699 shutdown_connections 2024-03-29T04:20:21.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.839+0000 7f3b5a724640 1 --2- 172.21.15.22:0/1745767699 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3b540cf360 0x7f3b5415c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.839+0000 7f3b5a724640 1 --2- 172.21.15.22:0/1745767699 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3b40067fc0 0x7f3b4006a480 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.839+0000 7f3b5a724640 1 --2- 172.21.15.22:0/1745767699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b540cef80 0x7f3b541503b0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.843+0000 7f3b5a724640 1 --2- 172.21.15.22:0/1745767699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b540b3840 0x7f3b54152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.843+0000 7f3b5a724640 1 -- 172.21.15.22:0/1745767699 >> 172.21.15.22:0/1745767699 conn(0x7f3b540d4310 msgr2=0x7f3b540d2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:21.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.843+0000 7f3b5a724640 1 -- 172.21.15.22:0/1745767699 shutdown_connections 2024-03-29T04:20:21.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.843+0000 7f3b5a724640 1 -- 172.21.15.22:0/1745767699 wait complete. 2024-03-29T04:20:21.839 INFO:tasks.ceph:Scrubbing osd.3 2024-03-29T04:20:21.839 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:20:21.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.975+0000 7faa50a66640 1 Processor -- start 2024-03-29T04:20:21.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.975+0000 7faa50a66640 1 -- start start 2024-03-29T04:20:21.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.975+0000 7faa50a66640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7faa4c15cf10 0x7faa4c166da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.975+0000 7faa50a66640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa4c15d320 con 0x7faa4c13f030 2024-03-29T04:20:21.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.975+0000 7faa50a66640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa4c142640 con 0x7faa4c06f2e0 2024-03-29T04:20:21.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.975+0000 7faa50a66640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7faa4c13f410 con 0x7faa4c15cf10 2024-03-29T04:20:21.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.975+0000 7faa4ad9d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7faa4c15cf10 0x7faa4c166da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.975+0000 7faa4a59c640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7faa4c13f030 0x7faa4c06ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:56210/0 (socket says 172.21.15.22:56210) 2024-03-29T04:20:21.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.975+0000 7faa4a59c640 1 -- 172.21.15.22:0/3711871688 learned_addr learned my addr 172.21.15.22:0/3711871688 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:21.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.975+0000 7faa4ad9d640 1 -- 172.21.15.22:0/3711871688 >> v1:172.21.15.151:6789/0 conn(0x7faa4c06f2e0 legacy=0x7faa4c15c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:21.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.975+0000 7faa4ad9d640 1 -- 172.21.15.22:0/3711871688 >> v1:172.21.15.22:6789/0 conn(0x7faa4c13f030 legacy=0x7faa4c06ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.975+0000 7faa4ad9d640 1 -- 172.21.15.22:0/3711871688 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4c141880 con 0x7faa4c15cf10 2024-03-29T04:20:21.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.975+0000 7faa4959a640 1 -- 172.21.15.22:0/3711871688 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 24825716 0 0) 0x7faa40001120 con 0x7faa4c13f030 2024-03-29T04:20:21.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa4ad9d640 1 --2- 172.21.15.22:0/3711871688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa4c15cf10 0x7faa4c166da0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7faa44009d90 tx=0x7faa4401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d5a32b7642303d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:21.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa4959a640 1 -- 172.21.15.22:0/3711871688 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa440b2070 con 0x7faa4c15cf10 2024-03-29T04:20:21.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa4959a640 1 -- 172.21.15.22:0/3711871688 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa440a5760 con 0x7faa4c15cf10 2024-03-29T04:20:21.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa4959a640 1 -- 172.21.15.22:0/3711871688 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa440a5b30 con 0x7faa4c15cf10 2024-03-29T04:20:21.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa50a66640 1 -- 172.21.15.22:0/3711871688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa4c15cf10 msgr2=0x7faa4c166da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa50a66640 1 --2- 172.21.15.22:0/3711871688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa4c15cf10 0x7faa4c166da0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7faa44009d90 tx=0x7faa4401c8a0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa50a66640 1 -- 172.21.15.22:0/3711871688 shutdown_connections 2024-03-29T04:20:21.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa50a66640 1 --2- 172.21.15.22:0/3711871688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa4c15cf10 0x7faa4c166da0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa50a66640 1 -- 172.21.15.22:0/3711871688 >> 172.21.15.22:0/3711871688 conn(0x7faa4c0b3460 msgr2=0x7faa4c06e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:21.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa50a66640 1 -- 172.21.15.22:0/3711871688 shutdown_connections 2024-03-29T04:20:21.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa50a66640 1 -- 172.21.15.22:0/3711871688 wait complete. 2024-03-29T04:20:21.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa50a66640 1 Processor -- start 2024-03-29T04:20:21.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa50a66640 1 -- start start 2024-03-29T04:20:21.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa50a66640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa4c06f2e0 0x7faa4c135710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa50a66640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa4c13f030 0x7faa4c139790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa50a66640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa4c15cf10 0x7faa4c152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa50a66640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c169c80 con 0x7faa4c15cf10 2024-03-29T04:20:21.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa50a66640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c169b00 con 0x7faa4c13f030 2024-03-29T04:20:21.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa50a66640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7faa4c169e00 con 0x7faa4c06f2e0 2024-03-29T04:20:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa4ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa4c15cf10 0x7faa4c152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa4ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa4c15cf10 0x7faa4c152040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52214/0 (socket says 172.21.15.22:52214) 2024-03-29T04:20:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.979+0000 7faa4ad9d640 1 -- 172.21.15.22:0/528073982 learned_addr learned my addr 172.21.15.22:0/528073982 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa4a59c640 1 --2- 172.21.15.22:0/528073982 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa4c06f2e0 0x7faa4c135710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa4ad9d640 1 -- 172.21.15.22:0/528073982 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa4c06f2e0 msgr2=0x7faa4c135710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa4ad9d640 1 --2- 172.21.15.22:0/528073982 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7faa4c06f2e0 0x7faa4c135710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa49d9b640 1 --2- 172.21.15.22:0/528073982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa4c13f030 0x7faa4c139790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa4ad9d640 1 -- 172.21.15.22:0/528073982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa4c13f030 msgr2=0x7faa4c139790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa4ad9d640 1 --2- 172.21.15.22:0/528073982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa4c13f030 0x7faa4c139790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa4ad9d640 1 -- 172.21.15.22:0/528073982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4c13a1e0 con 0x7faa4c15cf10 2024-03-29T04:20:21.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa49d9b640 1 --2- 172.21.15.22:0/528073982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa4c13f030 0x7faa4c139790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:21.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa4ad9d640 1 --2- 172.21.15.22:0/528073982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa4c15cf10 0x7faa4c152040 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7faa440b0040 tx=0x7faa44019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:21.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa3affd640 1 -- 172.21.15.22:0/528073982 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa440bd020 con 0x7faa4c15cf10 2024-03-29T04:20:21.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa3affd640 1 -- 172.21.15.22:0/528073982 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa440b8670 con 0x7faa4c15cf10 2024-03-29T04:20:21.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa50a66640 1 -- 172.21.15.22:0/528073982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa4c13a4c0 con 0x7faa4c15cf10 2024-03-29T04:20:21.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa3affd640 1 -- 172.21.15.22:0/528073982 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa440b2040 con 0x7faa4c15cf10 2024-03-29T04:20:21.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa50a66640 1 -- 172.21.15.22:0/528073982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa4c13aa80 con 0x7faa4c15cf10 2024-03-29T04:20:21.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa50a66640 1 -- 172.21.15.22:0/528073982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faa4c0687f0 con 0x7faa4c15cf10 2024-03-29T04:20:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa3affd640 1 -- 172.21.15.22:0/528073982 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7faa440c0020 con 0x7faa4c15cf10 2024-03-29T04:20:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa3affd640 1 --2- 172.21.15.22:0/528073982 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faa2c067fc0 0x7faa2c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa3affd640 1 -- 172.21.15.22:0/528073982 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7faa441219c0 con 0x7faa4c15cf10 2024-03-29T04:20:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa3affd640 1 --2- 172.21.15.22:0/528073982 >> [v2:172.21.15.22:6813/583588642,v1:172.21.15.22:6816/583588642] conn(0x7faa2c06e460 0x7faa2c0708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa3affd640 1 -- 172.21.15.22:0/528073982 --> [v2:172.21.15.22:6813/583588642,v1:172.21.15.22:6816/583588642] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faa440c3e50 con 0x7faa2c06e460 2024-03-29T04:20:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa3affd640 1 -- 172.21.15.22:0/528073982 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7faa44121e50 con 0x7faa4c15cf10 2024-03-29T04:20:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa4a59c640 1 --2- 172.21.15.22:0/528073982 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faa2c067fc0 0x7faa2c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa49d9b640 1 --2- 172.21.15.22:0/528073982 >> [v2:172.21.15.22:6813/583588642,v1:172.21.15.22:6816/583588642] conn(0x7faa2c06e460 0x7faa2c0708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa49d9b640 1 --2- 172.21.15.22:0/528073982 >> [v2:172.21.15.22:6813/583588642,v1:172.21.15.22:6816/583588642] conn(0x7faa2c06e460 0x7faa2c0708a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.983+0000 7faa4a59c640 1 --2- 172.21.15.22:0/528073982 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faa2c067fc0 0x7faa2c06a480 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faa40000f70 tx=0x7faa40009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.987+0000 7faa3affd640 1 -- 172.21.15.22:0/528073982 <== osd.3 v2:172.21.15.22:6813/583588642 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faa440c3e50 con 0x7faa2c06e460 2024-03-29T04:20:21.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.999+0000 7faa50a66640 1 -- 172.21.15.22:0/528073982 --> [v2:172.21.15.22:6813/583588642,v1:172.21.15.22:6816/583588642] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7faa4c15cc10 con 0x7faa2c06e460 2024-03-29T04:20:21.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.999+0000 7faa3affd640 1 -- 172.21.15.22:0/528073982 <== osd.3 v2:172.21.15.22:6813/583588642 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7faa4c15cc10 con 0x7faa2c06e460 2024-03-29T04:20:21.980 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-29T04:20:21.980 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:20:21.980 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-29T04:20:21.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.999+0000 7faa50a66640 1 -- 172.21.15.22:0/528073982 >> [v2:172.21.15.22:6813/583588642,v1:172.21.15.22:6816/583588642] conn(0x7faa2c06e460 msgr2=0x7faa2c0708a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.999+0000 7faa50a66640 1 --2- 172.21.15.22:0/528073982 >> [v2:172.21.15.22:6813/583588642,v1:172.21.15.22:6816/583588642] conn(0x7faa2c06e460 0x7faa2c0708a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:21.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.999+0000 7faa50a66640 1 -- 172.21.15.22:0/528073982 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faa2c067fc0 msgr2=0x7faa2c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.999+0000 7faa50a66640 1 --2- 172.21.15.22:0/528073982 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7faa2c067fc0 0x7faa2c06a480 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faa40000f70 tx=0x7faa40009040 comp rx=0 tx=0).stop 2024-03-29T04:20:21.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.999+0000 7faa50a66640 1 -- 172.21.15.22:0/528073982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa4c15cf10 msgr2=0x7faa4c152040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:21.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.999+0000 7faa50a66640 1 --2- 172.21.15.22:0/528073982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa4c15cf10 0x7faa4c152040 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7faa440b0040 tx=0x7faa44019640 comp rx=0 tx=0).stop 2024-03-29T04:20:21.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.999+0000 7faa4ad9d640 1 -- 172.21.15.22:0/528073982 reap_dead start 2024-03-29T04:20:21.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.999+0000 7faa50a66640 1 -- 172.21.15.22:0/528073982 shutdown_connections 2024-03-29T04:20:21.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.999+0000 7faa50a66640 1 -- 172.21.15.22:0/528073982 >> 172.21.15.22:0/528073982 conn(0x7faa4c0b3460 msgr2=0x7faa4c0700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:21.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.999+0000 7faa50a66640 1 -- 172.21.15.22:0/528073982 shutdown_connections 2024-03-29T04:20:21.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:21.999+0000 7faa50a66640 1 -- 172.21.15.22:0/528073982 wait complete. 2024-03-29T04:20:21.995 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-29T04:20:22.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.127+0000 7f6afac6c640 1 Processor -- start 2024-03-29T04:20:22.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.127+0000 7f6afac6c640 1 -- start start 2024-03-29T04:20:22.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.127+0000 7f6afac6c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6af415e600 0x7f6af40b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.127+0000 7f6afac6c640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6af40d1560 0x7f6af40d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.127+0000 7f6afac6c640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6af40d3e60 con 0x7f6af40d1100 2024-03-29T04:20:22.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.127+0000 7f6afac6c640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f6af40b7450 con 0x7f6af415e600 2024-03-29T04:20:22.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.127+0000 7f6afac6c640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f6af40d4140 con 0x7f6af40d1560 2024-03-29T04:20:22.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.127+0000 7f6af8a08640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6af415e600 0x7f6af40b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.127+0000 7f6af3fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f6af40d1100 0x7f6af40b8160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:56226/0 (socket says 172.21.15.22:56226) 2024-03-29T04:20:22.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.127+0000 7f6af3fff640 1 -- 172.21.15.22:0/1144353088 learned_addr learned my addr 172.21.15.22:0/1144353088 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:22.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.127+0000 7f6af8a08640 1 -- 172.21.15.22:0/1144353088 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6af40d1560 msgr2=0x7f6af40d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:22.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.127+0000 7f6af8a08640 1 --2- 172.21.15.22:0/1144353088 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6af40d1560 0x7f6af40d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.127+0000 7f6af8a08640 1 -- 172.21.15.22:0/1144353088 >> v1:172.21.15.22:6789/0 conn(0x7f6af40d1100 legacy=0x7f6af40b8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.127+0000 7f6af37fe640 1 -- 172.21.15.22:0/1144353088 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3589172046 0 0) 0x7f6af40d3e60 con 0x7f6af40d1100 2024-03-29T04:20:22.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.127+0000 7f6af8a08640 1 -- 172.21.15.22:0/1144353088 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af40b7680 con 0x7f6af415e600 2024-03-29T04:20:22.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.131+0000 7f6af8a08640 1 --2- 172.21.15.22:0/1144353088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6af415e600 0x7f6af40b9f00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40097c0 tx=0x7f6ae401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576fbe23749a810c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:22.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.131+0000 7f6af37fe640 1 -- 172.21.15.22:0/1144353088 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae40a7070 con 0x7f6af415e600 2024-03-29T04:20:22.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.131+0000 7f6af37fe640 1 -- 172.21.15.22:0/1144353088 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae40b0070 con 0x7f6af415e600 2024-03-29T04:20:22.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.131+0000 7f6af37fe640 1 -- 172.21.15.22:0/1144353088 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae40ab890 con 0x7f6af415e600 2024-03-29T04:20:22.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.131+0000 7f6afac6c640 1 -- 172.21.15.22:0/1144353088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6af415e600 msgr2=0x7f6af40b9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.131+0000 7f6afac6c640 1 --2- 172.21.15.22:0/1144353088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6af415e600 0x7f6af40b9f00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40097c0 tx=0x7f6ae401c470 comp rx=0 tx=0).stop 2024-03-29T04:20:22.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.131+0000 7f6afac6c640 1 -- 172.21.15.22:0/1144353088 shutdown_connections 2024-03-29T04:20:22.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.131+0000 7f6afac6c640 1 --2- 172.21.15.22:0/1144353088 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6af40d1560 0x7f6af40d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.131+0000 7f6afac6c640 1 --2- 172.21.15.22:0/1144353088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6af415e600 0x7f6af40b9f00 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.131+0000 7f6afac6c640 1 -- 172.21.15.22:0/1144353088 >> 172.21.15.22:0/1144353088 conn(0x7f6af40b6230 msgr2=0x7f6af40b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:22.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.131+0000 7f6afac6c640 1 -- 172.21.15.22:0/1144353088 shutdown_connections 2024-03-29T04:20:22.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.131+0000 7f6afac6c640 1 -- 172.21.15.22:0/1144353088 wait complete. 2024-03-29T04:20:22.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6afac6c640 1 Processor -- start 2024-03-29T04:20:22.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6afac6c640 1 -- start start 2024-03-29T04:20:22.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6afac6c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6af40d1100 0x7f6af40cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6afac6c640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6af40d1560 0x7f6af4149880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6afac6c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6af415e600 0x7f6af414dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6afac6c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6af40b9080 con 0x7f6af40d1100 2024-03-29T04:20:22.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6afac6c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6af40d3b50 con 0x7f6af415e600 2024-03-29T04:20:22.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6afac6c640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f6af40b9200 con 0x7f6af40d1560 2024-03-29T04:20:22.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6af8a08640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6af40d1100 0x7f6af40cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6af8a08640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6af40d1100 0x7f6af40cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52234/0 (socket says 172.21.15.22:52234) 2024-03-29T04:20:22.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6af8a08640 1 -- 172.21.15.22:0/4273863129 learned_addr learned my addr 172.21.15.22:0/4273863129 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:22.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6af9209640 1 --2- 172.21.15.22:0/4273863129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6af415e600 0x7f6af414dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6af3fff640 1 --2- 172.21.15.22:0/4273863129 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6af40d1560 0x7f6af4149880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6af8a08640 1 -- 172.21.15.22:0/4273863129 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6af40d1560 msgr2=0x7f6af4149880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6af8a08640 1 --2- 172.21.15.22:0/4273863129 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6af40d1560 0x7f6af4149880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6af8a08640 1 -- 172.21.15.22:0/4273863129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6af415e600 msgr2=0x7f6af414dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6af8a08640 1 --2- 172.21.15.22:0/4273863129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6af415e600 0x7f6af414dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6af8a08640 1 -- 172.21.15.22:0/4273863129 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af40cb490 con 0x7f6af40d1100 2024-03-29T04:20:22.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6af3fff640 1 --2- 172.21.15.22:0/4273863129 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6af40d1560 0x7f6af4149880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:22.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6af8a08640 1 --2- 172.21.15.22:0/4273863129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6af40d1100 0x7f6af40cc530 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6ae401a0a0 tx=0x7f6ae401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:22.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6af17fa640 1 -- 172.21.15.22:0/4273863129 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4004220 con 0x7f6af40d1100 2024-03-29T04:20:22.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6af17fa640 1 -- 172.21.15.22:0/4273863129 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae40043c0 con 0x7f6af40d1100 2024-03-29T04:20:22.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6afac6c640 1 -- 172.21.15.22:0/4273863129 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af40cb770 con 0x7f6af40d1100 2024-03-29T04:20:22.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6af17fa640 1 -- 172.21.15.22:0/4273863129 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4003820 con 0x7f6af40d1100 2024-03-29T04:20:22.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6afac6c640 1 -- 172.21.15.22:0/4273863129 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af40cbd30 con 0x7f6af40d1100 2024-03-29T04:20:22.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6afac6c640 1 -- 172.21.15.22:0/4273863129 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af4068890 con 0x7f6af40d1100 2024-03-29T04:20:22.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.135+0000 7f6af17fa640 1 -- 172.21.15.22:0/4273863129 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f6ae40039c0 con 0x7f6af40d1100 2024-03-29T04:20:22.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.139+0000 7f6af17fa640 1 --2- 172.21.15.22:0/4273863129 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6ae0067fc0 0x7f6ae006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.139+0000 7f6af17fa640 1 -- 172.21.15.22:0/4273863129 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f6ae41210c0 con 0x7f6af40d1100 2024-03-29T04:20:22.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.139+0000 7f6af3fff640 1 --2- 172.21.15.22:0/4273863129 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6ae0067fc0 0x7f6ae006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.139+0000 7f6af3fff640 1 --2- 172.21.15.22:0/4273863129 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6ae0067fc0 0x7f6ae006a480 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8007fd0 tx=0x7f6ae8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:22.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.139+0000 7f6af17fa640 1 -- 172.21.15.22:0/4273863129 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f6ae4016090 con 0x7f6af40d1100 2024-03-29T04:20:22.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.299+0000 7f6afac6c640 1 -- 172.21.15.22:0/4273863129 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f6af4149dc0 con 0x7f6ae0067fc0 2024-03-29T04:20:22.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.303+0000 7f6af17fa640 1 -- 172.21.15.22:0/4273863129 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6af4149dc0 con 0x7f6ae0067fc0 2024-03-29T04:20:22.283 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 3 to deep-scrub 2024-03-29T04:20:22.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.303+0000 7f6afac6c640 1 -- 172.21.15.22:0/4273863129 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6ae0067fc0 msgr2=0x7f6ae006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.303+0000 7f6afac6c640 1 --2- 172.21.15.22:0/4273863129 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6ae0067fc0 0x7f6ae006a480 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8007fd0 tx=0x7f6ae8009040 comp rx=0 tx=0).stop 2024-03-29T04:20:22.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.303+0000 7f6afac6c640 1 -- 172.21.15.22:0/4273863129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6af40d1100 msgr2=0x7f6af40cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.303+0000 7f6afac6c640 1 --2- 172.21.15.22:0/4273863129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6af40d1100 0x7f6af40cc530 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6ae401a0a0 tx=0x7f6ae401acb0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.303+0000 7f6afac6c640 1 -- 172.21.15.22:0/4273863129 shutdown_connections 2024-03-29T04:20:22.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.303+0000 7f6afac6c640 1 --2- 172.21.15.22:0/4273863129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6af415e600 0x7f6af414dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.303+0000 7f6afac6c640 1 --2- 172.21.15.22:0/4273863129 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f6af40d1560 0x7f6af4149880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.303+0000 7f6afac6c640 1 --2- 172.21.15.22:0/4273863129 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f6ae0067fc0 0x7f6ae006a480 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.303+0000 7f6afac6c640 1 --2- 172.21.15.22:0/4273863129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6af40d1100 0x7f6af40cc530 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.303+0000 7f6afac6c640 1 -- 172.21.15.22:0/4273863129 >> 172.21.15.22:0/4273863129 conn(0x7f6af40b6230 msgr2=0x7f6af4152350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:22.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.303+0000 7f6afac6c640 1 -- 172.21.15.22:0/4273863129 shutdown_connections 2024-03-29T04:20:22.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.303+0000 7f6afac6c640 1 -- 172.21.15.22:0/4273863129 wait complete. 2024-03-29T04:20:22.301 INFO:tasks.ceph:Scrubbing osd.4 2024-03-29T04:20:22.302 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:20:22.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.443+0000 7f15de1dc640 1 Processor -- start 2024-03-29T04:20:22.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.443+0000 7f15de1dc640 1 -- start start 2024-03-29T04:20:22.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.443+0000 7f15de1dc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f15d815f880 0x7f15d80cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.443+0000 7f15de1dc640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f15d80cc750 0x7f15d80d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.443+0000 7f15de1dc640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15d80d19b0 con 0x7f15d80d0750 2024-03-29T04:20:22.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.443+0000 7f15de1dc640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f15d815fc60 con 0x7f15d815f880 2024-03-29T04:20:22.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.443+0000 7f15de1dc640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f15d80d1f70 con 0x7f15d80cc750 2024-03-29T04:20:22.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.443+0000 7f15d77fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f15d815f880 0x7f15d80cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.443+0000 7f15d77fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f15d815f880 0x7f15d80cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52250/0 (socket says 172.21.15.22:52250) 2024-03-29T04:20:22.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.443+0000 7f15d77fe640 1 -- 172.21.15.22:0/178653630 learned_addr learned my addr 172.21.15.22:0/178653630 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:22.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.443+0000 7f15d77fe640 1 -- 172.21.15.22:0/178653630 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f15d80cc750 msgr2=0x7f15d80d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:22.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.443+0000 7f15d6ffd640 1 --2- 172.21.15.22:0/178653630 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f15d80cc750 0x7f15d80d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.443+0000 7f15d77fe640 1 --2- 172.21.15.22:0/178653630 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f15d80cc750 0x7f15d80d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.443+0000 7f15d77fe640 1 -- 172.21.15.22:0/178653630 >> v1:172.21.15.151:6789/0 conn(0x7f15d80d0750 legacy=0x7f15d815fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.443+0000 7f15d77fe640 1 -- 172.21.15.22:0/178653630 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d80d1c90 con 0x7f15d815f880 2024-03-29T04:20:22.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.443+0000 7f15d6ffd640 1 --2- 172.21.15.22:0/178653630 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f15d80cc750 0x7f15d80d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:22.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.447+0000 7f15d77fe640 1 --2- 172.21.15.22:0/178653630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d815f880 0x7f15d80cfcd0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009870 tx=0x7f15c801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e881c0581af26211 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:22.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.447+0000 7f15d67fc640 1 -- 172.21.15.22:0/178653630 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c80b0070 con 0x7f15d815f880 2024-03-29T04:20:22.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.447+0000 7f15d67fc640 1 -- 172.21.15.22:0/178653630 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15c80ab600 con 0x7f15d815f880 2024-03-29T04:20:22.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.447+0000 7f15d67fc640 1 -- 172.21.15.22:0/178653630 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c80ab9d0 con 0x7f15d815f880 2024-03-29T04:20:22.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.447+0000 7f15de1dc640 1 -- 172.21.15.22:0/178653630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d815f880 msgr2=0x7f15d80cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.447+0000 7f15de1dc640 1 --2- 172.21.15.22:0/178653630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d815f880 0x7f15d80cfcd0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009870 tx=0x7f15c801c4d0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.447+0000 7f15de1dc640 1 -- 172.21.15.22:0/178653630 shutdown_connections 2024-03-29T04:20:22.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.447+0000 7f15de1dc640 1 --2- 172.21.15.22:0/178653630 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f15d80cc750 0x7f15d80d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.447+0000 7f15de1dc640 1 --2- 172.21.15.22:0/178653630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d815f880 0x7f15d80cfcd0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.447+0000 7f15de1dc640 1 -- 172.21.15.22:0/178653630 >> 172.21.15.22:0/178653630 conn(0x7f15d8071b80 msgr2=0x7f15d80b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:22.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.447+0000 7f15de1dc640 1 -- 172.21.15.22:0/178653630 shutdown_connections 2024-03-29T04:20:22.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.447+0000 7f15de1dc640 1 -- 172.21.15.22:0/178653630 wait complete. 2024-03-29T04:20:22.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15de1dc640 1 Processor -- start 2024-03-29T04:20:22.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15de1dc640 1 -- start start 2024-03-29T04:20:22.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15de1dc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15d80cc750 0x7f15d80c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15de1dc640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f15d80d0750 0x7f15d81f8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15de1dc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d815f880 0x7f15d81f9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15de1dc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f15d80d1350 con 0x7f15d815f880 2024-03-29T04:20:22.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15de1dc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f15d80cf4c0 con 0x7f15d80cc750 2024-03-29T04:20:22.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15de1dc640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f15d815dd00 con 0x7f15d80d0750 2024-03-29T04:20:22.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15d7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d815f880 0x7f15d81f9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15d7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d815f880 0x7f15d81f9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52254/0 (socket says 172.21.15.22:52254) 2024-03-29T04:20:22.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15d7fff640 1 -- 172.21.15.22:0/169624901 learned_addr learned my addr 172.21.15.22:0/169624901 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:22.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15d6ffd640 1 --2- 172.21.15.22:0/169624901 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f15d80d0750 0x7f15d81f8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15d77fe640 1 --2- 172.21.15.22:0/169624901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15d80cc750 0x7f15d80c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15d7fff640 1 -- 172.21.15.22:0/169624901 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f15d80d0750 msgr2=0x7f15d81f8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15d7fff640 1 --2- 172.21.15.22:0/169624901 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f15d80d0750 0x7f15d81f8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15d7fff640 1 -- 172.21.15.22:0/169624901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15d80cc750 msgr2=0x7f15d80c6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15d7fff640 1 --2- 172.21.15.22:0/169624901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15d80cc750 0x7f15d80c6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15d7fff640 1 -- 172.21.15.22:0/169624901 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d815dfe0 con 0x7f15d815f880 2024-03-29T04:20:22.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15d77fe640 1 --2- 172.21.15.22:0/169624901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15d80cc750 0x7f15d80c6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:20:22.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15d7fff640 1 --2- 172.21.15.22:0/169624901 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d815f880 0x7f15d81f9360 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f15d000c710 tx=0x7f15d000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:22.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15bffff640 1 -- 172.21.15.22:0/169624901 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d0016020 con 0x7f15d815f880 2024-03-29T04:20:22.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15bffff640 1 -- 172.21.15.22:0/169624901 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15d0006d60 con 0x7f15d815f880 2024-03-29T04:20:22.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15bffff640 1 -- 172.21.15.22:0/169624901 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d0003bc0 con 0x7f15d815f880 2024-03-29T04:20:22.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15de1dc640 1 -- 172.21.15.22:0/169624901 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d815e2c0 con 0x7f15d815f880 2024-03-29T04:20:22.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15de1dc640 1 -- 172.21.15.22:0/169624901 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d815e880 con 0x7f15d815f880 2024-03-29T04:20:22.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15de1dc640 1 -- 172.21.15.22:0/169624901 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f15d80687f0 con 0x7f15d815f880 2024-03-29T04:20:22.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.451+0000 7f15bffff640 1 -- 172.21.15.22:0/169624901 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f15d000a850 con 0x7f15d815f880 2024-03-29T04:20:22.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.455+0000 7f15bffff640 1 --2- 172.21.15.22:0/169624901 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f15c4067fc0 0x7f15c406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.455+0000 7f15bffff640 1 -- 172.21.15.22:0/169624901 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f15d0086d70 con 0x7f15d815f880 2024-03-29T04:20:22.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.455+0000 7f15bffff640 1 --2- 172.21.15.22:0/169624901 >> [v2:172.21.15.151:6816/2643908371,v1:172.21.15.151:6817/2643908371] conn(0x7f15c406e460 0x7f15c40708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.455+0000 7f15bffff640 1 -- 172.21.15.22:0/169624901 --> [v2:172.21.15.151:6816/2643908371,v1:172.21.15.151:6817/2643908371] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f15d0017e70 con 0x7f15c406e460 2024-03-29T04:20:22.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.455+0000 7f15bffff640 1 -- 172.21.15.22:0/169624901 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f15d0087200 con 0x7f15d815f880 2024-03-29T04:20:22.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.455+0000 7f15d77fe640 1 --2- 172.21.15.22:0/169624901 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f15c4067fc0 0x7f15c406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.455+0000 7f15d6ffd640 1 --2- 172.21.15.22:0/169624901 >> [v2:172.21.15.151:6816/2643908371,v1:172.21.15.151:6817/2643908371] conn(0x7f15c406e460 0x7f15c40708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.455+0000 7f15d77fe640 1 --2- 172.21.15.22:0/169624901 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f15c4067fc0 0x7f15c406a480 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f15c80ad000 tx=0x7f15c8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:22.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.455+0000 7f15d6ffd640 1 --2- 172.21.15.22:0/169624901 >> [v2:172.21.15.151:6816/2643908371,v1:172.21.15.151:6817/2643908371] conn(0x7f15c406e460 0x7f15c40708a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:22.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.455+0000 7f15bffff640 1 -- 172.21.15.22:0/169624901 <== osd.4 v2:172.21.15.151:6816/2643908371 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f15d0017e70 con 0x7f15c406e460 2024-03-29T04:20:22.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.471+0000 7f15de1dc640 1 -- 172.21.15.22:0/169624901 --> [v2:172.21.15.151:6816/2643908371,v1:172.21.15.151:6817/2643908371] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f15d80b3230 con 0x7f15c406e460 2024-03-29T04:20:22.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.471+0000 7f15bffff640 1 -- 172.21.15.22:0/169624901 <== osd.4 v2:172.21.15.151:6816/2643908371 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f15d80b3230 con 0x7f15c406e460 2024-03-29T04:20:22.451 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-29T04:20:22.452 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:20:22.452 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-29T04:20:22.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.471+0000 7f15de1dc640 1 -- 172.21.15.22:0/169624901 >> [v2:172.21.15.151:6816/2643908371,v1:172.21.15.151:6817/2643908371] conn(0x7f15c406e460 msgr2=0x7f15c40708a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.471+0000 7f15de1dc640 1 --2- 172.21.15.22:0/169624901 >> [v2:172.21.15.151:6816/2643908371,v1:172.21.15.151:6817/2643908371] conn(0x7f15c406e460 0x7f15c40708a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.471+0000 7f15de1dc640 1 -- 172.21.15.22:0/169624901 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f15c4067fc0 msgr2=0x7f15c406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.471+0000 7f15de1dc640 1 --2- 172.21.15.22:0/169624901 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f15c4067fc0 0x7f15c406a480 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f15c80ad000 tx=0x7f15c8008040 comp rx=0 tx=0).stop 2024-03-29T04:20:22.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.471+0000 7f15de1dc640 1 -- 172.21.15.22:0/169624901 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d815f880 msgr2=0x7f15d81f9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.471+0000 7f15de1dc640 1 --2- 172.21.15.22:0/169624901 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d815f880 0x7f15d81f9360 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f15d000c710 tx=0x7f15d000cbd0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.471+0000 7f15d7fff640 1 -- 172.21.15.22:0/169624901 reap_dead start 2024-03-29T04:20:22.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.471+0000 7f15de1dc640 1 -- 172.21.15.22:0/169624901 shutdown_connections 2024-03-29T04:20:22.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.471+0000 7f15de1dc640 1 -- 172.21.15.22:0/169624901 >> 172.21.15.22:0/169624901 conn(0x7f15d8071b80 msgr2=0x7f15d80b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:22.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.471+0000 7f15de1dc640 1 -- 172.21.15.22:0/169624901 shutdown_connections 2024-03-29T04:20:22.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.471+0000 7f15de1dc640 1 -- 172.21.15.22:0/169624901 wait complete. 2024-03-29T04:20:22.469 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-29T04:20:22.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.595+0000 7f7b22b66640 1 Processor -- start 2024-03-29T04:20:22.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.595+0000 7f7b22b66640 1 -- start start 2024-03-29T04:20:22.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b22b66640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7b1c0b41f0 0x7f7b1c0cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b22b66640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7b1c06ca50 0x7f7b1c0ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b22b66640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b1c0ccef0 con 0x7f7b1c06c670 2024-03-29T04:20:22.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b22b66640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f7b1c0b34c0 con 0x7f7b1c0b41f0 2024-03-29T04:20:22.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b22b66640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7b1c0b3710 con 0x7f7b1c06ca50 2024-03-29T04:20:22.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b1bfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7b1c0b41f0 0x7f7b1c0cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b1bfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7b1c0b41f0 0x7f7b1c0cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52270/0 (socket says 172.21.15.22:52270) 2024-03-29T04:20:22.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b1bfff640 1 -- 172.21.15.22:0/1061574325 learned_addr learned my addr 172.21.15.22:0/1061574325 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:22.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b21103640 1 --2- 172.21.15.22:0/1061574325 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7b1c06ca50 0x7f7b1c0ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b1bfff640 1 -- 172.21.15.22:0/1061574325 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7b1c06ca50 msgr2=0x7f7b1c0ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:22.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b1bfff640 1 --2- 172.21.15.22:0/1061574325 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7b1c06ca50 0x7f7b1c0ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b1bfff640 1 -- 172.21.15.22:0/1061574325 >> v1:172.21.15.151:6789/0 conn(0x7f7b1c06c670 legacy=0x7f7b1c06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:22.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b1bfff640 1 -- 172.21.15.22:0/1061574325 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1c0b39f0 con 0x7f7b1c0b41f0 2024-03-29T04:20:22.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b21103640 1 --2- 172.21.15.22:0/1061574325 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7b1c06ca50 0x7f7b1c0ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:22.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b1bfff640 1 --2- 172.21.15.22:0/1061574325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b1c0b41f0 0x7f7b1c0cc910 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009870 tx=0x7f7b1001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e5cad3c3397e7b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:22.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b1b7fe640 1 -- 172.21.15.22:0/1061574325 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b100ab5f0 con 0x7f7b1c0b41f0 2024-03-29T04:20:22.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b1b7fe640 1 -- 172.21.15.22:0/1061574325 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b100b0070 con 0x7f7b1c0b41f0 2024-03-29T04:20:22.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b1b7fe640 1 -- 172.21.15.22:0/1061574325 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b100ab9b0 con 0x7f7b1c0b41f0 2024-03-29T04:20:22.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b22b66640 1 -- 172.21.15.22:0/1061574325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b1c0b41f0 msgr2=0x7f7b1c0cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b22b66640 1 --2- 172.21.15.22:0/1061574325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b1c0b41f0 0x7f7b1c0cc910 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009870 tx=0x7f7b1001c520 comp rx=0 tx=0).stop 2024-03-29T04:20:22.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b22b66640 1 -- 172.21.15.22:0/1061574325 shutdown_connections 2024-03-29T04:20:22.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b22b66640 1 --2- 172.21.15.22:0/1061574325 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7b1c06ca50 0x7f7b1c0ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b22b66640 1 --2- 172.21.15.22:0/1061574325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b1c0b41f0 0x7f7b1c0cc910 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.599+0000 7f7b22b66640 1 -- 172.21.15.22:0/1061574325 >> 172.21.15.22:0/1061574325 conn(0x7f7b1c0cfee0 msgr2=0x7f7b1c0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:22.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b22b66640 1 -- 172.21.15.22:0/1061574325 shutdown_connections 2024-03-29T04:20:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b22b66640 1 -- 172.21.15.22:0/1061574325 wait complete. 2024-03-29T04:20:22.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b22b66640 1 Processor -- start 2024-03-29T04:20:22.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b22b66640 1 -- start start 2024-03-29T04:20:22.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b22b66640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7b1c06c670 0x7f7b1c15e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b22b66640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b1c06ca50 0x7f7b1c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b22b66640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b1c0b41f0 0x7f7b1c13a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b22b66640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c169070 con 0x7f7b1c0b41f0 2024-03-29T04:20:22.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b22b66640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c168ef0 con 0x7f7b1c06ca50 2024-03-29T04:20:22.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b22b66640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f7b1c1691f0 con 0x7f7b1c06c670 2024-03-29T04:20:22.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b21103640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b1c0b41f0 0x7f7b1c13a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b21103640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b1c0b41f0 0x7f7b1c13a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52282/0 (socket says 172.21.15.22:52282) 2024-03-29T04:20:22.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b21103640 1 -- 172.21.15.22:0/3310389175 learned_addr learned my addr 172.21.15.22:0/3310389175 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:22.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b1bfff640 1 --2- 172.21.15.22:0/3310389175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b1c06ca50 0x7f7b1c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b20902640 1 --2- 172.21.15.22:0/3310389175 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7b1c06c670 0x7f7b1c15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b21103640 1 -- 172.21.15.22:0/3310389175 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7b1c06c670 msgr2=0x7f7b1c15e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b21103640 1 --2- 172.21.15.22:0/3310389175 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7b1c06c670 0x7f7b1c15e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b21103640 1 -- 172.21.15.22:0/3310389175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b1c06ca50 msgr2=0x7f7b1c159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b21103640 1 --2- 172.21.15.22:0/3310389175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b1c06ca50 0x7f7b1c159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b21103640 1 -- 172.21.15.22:0/3310389175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1c15b710 con 0x7f7b1c0b41f0 2024-03-29T04:20:22.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b20902640 1 --2- 172.21.15.22:0/3310389175 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7b1c06c670 0x7f7b1c15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:22.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b21103640 1 --2- 172.21.15.22:0/3310389175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b1c0b41f0 0x7f7b1c13a600 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400cf20 tx=0x7f7b1400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:22.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b197fa640 1 -- 172.21.15.22:0/3310389175 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1401a070 con 0x7f7b1c0b41f0 2024-03-29T04:20:22.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b197fa640 1 -- 172.21.15.22:0/3310389175 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b1400d840 con 0x7f7b1c0b41f0 2024-03-29T04:20:22.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b22b66640 1 -- 172.21.15.22:0/3310389175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b1c15b9f0 con 0x7f7b1c0b41f0 2024-03-29T04:20:22.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b197fa640 1 -- 172.21.15.22:0/3310389175 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1400dc30 con 0x7f7b1c0b41f0 2024-03-29T04:20:22.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b22b66640 1 -- 172.21.15.22:0/3310389175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b1c140000 con 0x7f7b1c0b41f0 2024-03-29T04:20:22.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.603+0000 7f7b22b66640 1 -- 172.21.15.22:0/3310389175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b1c15bbc0 con 0x7f7b1c0b41f0 2024-03-29T04:20:22.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.607+0000 7f7b197fa640 1 -- 172.21.15.22:0/3310389175 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f7b1401d070 con 0x7f7b1c0b41f0 2024-03-29T04:20:22.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.607+0000 7f7b197fa640 1 --2- 172.21.15.22:0/3310389175 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7b08067fc0 0x7f7b0806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.607+0000 7f7b197fa640 1 -- 172.21.15.22:0/3310389175 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f7b14086c10 con 0x7f7b1c0b41f0 2024-03-29T04:20:22.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.607+0000 7f7b20902640 1 --2- 172.21.15.22:0/3310389175 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7b08067fc0 0x7f7b0806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.607+0000 7f7b20902640 1 --2- 172.21.15.22:0/3310389175 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7b08067fc0 0x7f7b0806a480 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c1f88d0 tx=0x7f7b0c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:22.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.611+0000 7f7b197fa640 1 -- 172.21.15.22:0/3310389175 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f7b1c15bbc0 con 0x7f7b1c0b41f0 2024-03-29T04:20:22.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.763+0000 7f7b22b66640 1 -- 172.21.15.22:0/3310389175 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f7b1c13fb10 con 0x7f7b08067fc0 2024-03-29T04:20:22.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.767+0000 7f7b197fa640 1 -- 172.21.15.22:0/3310389175 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7b1c13fb10 con 0x7f7b08067fc0 2024-03-29T04:20:22.747 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 4 to deep-scrub 2024-03-29T04:20:22.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.767+0000 7f7b22b66640 1 -- 172.21.15.22:0/3310389175 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7b08067fc0 msgr2=0x7f7b0806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.767+0000 7f7b22b66640 1 --2- 172.21.15.22:0/3310389175 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7b08067fc0 0x7f7b0806a480 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c1f88d0 tx=0x7f7b0c008040 comp rx=0 tx=0).stop 2024-03-29T04:20:22.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.767+0000 7f7b22b66640 1 -- 172.21.15.22:0/3310389175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b1c0b41f0 msgr2=0x7f7b1c13a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.767+0000 7f7b22b66640 1 --2- 172.21.15.22:0/3310389175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b1c0b41f0 0x7f7b1c13a600 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400cf20 tx=0x7f7b1400a7d0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.767+0000 7f7b22b66640 1 -- 172.21.15.22:0/3310389175 shutdown_connections 2024-03-29T04:20:22.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.767+0000 7f7b22b66640 1 --2- 172.21.15.22:0/3310389175 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f7b08067fc0 0x7f7b0806a480 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.767+0000 7f7b22b66640 1 --2- 172.21.15.22:0/3310389175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b1c0b41f0 0x7f7b1c13a600 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.767+0000 7f7b22b66640 1 --2- 172.21.15.22:0/3310389175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b1c06ca50 0x7f7b1c159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.767+0000 7f7b22b66640 1 --2- 172.21.15.22:0/3310389175 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f7b1c06c670 0x7f7b1c15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.767+0000 7f7b22b66640 1 -- 172.21.15.22:0/3310389175 >> 172.21.15.22:0/3310389175 conn(0x7f7b1c0cfee0 msgr2=0x7f7b1c143a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:22.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.767+0000 7f7b22b66640 1 -- 172.21.15.22:0/3310389175 shutdown_connections 2024-03-29T04:20:22.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.767+0000 7f7b22b66640 1 -- 172.21.15.22:0/3310389175 wait complete. 2024-03-29T04:20:22.765 INFO:tasks.ceph:Scrubbing osd.5 2024-03-29T04:20:22.766 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:20:22.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.911+0000 7fb3e4dff640 1 Processor -- start 2024-03-29T04:20:22.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.911+0000 7fb3e4dff640 1 -- start start 2024-03-29T04:20:22.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.911+0000 7fb3e4dff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb3e00b5930 0x7fb3e015c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.911+0000 7fb3e4dff640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3e015d090 con 0x7fb3e00b5550 2024-03-29T04:20:22.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.911+0000 7fb3e4dff640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3e015d370 con 0x7fb3e0073490 2024-03-29T04:20:22.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.911+0000 7fb3e4dff640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fb3e0160d20 con 0x7fb3e00b5930 2024-03-29T04:20:22.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.911+0000 7fb3ddd9b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb3e00b5930 0x7fb3e015c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.911+0000 7fb3de59c640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb3e00b5550 0x7fb3e0160610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:56230/0 (socket says 172.21.15.22:56230) 2024-03-29T04:20:22.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.911+0000 7fb3de59c640 1 -- 172.21.15.22:0/1403170480 learned_addr learned my addr 172.21.15.22:0/1403170480 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:22.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.911+0000 7fb3ddd9b640 1 -- 172.21.15.22:0/1403170480 >> v1:172.21.15.151:6789/0 conn(0x7fb3e0073490 legacy=0x7fb3e006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:22.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.911+0000 7fb3ddd9b640 1 -- 172.21.15.22:0/1403170480 >> v1:172.21.15.22:6789/0 conn(0x7fb3e00b5550 legacy=0x7fb3e0160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.911+0000 7fb3ddd9b640 1 -- 172.21.15.22:0/1403170480 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e015fe10 con 0x7fb3e00b5930 2024-03-29T04:20:22.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.915+0000 7fb3ddd9b640 1 --2- 172.21.15.22:0/1403170480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e00b5930 0x7fb3e015c720 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40097c0 tx=0x7fb3d401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f8db97fe5094da server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:22.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.915+0000 7fb3dd59a640 1 -- 172.21.15.22:0/1403170480 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d40b1070 con 0x7fb3e00b5930 2024-03-29T04:20:22.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.915+0000 7fb3dd59a640 1 -- 172.21.15.22:0/1403170480 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d40ac5f0 con 0x7fb3e00b5930 2024-03-29T04:20:22.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.915+0000 7fb3dd59a640 1 -- 172.21.15.22:0/1403170480 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d40ac9c0 con 0x7fb3e00b5930 2024-03-29T04:20:22.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.915+0000 7fb3e4dff640 1 -- 172.21.15.22:0/1403170480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e00b5930 msgr2=0x7fb3e015c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.915+0000 7fb3e4dff640 1 --2- 172.21.15.22:0/1403170480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e00b5930 0x7fb3e015c720 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40097c0 tx=0x7fb3d401c430 comp rx=0 tx=0).stop 2024-03-29T04:20:22.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.915+0000 7fb3e4dff640 1 -- 172.21.15.22:0/1403170480 shutdown_connections 2024-03-29T04:20:22.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.915+0000 7fb3e4dff640 1 --2- 172.21.15.22:0/1403170480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e00b5930 0x7fb3e015c720 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.915+0000 7fb3e4dff640 1 -- 172.21.15.22:0/1403170480 >> 172.21.15.22:0/1403170480 conn(0x7fb3e00b1b80 msgr2=0x7fb3e00b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:22.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.915+0000 7fb3e4dff640 1 -- 172.21.15.22:0/1403170480 shutdown_connections 2024-03-29T04:20:22.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.915+0000 7fb3e4dff640 1 -- 172.21.15.22:0/1403170480 wait complete. 2024-03-29T04:20:22.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.915+0000 7fb3e4dff640 1 Processor -- start 2024-03-29T04:20:22.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.915+0000 7fb3e4dff640 1 -- start start 2024-03-29T04:20:22.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3e4dff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3e0073490 0x7fb3e015a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3e4dff640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb3e00b5550 0x7fb3e013e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3e4dff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e00b5930 0x7fb3e0157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3e4dff640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e00cb010 con 0x7fb3e00b5930 2024-03-29T04:20:22.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3e4dff640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e00cae90 con 0x7fb3e0073490 2024-03-29T04:20:22.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3e4dff640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e006cd80 con 0x7fb3e00b5550 2024-03-29T04:20:22.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3ded9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e00b5930 0x7fb3e0157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3ded9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e00b5930 0x7fb3e0157860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52296/0 (socket says 172.21.15.22:52296) 2024-03-29T04:20:22.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3ded9d640 1 -- 172.21.15.22:0/790582063 learned_addr learned my addr 172.21.15.22:0/790582063 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:22.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3de59c640 1 --2- 172.21.15.22:0/790582063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3e0073490 0x7fb3e015a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3ddd9b640 1 --2- 172.21.15.22:0/790582063 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb3e00b5550 0x7fb3e013e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3ded9d640 1 -- 172.21.15.22:0/790582063 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb3e00b5550 msgr2=0x7fb3e013e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3ded9d640 1 --2- 172.21.15.22:0/790582063 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb3e00b5550 0x7fb3e013e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3ded9d640 1 -- 172.21.15.22:0/790582063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3e0073490 msgr2=0x7fb3e015a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3ded9d640 1 --2- 172.21.15.22:0/790582063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3e0073490 0x7fb3e015a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3ded9d640 1 -- 172.21.15.22:0/790582063 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e013edf0 con 0x7fb3e00b5930 2024-03-29T04:20:22.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3de59c640 1 --2- 172.21.15.22:0/790582063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3e0073490 0x7fb3e015a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:20:22.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3ded9d640 1 --2- 172.21.15.22:0/790582063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e00b5930 0x7fb3e0157860 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800c7b0 tx=0x7fb3d800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:22.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3ceffd640 1 -- 172.21.15.22:0/790582063 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8004b70 con 0x7fb3e00b5930 2024-03-29T04:20:22.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3ceffd640 1 -- 172.21.15.22:0/790582063 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d8004d10 con 0x7fb3e00b5930 2024-03-29T04:20:22.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3ceffd640 1 -- 172.21.15.22:0/790582063 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d800aa90 con 0x7fb3e00b5930 2024-03-29T04:20:22.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3e4dff640 1 -- 172.21.15.22:0/790582063 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e01567c0 con 0x7fb3e00b5930 2024-03-29T04:20:22.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3e4dff640 1 -- 172.21.15.22:0/790582063 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e0156d80 con 0x7fb3e00b5930 2024-03-29T04:20:22.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3e4dff640 1 -- 172.21.15.22:0/790582063 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb3e00687f0 con 0x7fb3e00b5930 2024-03-29T04:20:22.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.919+0000 7fb3ceffd640 1 -- 172.21.15.22:0/790582063 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb3d800ac30 con 0x7fb3e00b5930 2024-03-29T04:20:22.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.923+0000 7fb3ceffd640 1 --2- 172.21.15.22:0/790582063 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb3c0067ff0 0x7fb3c006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.923+0000 7fb3ceffd640 1 -- 172.21.15.22:0/790582063 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fb3d8086d20 con 0x7fb3e00b5930 2024-03-29T04:20:22.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.923+0000 7fb3ceffd640 1 --2- 172.21.15.22:0/790582063 >> [v2:172.21.15.151:6800/3763105575,v1:172.21.15.151:6802/3763105575] conn(0x7fb3c006e490 0x7fb3c00708d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:22.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.923+0000 7fb3ceffd640 1 -- 172.21.15.22:0/790582063 --> [v2:172.21.15.151:6800/3763105575,v1:172.21.15.151:6802/3763105575] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb3d800bf30 con 0x7fb3c006e490 2024-03-29T04:20:22.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.923+0000 7fb3ceffd640 1 -- 172.21.15.22:0/790582063 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fb3d8087180 con 0x7fb3e00b5930 2024-03-29T04:20:22.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.923+0000 7fb3de59c640 1 --2- 172.21.15.22:0/790582063 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb3c0067ff0 0x7fb3c006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.923+0000 7fb3ddd9b640 1 --2- 172.21.15.22:0/790582063 >> [v2:172.21.15.151:6800/3763105575,v1:172.21.15.151:6802/3763105575] conn(0x7fb3c006e490 0x7fb3c00708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:22.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.923+0000 7fb3de59c640 1 --2- 172.21.15.22:0/790582063 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb3c0067ff0 0x7fb3c006a4b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00b19d0 tx=0x7fb3d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:22.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.923+0000 7fb3ddd9b640 1 --2- 172.21.15.22:0/790582063 >> [v2:172.21.15.151:6800/3763105575,v1:172.21.15.151:6802/3763105575] conn(0x7fb3c006e490 0x7fb3c00708d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:22.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.923+0000 7fb3ceffd640 1 -- 172.21.15.22:0/790582063 <== osd.5 v2:172.21.15.151:6800/3763105575 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3d800bf30 con 0x7fb3c006e490 2024-03-29T04:20:22.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.939+0000 7fb3e4dff640 1 -- 172.21.15.22:0/790582063 --> [v2:172.21.15.151:6800/3763105575,v1:172.21.15.151:6802/3763105575] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb3e006eed0 con 0x7fb3c006e490 2024-03-29T04:20:22.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.939+0000 7fb3ceffd640 1 -- 172.21.15.22:0/790582063 <== osd.5 v2:172.21.15.151:6800/3763105575 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb3e006eed0 con 0x7fb3c006e490 2024-03-29T04:20:22.920 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-29T04:20:22.920 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:20:22.920 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-29T04:20:22.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.939+0000 7fb3e4dff640 1 -- 172.21.15.22:0/790582063 >> [v2:172.21.15.151:6800/3763105575,v1:172.21.15.151:6802/3763105575] conn(0x7fb3c006e490 msgr2=0x7fb3c00708d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.939+0000 7fb3e4dff640 1 --2- 172.21.15.22:0/790582063 >> [v2:172.21.15.151:6800/3763105575,v1:172.21.15.151:6802/3763105575] conn(0x7fb3c006e490 0x7fb3c00708d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:22.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.939+0000 7fb3e4dff640 1 -- 172.21.15.22:0/790582063 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb3c0067ff0 msgr2=0x7fb3c006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.939+0000 7fb3e4dff640 1 --2- 172.21.15.22:0/790582063 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb3c0067ff0 0x7fb3c006a4b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00b19d0 tx=0x7fb3d0009040 comp rx=0 tx=0).stop 2024-03-29T04:20:22.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.939+0000 7fb3e4dff640 1 -- 172.21.15.22:0/790582063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e00b5930 msgr2=0x7fb3e0157860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:22.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.939+0000 7fb3e4dff640 1 --2- 172.21.15.22:0/790582063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e00b5930 0x7fb3e0157860 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800c7b0 tx=0x7fb3d800cc70 comp rx=0 tx=0).stop 2024-03-29T04:20:22.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.939+0000 7fb3ded9d640 1 -- 172.21.15.22:0/790582063 reap_dead start 2024-03-29T04:20:22.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.939+0000 7fb3e4dff640 1 -- 172.21.15.22:0/790582063 shutdown_connections 2024-03-29T04:20:22.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.939+0000 7fb3e4dff640 1 -- 172.21.15.22:0/790582063 >> 172.21.15.22:0/790582063 conn(0x7fb3e00b1b80 msgr2=0x7fb3e0158950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:22.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.939+0000 7fb3e4dff640 1 -- 172.21.15.22:0/790582063 shutdown_connections 2024-03-29T04:20:22.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:22.939+0000 7fb3e4dff640 1 -- 172.21.15.22:0/790582063 wait complete. 2024-03-29T04:20:22.937 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-29T04:20:23.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.075+0000 7f576cf3a640 1 Processor -- start 2024-03-29T04:20:23.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.075+0000 7f576cf3a640 1 -- start start 2024-03-29T04:20:23.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.075+0000 7f576cf3a640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f576806c670 0x7f57680cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.075+0000 7f576cf3a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f57680b41f0 0x7f576806e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.075+0000 7f576cf3a640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f576806e5c0 con 0x7f576806ca50 2024-03-29T04:20:23.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.075+0000 7f576cf3a640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f57680b34c0 con 0x7f57680b41f0 2024-03-29T04:20:23.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.075+0000 7f576cf3a640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f57680ccef0 con 0x7f576806c670 2024-03-29T04:20:23.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.075+0000 7f5765d9b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f57680b41f0 0x7f576806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.075+0000 7f5765d9b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f57680b41f0 0x7f576806e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52300/0 (socket says 172.21.15.22:52300) 2024-03-29T04:20:23.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.075+0000 7f5765d9b640 1 -- 172.21.15.22:0/1985813350 learned_addr learned my addr 172.21.15.22:0/1985813350 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:23.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.075+0000 7f576659c640 1 --2- 172.21.15.22:0/1985813350 >> v2:172.21.15.151:3300/0 conn(0x7f576806c670 0x7f57680cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.075+0000 7f5765d9b640 1 -- 172.21.15.22:0/1985813350 >> v2:172.21.15.151:3300/0 conn(0x7f576806c670 msgr2=0x7f57680cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:23.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.075+0000 7f5765d9b640 1 --2- 172.21.15.22:0/1985813350 >> v2:172.21.15.151:3300/0 conn(0x7f576806c670 0x7f57680cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.075+0000 7f5765d9b640 1 -- 172.21.15.22:0/1985813350 >> v1:172.21.15.151:6789/0 conn(0x7f576806ca50 legacy=0x7f576815fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:23.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.075+0000 7f5765d9b640 1 -- 172.21.15.22:0/1985813350 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57680b3710 con 0x7f57680b41f0 2024-03-29T04:20:23.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f5765d9b640 1 --2- 172.21.15.22:0/1985813350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57680b41f0 0x7f576806e080 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f576806ea10 tx=0x7f575c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16a5db836b808f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:23.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f576559a640 1 -- 172.21.15.22:0/1985813350 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c0b0070 con 0x7f57680b41f0 2024-03-29T04:20:23.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f576559a640 1 -- 172.21.15.22:0/1985813350 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f575c0ab600 con 0x7f57680b41f0 2024-03-29T04:20:23.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f576559a640 1 -- 172.21.15.22:0/1985813350 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c0ab9d0 con 0x7f57680b41f0 2024-03-29T04:20:23.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f576cf3a640 1 -- 172.21.15.22:0/1985813350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57680b41f0 msgr2=0x7f576806e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f576cf3a640 1 --2- 172.21.15.22:0/1985813350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57680b41f0 0x7f576806e080 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f576806ea10 tx=0x7f575c01c4e0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f576cf3a640 1 -- 172.21.15.22:0/1985813350 shutdown_connections 2024-03-29T04:20:23.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f576cf3a640 1 --2- 172.21.15.22:0/1985813350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57680b41f0 0x7f576806e080 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f576cf3a640 1 --2- 172.21.15.22:0/1985813350 >> v2:172.21.15.151:3300/0 conn(0x7f576806c670 0x7f57680cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f576cf3a640 1 -- 172.21.15.22:0/1985813350 >> 172.21.15.22:0/1985813350 conn(0x7f57680cfee0 msgr2=0x7f57680d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:23.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f576cf3a640 1 -- 172.21.15.22:0/1985813350 shutdown_connections 2024-03-29T04:20:23.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f576cf3a640 1 -- 172.21.15.22:0/1985813350 wait complete. 2024-03-29T04:20:23.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f576cf3a640 1 Processor -- start 2024-03-29T04:20:23.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f576cf3a640 1 -- start start 2024-03-29T04:20:23.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f576cf3a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f576806c670 0x7f576815d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f576cf3a640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f576806ca50 0x7f57681396e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f576cf3a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57680b41f0 0x7f5768153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f576cf3a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f576806cfe0 con 0x7f57680b41f0 2024-03-29T04:20:23.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f576cf3a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f576806ce60 con 0x7f576806c670 2024-03-29T04:20:23.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f576cf3a640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f5768168ef0 con 0x7f576806ca50 2024-03-29T04:20:23.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f5766d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57680b41f0 0x7f5768153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f5766d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57680b41f0 0x7f5768153e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52306/0 (socket says 172.21.15.22:52306) 2024-03-29T04:20:23.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f5766d9d640 1 -- 172.21.15.22:0/642315939 learned_addr learned my addr 172.21.15.22:0/642315939 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:23.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f5766d9d640 1 -- 172.21.15.22:0/642315939 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f576806ca50 msgr2=0x7f57681396e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f5766d9d640 1 --2- 172.21.15.22:0/642315939 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f576806ca50 0x7f57681396e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f576659c640 1 --2- 172.21.15.22:0/642315939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f576806c670 0x7f576815d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f5766d9d640 1 -- 172.21.15.22:0/642315939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f576806c670 msgr2=0x7f576815d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f5766d9d640 1 --2- 172.21.15.22:0/642315939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f576806c670 0x7f576815d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f5766d9d640 1 -- 172.21.15.22:0/642315939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5768143c50 con 0x7f57680b41f0 2024-03-29T04:20:23.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f576659c640 1 --2- 172.21.15.22:0/642315939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f576806c670 0x7f576815d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:23.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.079+0000 7f5766d9d640 1 --2- 172.21.15.22:0/642315939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57680b41f0 0x7f5768153e10 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f576000c780 tx=0x7f576000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:23.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.083+0000 7f5756ffd640 1 -- 172.21.15.22:0/642315939 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760004b70 con 0x7f57680b41f0 2024-03-29T04:20:23.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.083+0000 7f5756ffd640 1 -- 172.21.15.22:0/642315939 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5760004d10 con 0x7f57680b41f0 2024-03-29T04:20:23.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.083+0000 7f5756ffd640 1 -- 172.21.15.22:0/642315939 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576000aa90 con 0x7f57680b41f0 2024-03-29T04:20:23.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.083+0000 7f576cf3a640 1 -- 172.21.15.22:0/642315939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576806f320 con 0x7f57680b41f0 2024-03-29T04:20:23.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.083+0000 7f576cf3a640 1 -- 172.21.15.22:0/642315939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57681521e0 con 0x7f57680b41f0 2024-03-29T04:20:23.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.083+0000 7f576cf3a640 1 -- 172.21.15.22:0/642315939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5768068890 con 0x7f57680b41f0 2024-03-29T04:20:23.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.083+0000 7f5756ffd640 1 -- 172.21.15.22:0/642315939 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f576000ac30 con 0x7f57680b41f0 2024-03-29T04:20:23.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.083+0000 7f5756ffd640 1 --2- 172.21.15.22:0/642315939 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5748067ff0 0x7f574806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.083+0000 7f5756ffd640 1 -- 172.21.15.22:0/642315939 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f5760086830 con 0x7f57680b41f0 2024-03-29T04:20:23.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.083+0000 7f576659c640 1 --2- 172.21.15.22:0/642315939 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5748067ff0 0x7f574806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.083+0000 7f576659c640 1 --2- 172.21.15.22:0/642315939 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5748067ff0 0x7f574806a4b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f57681512f0 tx=0x7f5758008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:23.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.087+0000 7f5756ffd640 1 -- 172.21.15.22:0/642315939 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f57600571e0 con 0x7f57680b41f0 2024-03-29T04:20:23.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.251+0000 7f576cf3a640 1 -- 172.21.15.22:0/642315939 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f5768067510 con 0x7f5748067ff0 2024-03-29T04:20:23.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.251+0000 7f5756ffd640 1 -- 172.21.15.22:0/642315939 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5768067510 con 0x7f5748067ff0 2024-03-29T04:20:23.234 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 5 to deep-scrub 2024-03-29T04:20:23.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.255+0000 7f576cf3a640 1 -- 172.21.15.22:0/642315939 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5748067ff0 msgr2=0x7f574806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.255+0000 7f576cf3a640 1 --2- 172.21.15.22:0/642315939 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5748067ff0 0x7f574806a4b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f57681512f0 tx=0x7f5758008040 comp rx=0 tx=0).stop 2024-03-29T04:20:23.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.255+0000 7f576cf3a640 1 -- 172.21.15.22:0/642315939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57680b41f0 msgr2=0x7f5768153e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.255+0000 7f576cf3a640 1 --2- 172.21.15.22:0/642315939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57680b41f0 0x7f5768153e10 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f576000c780 tx=0x7f576000cc40 comp rx=0 tx=0).stop 2024-03-29T04:20:23.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.255+0000 7f576cf3a640 1 -- 172.21.15.22:0/642315939 shutdown_connections 2024-03-29T04:20:23.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.255+0000 7f576cf3a640 1 --2- 172.21.15.22:0/642315939 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f5748067ff0 0x7f574806a4b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.255+0000 7f576cf3a640 1 --2- 172.21.15.22:0/642315939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57680b41f0 0x7f5768153e10 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.255+0000 7f576cf3a640 1 --2- 172.21.15.22:0/642315939 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f576806ca50 0x7f57681396e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.255+0000 7f576cf3a640 1 --2- 172.21.15.22:0/642315939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f576806c670 0x7f576815d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.255+0000 7f576cf3a640 1 -- 172.21.15.22:0/642315939 >> 172.21.15.22:0/642315939 conn(0x7f57680cfee0 msgr2=0x7f5768143540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:23.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.255+0000 7f576cf3a640 1 -- 172.21.15.22:0/642315939 shutdown_connections 2024-03-29T04:20:23.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.255+0000 7f576cf3a640 1 -- 172.21.15.22:0/642315939 wait complete. 2024-03-29T04:20:23.254 INFO:tasks.ceph:Scrubbing osd.6 2024-03-29T04:20:23.254 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:20:23.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.399+0000 7ff6e9729640 1 Processor -- start 2024-03-29T04:20:23.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.399+0000 7ff6e9729640 1 -- start start 2024-03-29T04:20:23.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e9729640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff6e415c120 0x7ff6e406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e9729640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6e406ccb0 0x7ff6e406e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e9729640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6e406d230 con 0x7ff6e406e970 2024-03-29T04:20:23.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e9729640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7ff6e406eda0 con 0x7ff6e415c120 2024-03-29T04:20:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e9729640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff6e406d510 con 0x7ff6e406ccb0 2024-03-29T04:20:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e2ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff6e415c120 0x7ff6e406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e2ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff6e415c120 0x7ff6e406c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52308/0 (socket says 172.21.15.22:52308) 2024-03-29T04:20:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e2ffd640 1 -- 172.21.15.22:0/2087283081 learned_addr learned my addr 172.21.15.22:0/2087283081 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e27fc640 1 --2- 172.21.15.22:0/2087283081 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6e406ccb0 0x7ff6e406e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e2ffd640 1 -- 172.21.15.22:0/2087283081 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6e406ccb0 msgr2=0x7ff6e406e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e2ffd640 1 --2- 172.21.15.22:0/2087283081 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6e406ccb0 0x7ff6e406e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e2ffd640 1 -- 172.21.15.22:0/2087283081 >> v1:172.21.15.151:6789/0 conn(0x7ff6e406e970 legacy=0x7ff6e415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e2ffd640 1 -- 172.21.15.22:0/2087283081 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e406d7f0 con 0x7ff6e415c120 2024-03-29T04:20:23.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e27fc640 1 --2- 172.21.15.22:0/2087283081 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6e406ccb0 0x7ff6e406e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:23.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e2ffd640 1 --2- 172.21.15.22:0/2087283081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6e415c120 0x7ff6e406c770 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009870 tx=0x7ff6d401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f827b98c8f7258cb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:23.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e1ffb640 1 -- 172.21.15.22:0/2087283081 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40a95f0 con 0x7ff6e415c120 2024-03-29T04:20:23.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e1ffb640 1 -- 172.21.15.22:0/2087283081 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6d40a9790 con 0x7ff6e415c120 2024-03-29T04:20:23.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e1ffb640 1 -- 172.21.15.22:0/2087283081 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40a9b60 con 0x7ff6e415c120 2024-03-29T04:20:23.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e9729640 1 -- 172.21.15.22:0/2087283081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6e415c120 msgr2=0x7ff6e406c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e9729640 1 --2- 172.21.15.22:0/2087283081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6e415c120 0x7ff6e406c770 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009870 tx=0x7ff6d401c530 comp rx=0 tx=0).stop 2024-03-29T04:20:23.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e9729640 1 -- 172.21.15.22:0/2087283081 shutdown_connections 2024-03-29T04:20:23.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e9729640 1 --2- 172.21.15.22:0/2087283081 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6e406ccb0 0x7ff6e406e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e9729640 1 --2- 172.21.15.22:0/2087283081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6e415c120 0x7ff6e406c770 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e9729640 1 -- 172.21.15.22:0/2087283081 >> 172.21.15.22:0/2087283081 conn(0x7ff6e4072540 msgr2=0x7ff6e4072950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:23.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e9729640 1 -- 172.21.15.22:0/2087283081 shutdown_connections 2024-03-29T04:20:23.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.403+0000 7ff6e9729640 1 -- 172.21.15.22:0/2087283081 wait complete. 2024-03-29T04:20:23.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e9729640 1 Processor -- start 2024-03-29T04:20:23.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e9729640 1 -- start start 2024-03-29T04:20:23.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e9729640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6e406ccb0 0x7ff6e41567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e9729640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6e406e970 0x7ff6e414ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e9729640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6e415c120 0x7ff6e4154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e9729640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e415c680 con 0x7ff6e415c120 2024-03-29T04:20:23.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e9729640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e415c500 con 0x7ff6e406ccb0 2024-03-29T04:20:23.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e9729640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7ff6e4167e30 con 0x7ff6e406e970 2024-03-29T04:20:23.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6e415c120 0x7ff6e4154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6e415c120 0x7ff6e4154110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52316/0 (socket says 172.21.15.22:52316) 2024-03-29T04:20:23.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e37fe640 1 -- 172.21.15.22:0/2953205381 learned_addr learned my addr 172.21.15.22:0/2953205381 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:23.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e2ffd640 1 --2- 172.21.15.22:0/2953205381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6e406ccb0 0x7ff6e41567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e27fc640 1 --2- 172.21.15.22:0/2953205381 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6e406e970 0x7ff6e414ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e37fe640 1 -- 172.21.15.22:0/2953205381 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6e406e970 msgr2=0x7ff6e414ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e37fe640 1 --2- 172.21.15.22:0/2953205381 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7ff6e406e970 0x7ff6e414ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e37fe640 1 -- 172.21.15.22:0/2953205381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6e406ccb0 msgr2=0x7ff6e41567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e37fe640 1 --2- 172.21.15.22:0/2953205381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6e406ccb0 0x7ff6e41567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e37fe640 1 -- 172.21.15.22:0/2953205381 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e414d360 con 0x7ff6e415c120 2024-03-29T04:20:23.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e2ffd640 1 --2- 172.21.15.22:0/2953205381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6e406ccb0 0x7ff6e41567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:20:23.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e37fe640 1 --2- 172.21.15.22:0/2953205381 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6e415c120 0x7ff6e4154110 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00c7a0 tx=0x7ff6dc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:23.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6cb7fe640 1 -- 172.21.15.22:0/2953205381 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc003b80 con 0x7ff6e415c120 2024-03-29T04:20:23.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6cb7fe640 1 -- 172.21.15.22:0/2953205381 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6dc003d20 con 0x7ff6e415c120 2024-03-29T04:20:23.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e9729640 1 -- 172.21.15.22:0/2953205381 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e4156e10 con 0x7ff6e415c120 2024-03-29T04:20:23.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6cb7fe640 1 -- 172.21.15.22:0/2953205381 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc00aa90 con 0x7ff6e415c120 2024-03-29T04:20:23.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e9729640 1 -- 172.21.15.22:0/2953205381 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e41573d0 con 0x7ff6e415c120 2024-03-29T04:20:23.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.407+0000 7ff6e9729640 1 -- 172.21.15.22:0/2953205381 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff6e40687f0 con 0x7ff6e415c120 2024-03-29T04:20:23.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.411+0000 7ff6cb7fe640 1 -- 172.21.15.22:0/2953205381 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff6dc00ac30 con 0x7ff6e415c120 2024-03-29T04:20:23.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.411+0000 7ff6cb7fe640 1 --2- 172.21.15.22:0/2953205381 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff6d0067fc0 0x7ff6d006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.411+0000 7ff6cb7fe640 1 -- 172.21.15.22:0/2953205381 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7ff6dc086ce0 con 0x7ff6e415c120 2024-03-29T04:20:23.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.411+0000 7ff6cb7fe640 1 --2- 172.21.15.22:0/2953205381 >> [v2:172.21.15.151:6824/2519745336,v1:172.21.15.151:6825/2519745336] conn(0x7ff6d006e460 0x7ff6d00708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.411+0000 7ff6cb7fe640 1 -- 172.21.15.22:0/2953205381 --> [v2:172.21.15.151:6824/2519745336,v1:172.21.15.151:6825/2519745336] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff6d0070f70 con 0x7ff6d006e460 2024-03-29T04:20:23.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.411+0000 7ff6cb7fe640 1 -- 172.21.15.22:0/2953205381 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff6dc087090 con 0x7ff6e415c120 2024-03-29T04:20:23.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.411+0000 7ff6e2ffd640 1 --2- 172.21.15.22:0/2953205381 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff6d0067fc0 0x7ff6d006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.411+0000 7ff6e27fc640 1 --2- 172.21.15.22:0/2953205381 >> [v2:172.21.15.151:6824/2519745336,v1:172.21.15.151:6825/2519745336] conn(0x7ff6d006e460 0x7ff6d00708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.411+0000 7ff6e2ffd640 1 --2- 172.21.15.22:0/2953205381 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff6d0067fc0 0x7ff6d006a480 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40ae000 tx=0x7ff6d4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:23.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.411+0000 7ff6e27fc640 1 --2- 172.21.15.22:0/2953205381 >> [v2:172.21.15.151:6824/2519745336,v1:172.21.15.151:6825/2519745336] conn(0x7ff6d006e460 0x7ff6d00708a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:23.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.411+0000 7ff6cb7fe640 1 -- 172.21.15.22:0/2953205381 <== osd.6 v2:172.21.15.151:6824/2519745336 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6d0070f70 con 0x7ff6d006e460 2024-03-29T04:20:23.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.427+0000 7ff6e9729640 1 -- 172.21.15.22:0/2953205381 --> [v2:172.21.15.151:6824/2519745336,v1:172.21.15.151:6825/2519745336] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff6e40b51b0 con 0x7ff6d006e460 2024-03-29T04:20:23.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.427+0000 7ff6cb7fe640 1 -- 172.21.15.22:0/2953205381 <== osd.6 v2:172.21.15.151:6824/2519745336 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff6e40b51b0 con 0x7ff6d006e460 2024-03-29T04:20:23.408 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-29T04:20:23.408 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:20:23.408 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-29T04:20:23.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.427+0000 7ff6e9729640 1 -- 172.21.15.22:0/2953205381 >> [v2:172.21.15.151:6824/2519745336,v1:172.21.15.151:6825/2519745336] conn(0x7ff6d006e460 msgr2=0x7ff6d00708a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.427+0000 7ff6e9729640 1 --2- 172.21.15.22:0/2953205381 >> [v2:172.21.15.151:6824/2519745336,v1:172.21.15.151:6825/2519745336] conn(0x7ff6d006e460 0x7ff6d00708a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.427+0000 7ff6e9729640 1 -- 172.21.15.22:0/2953205381 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff6d0067fc0 msgr2=0x7ff6d006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.427+0000 7ff6e9729640 1 --2- 172.21.15.22:0/2953205381 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7ff6d0067fc0 0x7ff6d006a480 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40ae000 tx=0x7ff6d4009040 comp rx=0 tx=0).stop 2024-03-29T04:20:23.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.427+0000 7ff6e9729640 1 -- 172.21.15.22:0/2953205381 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6e415c120 msgr2=0x7ff6e4154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.427+0000 7ff6e9729640 1 --2- 172.21.15.22:0/2953205381 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6e415c120 0x7ff6e4154110 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00c7a0 tx=0x7ff6dc00cc60 comp rx=0 tx=0).stop 2024-03-29T04:20:23.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.427+0000 7ff6e37fe640 1 -- 172.21.15.22:0/2953205381 reap_dead start 2024-03-29T04:20:23.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.427+0000 7ff6e9729640 1 -- 172.21.15.22:0/2953205381 shutdown_connections 2024-03-29T04:20:23.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.427+0000 7ff6e9729640 1 -- 172.21.15.22:0/2953205381 >> 172.21.15.22:0/2953205381 conn(0x7ff6e4072540 msgr2=0x7ff6e4072920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:23.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.427+0000 7ff6e9729640 1 -- 172.21.15.22:0/2953205381 shutdown_connections 2024-03-29T04:20:23.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.427+0000 7ff6e9729640 1 -- 172.21.15.22:0/2953205381 wait complete. 2024-03-29T04:20:23.425 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-29T04:20:23.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.559+0000 7f922ee9b640 1 Processor -- start 2024-03-29T04:20:23.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.559+0000 7f922ee9b640 1 -- start start 2024-03-29T04:20:23.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.559+0000 7f922ee9b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f92280b9c70 0x7f922815ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.559+0000 7f922ee9b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f922815f940 con 0x7f922815e620 2024-03-29T04:20:23.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.559+0000 7f922ee9b640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92280cea20 con 0x7f92280b4360 2024-03-29T04:20:23.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.559+0000 7f922ee9b640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f922815f0a0 con 0x7f92280b9c70 2024-03-29T04:20:23.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.559+0000 7f9227fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f92280b9c70 0x7f922815ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.559+0000 7f922cc37640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f922815e620 0x7f92280d2610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:56244/0 (socket says 172.21.15.22:56244) 2024-03-29T04:20:23.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.559+0000 7f922cc37640 1 -- 172.21.15.22:0/3621026233 learned_addr learned my addr 172.21.15.22:0/3621026233 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:23.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.559+0000 7f9227fff640 1 -- 172.21.15.22:0/3621026233 >> v1:172.21.15.151:6789/0 conn(0x7f92280b4360 legacy=0x7f92280d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:23.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.559+0000 7f9227fff640 1 -- 172.21.15.22:0/3621026233 >> v1:172.21.15.22:6789/0 conn(0x7f922815e620 legacy=0x7f92280d2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.559+0000 7f92277fe640 1 -- 172.21.15.22:0/3621026233 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2353380944 0 0) 0x7f922815f940 con 0x7f922815e620 2024-03-29T04:20:23.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.559+0000 7f9227fff640 1 -- 172.21.15.22:0/3621026233 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f922815f380 con 0x7f92280b9c70 2024-03-29T04:20:23.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.563+0000 7f9227fff640 1 --2- 172.21.15.22:0/3621026233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92280b9c70 0x7f922815ea10 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f921c0097f0 tx=0x7f921c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29ca064e75d22a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:23.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.563+0000 7f92277fe640 1 -- 172.21.15.22:0/3621026233 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c0b1070 con 0x7f92280b9c70 2024-03-29T04:20:23.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.563+0000 7f92277fe640 1 -- 172.21.15.22:0/3621026233 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f921c0a8050 con 0x7f92280b9c70 2024-03-29T04:20:23.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.563+0000 7f92277fe640 1 -- 172.21.15.22:0/3621026233 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c0ac860 con 0x7f92280b9c70 2024-03-29T04:20:23.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.563+0000 7f922ee9b640 1 -- 172.21.15.22:0/3621026233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92280b9c70 msgr2=0x7f922815ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.563+0000 7f922ee9b640 1 --2- 172.21.15.22:0/3621026233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92280b9c70 0x7f922815ea10 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f921c0097f0 tx=0x7f921c01c3e0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.563+0000 7f922ee9b640 1 -- 172.21.15.22:0/3621026233 shutdown_connections 2024-03-29T04:20:23.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.563+0000 7f922ee9b640 1 --2- 172.21.15.22:0/3621026233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92280b9c70 0x7f922815ea10 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.563+0000 7f922ee9b640 1 -- 172.21.15.22:0/3621026233 >> 172.21.15.22:0/3621026233 conn(0x7f92280b62a0 msgr2=0x7f92280b66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:23.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.563+0000 7f922ee9b640 1 -- 172.21.15.22:0/3621026233 shutdown_connections 2024-03-29T04:20:23.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.563+0000 7f922ee9b640 1 -- 172.21.15.22:0/3621026233 wait complete. 2024-03-29T04:20:23.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922ee9b640 1 Processor -- start 2024-03-29T04:20:23.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922ee9b640 1 -- start start 2024-03-29T04:20:23.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922ee9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92280b4360 0x7f92280c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922ee9b640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f92280b9c70 0x7f92281569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922ee9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9228157220 0x7f92280c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922ee9b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f92280d1af0 con 0x7f9228157220 2024-03-29T04:20:23.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922ee9b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f92280d1970 con 0x7f92280b4360 2024-03-29T04:20:23.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922ee9b640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f92280b4770 con 0x7f92280b9c70 2024-03-29T04:20:23.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922d438640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9228157220 0x7f92280c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922d438640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9228157220 0x7f92280c96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52326/0 (socket says 172.21.15.22:52326) 2024-03-29T04:20:23.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922d438640 1 -- 172.21.15.22:0/49463253 learned_addr learned my addr 172.21.15.22:0/49463253 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:23.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922cc37640 1 --2- 172.21.15.22:0/49463253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92280b4360 0x7f92280c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f9227fff640 1 --2- 172.21.15.22:0/49463253 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f92280b9c70 0x7f92281569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922d438640 1 -- 172.21.15.22:0/49463253 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f92280b9c70 msgr2=0x7f92281569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922d438640 1 --2- 172.21.15.22:0/49463253 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f92280b9c70 0x7f92281569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922d438640 1 -- 172.21.15.22:0/49463253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92280b4360 msgr2=0x7f92280c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922d438640 1 --2- 172.21.15.22:0/49463253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92280b4360 0x7f92280c7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922d438640 1 -- 172.21.15.22:0/49463253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f922815ddf0 con 0x7f9228157220 2024-03-29T04:20:23.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922cc37640 1 --2- 172.21.15.22:0/49463253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92280b4360 0x7f92280c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:20:23.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922d438640 1 --2- 172.21.15.22:0/49463253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9228157220 0x7f92280c96b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f922000c460 tx=0x7f922000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:23.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f92257fa640 1 -- 172.21.15.22:0/49463253 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9220016020 con 0x7f9228157220 2024-03-29T04:20:23.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f92257fa640 1 -- 172.21.15.22:0/49463253 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9220005220 con 0x7f9228157220 2024-03-29T04:20:23.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f92257fa640 1 -- 172.21.15.22:0/49463253 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9220007030 con 0x7f9228157220 2024-03-29T04:20:23.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922ee9b640 1 -- 172.21.15.22:0/49463253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92280b85d0 con 0x7f9228157220 2024-03-29T04:20:23.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922ee9b640 1 -- 172.21.15.22:0/49463253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92280b8890 con 0x7f9228157220 2024-03-29T04:20:23.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f922ee9b640 1 -- 172.21.15.22:0/49463253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9228068890 con 0x7f9228157220 2024-03-29T04:20:23.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f92257fa640 1 -- 172.21.15.22:0/49463253 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f92200071d0 con 0x7f9228157220 2024-03-29T04:20:23.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.567+0000 7f92257fa640 1 --2- 172.21.15.22:0/49463253 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9214067fc0 0x7f921406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.571+0000 7f92257fa640 1 -- 172.21.15.22:0/49463253 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f9220085c30 con 0x7f9228157220 2024-03-29T04:20:23.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.571+0000 7f922cc37640 1 --2- 172.21.15.22:0/49463253 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9214067fc0 0x7f921406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.571+0000 7f922cc37640 1 --2- 172.21.15.22:0/49463253 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9214067fc0 0x7f921406a480 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f92180012f0 tx=0x7f9218009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:23.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.575+0000 7f92257fa640 1 -- 172.21.15.22:0/49463253 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9220020020 con 0x7f9228157220 2024-03-29T04:20:23.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.731+0000 7f922ee9b640 1 -- 172.21.15.22:0/49463253 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f92280ce730 con 0x7f9214067fc0 2024-03-29T04:20:23.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.731+0000 7f92257fa640 1 -- 172.21.15.22:0/49463253 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f92280ce730 con 0x7f9214067fc0 2024-03-29T04:20:23.714 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 6 to deep-scrub 2024-03-29T04:20:23.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.735+0000 7f922ee9b640 1 -- 172.21.15.22:0/49463253 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9214067fc0 msgr2=0x7f921406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.735+0000 7f922ee9b640 1 --2- 172.21.15.22:0/49463253 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9214067fc0 0x7f921406a480 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f92180012f0 tx=0x7f9218009040 comp rx=0 tx=0).stop 2024-03-29T04:20:23.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.735+0000 7f922ee9b640 1 -- 172.21.15.22:0/49463253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9228157220 msgr2=0x7f92280c96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.735+0000 7f922ee9b640 1 --2- 172.21.15.22:0/49463253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9228157220 0x7f92280c96b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f922000c460 tx=0x7f922000c920 comp rx=0 tx=0).stop 2024-03-29T04:20:23.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.735+0000 7f922ee9b640 1 -- 172.21.15.22:0/49463253 shutdown_connections 2024-03-29T04:20:23.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.735+0000 7f922ee9b640 1 --2- 172.21.15.22:0/49463253 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9214067fc0 0x7f921406a480 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.735+0000 7f922ee9b640 1 --2- 172.21.15.22:0/49463253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9228157220 0x7f92280c96b0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.735+0000 7f922ee9b640 1 --2- 172.21.15.22:0/49463253 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f92280b9c70 0x7f92281569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.735+0000 7f922ee9b640 1 --2- 172.21.15.22:0/49463253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92280b4360 0x7f92280c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.735+0000 7f922ee9b640 1 -- 172.21.15.22:0/49463253 >> 172.21.15.22:0/49463253 conn(0x7f92280b62a0 msgr2=0x7f92280d2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:23.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.735+0000 7f922ee9b640 1 -- 172.21.15.22:0/49463253 shutdown_connections 2024-03-29T04:20:23.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.735+0000 7f922ee9b640 1 -- 172.21.15.22:0/49463253 wait complete. 2024-03-29T04:20:23.732 INFO:tasks.ceph:Scrubbing osd.7 2024-03-29T04:20:23.732 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:20:23.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f2387c58640 1 Processor -- start 2024-03-29T04:20:23.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f2387c58640 1 -- start start 2024-03-29T04:20:23.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f2387c58640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f23800b5930 0x7f238015c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f2387c58640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f238015d090 con 0x7f23800b5550 2024-03-29T04:20:23.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f2387c58640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f238015d370 con 0x7f2380073490 2024-03-29T04:20:23.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f2387c58640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f2380160d20 con 0x7f23800b5930 2024-03-29T04:20:23.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f23859f4640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f23800b5550 0x7f2380160610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:56252/0 (socket says 172.21.15.22:56252) 2024-03-29T04:20:23.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f23859f4640 1 -- 172.21.15.22:0/2265704652 learned_addr learned my addr 172.21.15.22:0/2265704652 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:23.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f23849f2640 1 -- 172.21.15.22:0/2265704652 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1588060777 0 0) 0x7f238015d090 con 0x7f23800b5550 2024-03-29T04:20:23.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f23851f3640 1 --2- 172.21.15.22:0/2265704652 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f23800b5930 0x7f238015c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f23849f2640 1 -- 172.21.15.22:0/2265704652 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f236c003530 con 0x7f23800b5550 2024-03-29T04:20:23.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f23849f2640 1 -- 172.21.15.22:0/2265704652 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3424272496 0 0) 0x7f236c003530 con 0x7f23800b5550 2024-03-29T04:20:23.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f23849f2640 1 -- 172.21.15.22:0/2265704652 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f23800b5930 msgr2=0x7f238015c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:23.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f23849f2640 1 --2- 172.21.15.22:0/2265704652 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f23800b5930 0x7f238015c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f23849f2640 1 -- 172.21.15.22:0/2265704652 >> v1:172.21.15.151:6789/0 conn(0x7f2380073490 legacy=0x7f238006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f23849f2640 1 -- 172.21.15.22:0/2265704652 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238015fe10 con 0x7f23800b5550 2024-03-29T04:20:23.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f23849f2640 1 -- 172.21.15.22:0/2265704652 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f237c001680 con 0x7f23800b5550 2024-03-29T04:20:23.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f23849f2640 1 -- 172.21.15.22:0/2265704652 >> v1:172.21.15.22:6789/0 conn(0x7f23800b5550 legacy=0x7f2380160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f23849f2640 1 --2- 172.21.15.22:0/2265704652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f236c0039f0 0x7f236c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f23849f2640 1 --2- 172.21.15.22:0/2265704652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f236c004810 0x7f236c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f23849f2640 1 --2- 172.21.15.22:0/2265704652 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f23800b5930 0x7f236c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f23849f2640 1 -- 172.21.15.22:0/2265704652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2380160d20 con 0x7f236c0039f0 2024-03-29T04:20:23.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f23849f2640 1 -- 172.21.15.22:0/2265704652 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f236c00ba80 con 0x7f236c004810 2024-03-29T04:20:23.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f23849f2640 1 -- 172.21.15.22:0/2265704652 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f236c00bc80 con 0x7f23800b5930 2024-03-29T04:20:23.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.867+0000 7f23859f4640 1 --2- 172.21.15.22:0/2265704652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f236c0039f0 0x7f236c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f23851f3640 1 --2- 172.21.15.22:0/2265704652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f236c004810 0x7f236c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f23859f4640 1 -- 172.21.15.22:0/2265704652 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f23800b5930 msgr2=0x7f236c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:23.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f23861f5640 1 --2- 172.21.15.22:0/2265704652 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f23800b5930 0x7f236c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f23859f4640 1 --2- 172.21.15.22:0/2265704652 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f23800b5930 0x7f236c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f23859f4640 1 -- 172.21.15.22:0/2265704652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f236c004810 msgr2=0x7f236c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:23.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f23859f4640 1 --2- 172.21.15.22:0/2265704652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f236c004810 0x7f236c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f23859f4640 1 -- 172.21.15.22:0/2265704652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236c003530 con 0x7f236c0039f0 2024-03-29T04:20:23.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f23851f3640 1 --2- 172.21.15.22:0/2265704652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f236c004810 0x7f236c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:23.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f23861f5640 1 --2- 172.21.15.22:0/2265704652 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f23800b5930 0x7f236c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:23.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f23859f4640 1 --2- 172.21.15.22:0/2265704652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f236c0039f0 0x7f236c003e10 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f237c00c190 tx=0x7f237c01cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bab158c6ba954d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:23.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f23849f2640 1 -- 172.21.15.22:0/2265704652 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c007d00 con 0x7f236c0039f0 2024-03-29T04:20:23.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f23849f2640 1 -- 172.21.15.22:0/2265704652 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f237c0a9020 con 0x7f236c0039f0 2024-03-29T04:20:23.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f23849f2640 1 -- 172.21.15.22:0/2265704652 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c005ae0 con 0x7f236c0039f0 2024-03-29T04:20:23.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f2387c58640 1 -- 172.21.15.22:0/2265704652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f236c0039f0 msgr2=0x7f236c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f2387c58640 1 --2- 172.21.15.22:0/2265704652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f236c0039f0 0x7f236c003e10 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f237c00c190 tx=0x7f237c01cbf0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f23861f5640 1 -- 172.21.15.22:0/2265704652 reap_dead start 2024-03-29T04:20:23.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f2387c58640 1 -- 172.21.15.22:0/2265704652 shutdown_connections 2024-03-29T04:20:23.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f2387c58640 1 -- 172.21.15.22:0/2265704652 >> 172.21.15.22:0/2265704652 conn(0x7f23800b1b80 msgr2=0x7f23800b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:23.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f2387c58640 1 -- 172.21.15.22:0/2265704652 shutdown_connections 2024-03-29T04:20:23.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f2387c58640 1 -- 172.21.15.22:0/2265704652 wait complete. 2024-03-29T04:20:23.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f2387c58640 1 Processor -- start 2024-03-29T04:20:23.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.871+0000 7f2387c58640 1 -- start start 2024-03-29T04:20:23.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f2387c58640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f236c0039f0 0x7f2380155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f2387c58640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2380156a50 0x7f23800b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f2387c58640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2380156470 0x7f23800b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f2387c58640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f23800ca680 con 0x7f2380156470 2024-03-29T04:20:23.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f2387c58640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f238006cf90 con 0x7f2380156a50 2024-03-29T04:20:23.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f2387c58640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f23800ca800 con 0x7f236c0039f0 2024-03-29T04:20:23.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f23861f5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2380156470 0x7f23800b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f23861f5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2380156470 0x7f23800b4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52342/0 (socket says 172.21.15.22:52342) 2024-03-29T04:20:23.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f23861f5640 1 -- 172.21.15.22:0/2757387007 learned_addr learned my addr 172.21.15.22:0/2757387007 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:23.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f23851f3640 1 --2- 172.21.15.22:0/2757387007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2380156a50 0x7f23800b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f23859f4640 1 --2- 172.21.15.22:0/2757387007 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f236c0039f0 0x7f2380155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f23861f5640 1 -- 172.21.15.22:0/2757387007 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f236c0039f0 msgr2=0x7f2380155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f23861f5640 1 --2- 172.21.15.22:0/2757387007 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f236c0039f0 0x7f2380155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f23861f5640 1 -- 172.21.15.22:0/2757387007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2380156a50 msgr2=0x7f23800b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f23861f5640 1 --2- 172.21.15.22:0/2757387007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2380156a50 0x7f23800b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f23861f5640 1 -- 172.21.15.22:0/2757387007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23800c5260 con 0x7f2380156470 2024-03-29T04:20:23.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f23861f5640 1 --2- 172.21.15.22:0/2757387007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2380156470 0x7f23800b4060 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f23800ca420 tx=0x7f237800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:23.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f23767fc640 1 -- 172.21.15.22:0/2757387007 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378006d60 con 0x7f2380156470 2024-03-29T04:20:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f23767fc640 1 -- 172.21.15.22:0/2757387007 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2378003980 con 0x7f2380156470 2024-03-29T04:20:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f23767fc640 1 -- 172.21.15.22:0/2757387007 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378003db0 con 0x7f2380156470 2024-03-29T04:20:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f2387c58640 1 -- 172.21.15.22:0/2757387007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23800c5700 con 0x7f2380156470 2024-03-29T04:20:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f2387c58640 1 -- 172.21.15.22:0/2757387007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23800b3600 con 0x7f2380156470 2024-03-29T04:20:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f2387c58640 1 -- 172.21.15.22:0/2757387007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2380156250 con 0x7f2380156470 2024-03-29T04:20:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f23767fc640 1 -- 172.21.15.22:0/2757387007 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f237801b020 con 0x7f2380156470 2024-03-29T04:20:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f23767fc640 1 --2- 172.21.15.22:0/2757387007 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2364067fc0 0x7f236406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f23767fc640 1 -- 172.21.15.22:0/2757387007 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f23780865c0 con 0x7f2380156470 2024-03-29T04:20:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f23767fc640 1 --2- 172.21.15.22:0/2757387007 >> [v2:172.21.15.151:6801/2961229122,v1:172.21.15.151:6803/2961229122] conn(0x7f236406e460 0x7f23640708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:23.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f23767fc640 1 -- 172.21.15.22:0/2757387007 --> [v2:172.21.15.151:6801/2961229122,v1:172.21.15.151:6803/2961229122] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2378007a20 con 0x7f236406e460 2024-03-29T04:20:23.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f23767fc640 1 -- 172.21.15.22:0/2757387007 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f2378086a50 con 0x7f2380156470 2024-03-29T04:20:23.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.875+0000 7f23859f4640 1 --2- 172.21.15.22:0/2757387007 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2364067fc0 0x7f236406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.879+0000 7f23851f3640 1 --2- 172.21.15.22:0/2757387007 >> [v2:172.21.15.151:6801/2961229122,v1:172.21.15.151:6803/2961229122] conn(0x7f236406e460 0x7f23640708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:23.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.879+0000 7f23859f4640 1 --2- 172.21.15.22:0/2757387007 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2364067fc0 0x7f236406a480 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f237c0aa000 tx=0x7f238015d260 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:23.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.879+0000 7f23851f3640 1 --2- 172.21.15.22:0/2757387007 >> [v2:172.21.15.151:6801/2961229122,v1:172.21.15.151:6803/2961229122] conn(0x7f236406e460 0x7f23640708a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:23.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.879+0000 7f23767fc640 1 -- 172.21.15.22:0/2757387007 <== osd.7 v2:172.21.15.151:6801/2961229122 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2378007a20 con 0x7f236406e460 2024-03-29T04:20:23.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.891+0000 7f2387c58640 1 -- 172.21.15.22:0/2757387007 --> [v2:172.21.15.151:6801/2961229122,v1:172.21.15.151:6803/2961229122] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f23800687f0 con 0x7f236406e460 2024-03-29T04:20:23.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.895+0000 7f23767fc640 1 -- 172.21.15.22:0/2757387007 <== osd.7 v2:172.21.15.151:6801/2961229122 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f23800687f0 con 0x7f236406e460 2024-03-29T04:20:23.875 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-29T04:20:23.875 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:20:23.875 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-29T04:20:23.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.895+0000 7f2387c58640 1 -- 172.21.15.22:0/2757387007 >> [v2:172.21.15.151:6801/2961229122,v1:172.21.15.151:6803/2961229122] conn(0x7f236406e460 msgr2=0x7f23640708a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.895+0000 7f2387c58640 1 --2- 172.21.15.22:0/2757387007 >> [v2:172.21.15.151:6801/2961229122,v1:172.21.15.151:6803/2961229122] conn(0x7f236406e460 0x7f23640708a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:23.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.895+0000 7f2387c58640 1 -- 172.21.15.22:0/2757387007 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2364067fc0 msgr2=0x7f236406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.895+0000 7f2387c58640 1 --2- 172.21.15.22:0/2757387007 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f2364067fc0 0x7f236406a480 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f237c0aa000 tx=0x7f238015d260 comp rx=0 tx=0).stop 2024-03-29T04:20:23.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.895+0000 7f2387c58640 1 -- 172.21.15.22:0/2757387007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2380156470 msgr2=0x7f23800b4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:23.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.895+0000 7f2387c58640 1 --2- 172.21.15.22:0/2757387007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2380156470 0x7f23800b4060 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f23800ca420 tx=0x7f237800dc00 comp rx=0 tx=0).stop 2024-03-29T04:20:23.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.895+0000 7f23861f5640 1 -- 172.21.15.22:0/2757387007 reap_dead start 2024-03-29T04:20:23.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.895+0000 7f2387c58640 1 -- 172.21.15.22:0/2757387007 shutdown_connections 2024-03-29T04:20:23.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.895+0000 7f2387c58640 1 -- 172.21.15.22:0/2757387007 >> 172.21.15.22:0/2757387007 conn(0x7f23800b1b80 msgr2=0x7f23800b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:23.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.895+0000 7f2387c58640 1 -- 172.21.15.22:0/2757387007 shutdown_connections 2024-03-29T04:20:23.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:23.895+0000 7f2387c58640 1 -- 172.21.15.22:0/2757387007 wait complete. 2024-03-29T04:20:23.890 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-29T04:20:24.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f327b8fd640 1 Processor -- start 2024-03-29T04:20:24.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f327b8fd640 1 -- start start 2024-03-29T04:20:24.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f327b8fd640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f32740b4a70 0x7f327415ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:24.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f327b8fd640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f327415f940 con 0x7f32740b9c70 2024-03-29T04:20:24.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f327b8fd640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32740cea20 con 0x7f327415e620 2024-03-29T04:20:24.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f327b8fd640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f327415f0a0 con 0x7f32740b4a70 2024-03-29T04:20:24.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f3278e98640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f32740b9c70 0x7f32740b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:56254/0 (socket says 172.21.15.22:56254) 2024-03-29T04:20:24.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f3278e98640 1 -- 172.21.15.22:0/2620660379 learned_addr learned my addr 172.21.15.22:0/2620660379 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:24.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f3279e9a640 1 --2- 172.21.15.22:0/2620660379 >> v2:172.21.15.151:3300/0 conn(0x7f32740b4a70 0x7f327415ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:24.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f326bfff640 1 -- 172.21.15.22:0/2620660379 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3704028337 0 0) 0x7f327415f940 con 0x7f32740b9c70 2024-03-29T04:20:24.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f326bfff640 1 -- 172.21.15.22:0/2620660379 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3260003530 con 0x7f32740b9c70 2024-03-29T04:20:24.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f326bfff640 1 -- 172.21.15.22:0/2620660379 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1660950376 0 0) 0x7f3260003530 con 0x7f32740b9c70 2024-03-29T04:20:24.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f326bfff640 1 -- 172.21.15.22:0/2620660379 >> v2:172.21.15.151:3300/0 conn(0x7f32740b4a70 msgr2=0x7f327415ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:24.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f326bfff640 1 --2- 172.21.15.22:0/2620660379 >> v2:172.21.15.151:3300/0 conn(0x7f32740b4a70 0x7f327415ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f326bfff640 1 -- 172.21.15.22:0/2620660379 >> v1:172.21.15.151:6789/0 conn(0x7f327415e620 legacy=0x7f32740d2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:24.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f326bfff640 1 -- 172.21.15.22:0/2620660379 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327415f380 con 0x7f32740b9c70 2024-03-29T04:20:24.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f326bfff640 1 -- 172.21.15.22:0/2620660379 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f3264001680 con 0x7f32740b9c70 2024-03-29T04:20:24.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f326bfff640 1 -- 172.21.15.22:0/2620660379 >> v1:172.21.15.22:6789/0 conn(0x7f32740b9c70 legacy=0x7f32740b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:24.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f326bfff640 1 --2- 172.21.15.22:0/2620660379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3260003a10 0x7f3260003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:24.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f326bfff640 1 --2- 172.21.15.22:0/2620660379 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3260004810 0x7f3260004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:24.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f326bfff640 1 --2- 172.21.15.22:0/2620660379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3260005250 0x7f326000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:24.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f326bfff640 1 -- 172.21.15.22:0/2620660379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f327415f0a0 con 0x7f3260003a10 2024-03-29T04:20:24.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f326bfff640 1 -- 172.21.15.22:0/2620660379 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f326000c1a0 con 0x7f3260005250 2024-03-29T04:20:24.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f326bfff640 1 -- 172.21.15.22:0/2620660379 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f326000c3a0 con 0x7f3260004810 2024-03-29T04:20:24.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f3279699640 1 --2- 172.21.15.22:0/2620660379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3260003a10 0x7f3260003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:24.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f3279699640 1 -- 172.21.15.22:0/2620660379 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3260004810 msgr2=0x7f3260004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:24.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f3279699640 1 --2- 172.21.15.22:0/2620660379 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f3260004810 0x7f3260004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f3279699640 1 -- 172.21.15.22:0/2620660379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3260005250 msgr2=0x7f326000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:24.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f3279699640 1 --2- 172.21.15.22:0/2620660379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3260005250 0x7f326000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f3279e9a640 1 -- 172.21.15.22:0/2620660379 reap_dead start 2024-03-29T04:20:24.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.019+0000 7f3279699640 1 -- 172.21.15.22:0/2620660379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3260003530 con 0x7f3260003a10 2024-03-29T04:20:24.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.023+0000 7f3279699640 1 --2- 172.21.15.22:0/2620660379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3260003a10 0x7f3260003e30 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3260004790 tx=0x7f327001c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3274c08e569b78 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:24.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.023+0000 7f326bfff640 1 -- 172.21.15.22:0/2620660379 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32700013a0 con 0x7f3260003a10 2024-03-29T04:20:24.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.023+0000 7f326bfff640 1 -- 172.21.15.22:0/2620660379 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32700052f0 con 0x7f3260003a10 2024-03-29T04:20:24.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.023+0000 7f326bfff640 1 -- 172.21.15.22:0/2620660379 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270003840 con 0x7f3260003a10 2024-03-29T04:20:24.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.023+0000 7f327b8fd640 1 -- 172.21.15.22:0/2620660379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3260003a10 msgr2=0x7f3260003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:24.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.023+0000 7f327b8fd640 1 --2- 172.21.15.22:0/2620660379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3260003a10 0x7f3260003e30 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3260004790 tx=0x7f327001c370 comp rx=0 tx=0).stop 2024-03-29T04:20:24.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.023+0000 7f327b8fd640 1 -- 172.21.15.22:0/2620660379 shutdown_connections 2024-03-29T04:20:24.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.023+0000 7f327b8fd640 1 --2- 172.21.15.22:0/2620660379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3260003a10 0x7f3260003e30 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.023+0000 7f327b8fd640 1 -- 172.21.15.22:0/2620660379 >> 172.21.15.22:0/2620660379 conn(0x7f32740b62a0 msgr2=0x7f32740b66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:24.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.023+0000 7f327b8fd640 1 -- 172.21.15.22:0/2620660379 shutdown_connections 2024-03-29T04:20:24.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.023+0000 7f327b8fd640 1 -- 172.21.15.22:0/2620660379 wait complete. 2024-03-29T04:20:24.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.023+0000 7f327b8fd640 1 Processor -- start 2024-03-29T04:20:24.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.023+0000 7f327b8fd640 1 -- start start 2024-03-29T04:20:24.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f327b8fd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3260003a10 0x7f32740cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:24.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f327b8fd640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f32740cb4a0 0x7f32740cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:24.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f327b8fd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32740b85e0 0x7f32740b8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:24.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f327b8fd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f32740b4770 con 0x7f32740b85e0 2024-03-29T04:20:24.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f327b8fd640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f327415ee20 con 0x7f3260003a10 2024-03-29T04:20:24.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f327b8fd640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f32740b48f0 con 0x7f32740cb4a0 2024-03-29T04:20:24.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f3279e9a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32740b85e0 0x7f32740b8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:24.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f3279e9a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32740b85e0 0x7f32740b8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52348/0 (socket says 172.21.15.22:52348) 2024-03-29T04:20:24.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f3279e9a640 1 -- 172.21.15.22:0/517702431 learned_addr learned my addr 172.21.15.22:0/517702431 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:24.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f3279699640 1 --2- 172.21.15.22:0/517702431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3260003a10 0x7f32740cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:24.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f3278e98640 1 --2- 172.21.15.22:0/517702431 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f32740cb4a0 0x7f32740cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:24.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f3279e9a640 1 -- 172.21.15.22:0/517702431 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f32740cb4a0 msgr2=0x7f32740cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:24.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f3279e9a640 1 --2- 172.21.15.22:0/517702431 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f32740cb4a0 0x7f32740cb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f3279e9a640 1 -- 172.21.15.22:0/517702431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3260003a10 msgr2=0x7f32740cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:24.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f3279e9a640 1 --2- 172.21.15.22:0/517702431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3260003a10 0x7f32740cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f3279e9a640 1 -- 172.21.15.22:0/517702431 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32740b7560 con 0x7f32740b85e0 2024-03-29T04:20:24.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f3279e9a640 1 --2- 172.21.15.22:0/517702431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32740b85e0 0x7f32740b8a40 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f32740b5bc0 tx=0x7f326c00ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:24.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f3269ffb640 1 -- 172.21.15.22:0/517702431 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c00bb70 con 0x7f32740b85e0 2024-03-29T04:20:24.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f3269ffb640 1 -- 172.21.15.22:0/517702431 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f326c01a070 con 0x7f32740b85e0 2024-03-29T04:20:24.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f327b8fd640 1 -- 172.21.15.22:0/517702431 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32740b7820 con 0x7f32740b85e0 2024-03-29T04:20:24.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f3269ffb640 1 -- 172.21.15.22:0/517702431 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c014400 con 0x7f32740b85e0 2024-03-29T04:20:24.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f327b8fd640 1 -- 172.21.15.22:0/517702431 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32740b7de0 con 0x7f32740b85e0 2024-03-29T04:20:24.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f327b8fd640 1 -- 172.21.15.22:0/517702431 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3274068890 con 0x7f32740b85e0 2024-03-29T04:20:24.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f3269ffb640 1 -- 172.21.15.22:0/517702431 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f326c013070 con 0x7f32740b85e0 2024-03-29T04:20:24.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f3269ffb640 1 --2- 172.21.15.22:0/517702431 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3258067fc0 0x7f325806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:24.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f3269ffb640 1 -- 172.21.15.22:0/517702431 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f326c0865c0 con 0x7f32740b85e0 2024-03-29T04:20:24.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.027+0000 7f3279699640 1 --2- 172.21.15.22:0/517702431 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3258067fc0 0x7f325806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:24.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.031+0000 7f3279699640 1 --2- 172.21.15.22:0/517702431 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3258067fc0 0x7f325806a480 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f32600057f0 tx=0x7f32700011a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:24.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.031+0000 7f3269ffb640 1 -- 172.21.15.22:0/517702431 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f326c024020 con 0x7f32740b85e0 2024-03-29T04:20:24.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.199+0000 7f327b8fd640 1 -- 172.21.15.22:0/517702431 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f327415f940 con 0x7f3258067fc0 2024-03-29T04:20:24.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.199+0000 7f3269ffb640 1 -- 172.21.15.22:0/517702431 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f327415f940 con 0x7f3258067fc0 2024-03-29T04:20:24.180 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 7 to deep-scrub 2024-03-29T04:20:24.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.199+0000 7f327b8fd640 1 -- 172.21.15.22:0/517702431 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3258067fc0 msgr2=0x7f325806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:24.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.199+0000 7f327b8fd640 1 --2- 172.21.15.22:0/517702431 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3258067fc0 0x7f325806a480 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f32600057f0 tx=0x7f32700011a0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.199+0000 7f327b8fd640 1 -- 172.21.15.22:0/517702431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32740b85e0 msgr2=0x7f32740b8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:24.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.199+0000 7f327b8fd640 1 --2- 172.21.15.22:0/517702431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32740b85e0 0x7f32740b8a40 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f32740b5bc0 tx=0x7f326c00ef50 comp rx=0 tx=0).stop 2024-03-29T04:20:24.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.199+0000 7f327b8fd640 1 -- 172.21.15.22:0/517702431 shutdown_connections 2024-03-29T04:20:24.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.199+0000 7f327b8fd640 1 --2- 172.21.15.22:0/517702431 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f3258067fc0 0x7f325806a480 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.199+0000 7f327b8fd640 1 --2- 172.21.15.22:0/517702431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32740b85e0 0x7f32740b8a40 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.199+0000 7f327b8fd640 1 --2- 172.21.15.22:0/517702431 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f32740cb4a0 0x7f32740cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.199+0000 7f327b8fd640 1 --2- 172.21.15.22:0/517702431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3260003a10 0x7f32740cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.199+0000 7f327b8fd640 1 -- 172.21.15.22:0/517702431 >> 172.21.15.22:0/517702431 conn(0x7f32740b62a0 msgr2=0x7f32741577c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:24.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.199+0000 7f327b8fd640 1 -- 172.21.15.22:0/517702431 shutdown_connections 2024-03-29T04:20:24.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.199+0000 7f327b8fd640 1 -- 172.21.15.22:0/517702431 wait complete. 2024-03-29T04:20:24.200 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:20:24.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.331+0000 7f166169e640 1 Processor -- start 2024-03-29T04:20:24.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.331+0000 7f166169e640 1 -- start start 2024-03-29T04:20:24.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.335+0000 7f166169e640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f165c06c670 0x7f165c0cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:24.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.335+0000 7f166169e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f165c0b41f0 0x7f165c06e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:24.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.335+0000 7f166169e640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f165c06ca50 0x7f165c15fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:24.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.335+0000 7f166169e640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f165c0b34c0 con 0x7f165c0b41f0 2024-03-29T04:20:24.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.335+0000 7f166169e640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f165c06e5c0 con 0x7f165c06ca50 2024-03-29T04:20:24.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.335+0000 7f166169e640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f165c0ccef0 con 0x7f165c06c670 2024-03-29T04:20:24.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.335+0000 7f165a7fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f165c0b41f0 0x7f165c06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:24.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.335+0000 7f165a7fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f165c0b41f0 0x7f165c06e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52364/0 (socket says 172.21.15.22:52364) 2024-03-29T04:20:24.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.335+0000 7f165a7fc640 1 -- 172.21.15.22:0/3655150066 learned_addr learned my addr 172.21.15.22:0/3655150066 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:24.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.335+0000 7f165b7fe640 1 --2- 172.21.15.22:0/3655150066 >> v2:172.21.15.151:3300/0 conn(0x7f165c06ca50 0x7f165c15fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:24.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.335+0000 7f165affd640 1 --2- 172.21.15.22:0/3655150066 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f165c06c670 0x7f165c0cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:24.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.335+0000 7f165a7fc640 1 -- 172.21.15.22:0/3655150066 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f165c06c670 msgr2=0x7f165c0cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:24.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.335+0000 7f165a7fc640 1 --2- 172.21.15.22:0/3655150066 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f165c06c670 0x7f165c0cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.335+0000 7f165a7fc640 1 -- 172.21.15.22:0/3655150066 >> v2:172.21.15.151:3300/0 conn(0x7f165c06ca50 msgr2=0x7f165c15fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:24.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.335+0000 7f165a7fc640 1 --2- 172.21.15.22:0/3655150066 >> v2:172.21.15.151:3300/0 conn(0x7f165c06ca50 0x7f165c15fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.335+0000 7f165a7fc640 1 -- 172.21.15.22:0/3655150066 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165c0b3710 con 0x7f165c0b41f0 2024-03-29T04:20:24.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.335+0000 7f165a7fc640 1 --2- 172.21.15.22:0/3655150066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f165c0b41f0 0x7f165c06e080 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f165c06ebe0 tx=0x7f164401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6bbc891e9ed78d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:24.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.335+0000 7f1659ffb640 1 -- 172.21.15.22:0/3655150066 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16440b0070 con 0x7f165c0b41f0 2024-03-29T04:20:24.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.335+0000 7f1659ffb640 1 -- 172.21.15.22:0/3655150066 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16440ab600 con 0x7f165c0b41f0 2024-03-29T04:20:24.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.335+0000 7f1659ffb640 1 -- 172.21.15.22:0/3655150066 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16440ab9d0 con 0x7f165c0b41f0 2024-03-29T04:20:24.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f166169e640 1 -- 172.21.15.22:0/3655150066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f165c0b41f0 msgr2=0x7f165c06e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:24.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f166169e640 1 --2- 172.21.15.22:0/3655150066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f165c0b41f0 0x7f165c06e080 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f165c06ebe0 tx=0x7f164401c260 comp rx=0 tx=0).stop 2024-03-29T04:20:24.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f166169e640 1 -- 172.21.15.22:0/3655150066 shutdown_connections 2024-03-29T04:20:24.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f166169e640 1 --2- 172.21.15.22:0/3655150066 >> v2:172.21.15.151:3300/0 conn(0x7f165c06ca50 0x7f165c15fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f166169e640 1 --2- 172.21.15.22:0/3655150066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f165c0b41f0 0x7f165c06e080 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f166169e640 1 --2- 172.21.15.22:0/3655150066 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f165c06c670 0x7f165c0cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f166169e640 1 -- 172.21.15.22:0/3655150066 >> 172.21.15.22:0/3655150066 conn(0x7f165c0cfee0 msgr2=0x7f165c0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:24.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f166169e640 1 -- 172.21.15.22:0/3655150066 shutdown_connections 2024-03-29T04:20:24.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f166169e640 1 -- 172.21.15.22:0/3655150066 wait complete. 2024-03-29T04:20:24.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f166169e640 1 Processor -- start 2024-03-29T04:20:24.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f166169e640 1 -- start start 2024-03-29T04:20:24.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f166169e640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f165c06c670 0x7f165c159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:24.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f166169e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f165c06ca50 0x7f165c13e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:24.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f166169e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f165c0b41f0 0x7f165c13ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:24.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f166169e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f165c06cfe0 con 0x7f165c0b41f0 2024-03-29T04:20:24.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f166169e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f165c06ce60 con 0x7f165c06ca50 2024-03-29T04:20:24.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f166169e640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f165c0ce5b0 con 0x7f165c06c670 2024-03-29T04:20:24.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f165b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f165c0b41f0 0x7f165c13ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:24.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f165b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f165c0b41f0 0x7f165c13ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52378/0 (socket says 172.21.15.22:52378) 2024-03-29T04:20:24.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f165b7fe640 1 -- 172.21.15.22:0/2676375344 learned_addr learned my addr 172.21.15.22:0/2676375344 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:24.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f165affd640 1 --2- 172.21.15.22:0/2676375344 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f165c06c670 0x7f165c159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:24.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f165a7fc640 1 --2- 172.21.15.22:0/2676375344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f165c06ca50 0x7f165c13e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:24.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f165b7fe640 1 -- 172.21.15.22:0/2676375344 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f165c06c670 msgr2=0x7f165c159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:24.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f165b7fe640 1 --2- 172.21.15.22:0/2676375344 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f165c06c670 0x7f165c159a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f165b7fe640 1 -- 172.21.15.22:0/2676375344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f165c06ca50 msgr2=0x7f165c13e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f165b7fe640 1 --2- 172.21.15.22:0/2676375344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f165c06ca50 0x7f165c13e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f165b7fe640 1 -- 172.21.15.22:0/2676375344 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165c1409f0 con 0x7f165c0b41f0 2024-03-29T04:20:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f165affd640 1 --2- 172.21.15.22:0/2676375344 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f165c06c670 0x7f165c159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f165a7fc640 1 --2- 172.21.15.22:0/2676375344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f165c06ca50 0x7f165c13e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f165b7fe640 1 --2- 172.21.15.22:0/2676375344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f165c0b41f0 0x7f165c13ed10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f1654007e70 tx=0x7f165400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f16537fe640 1 -- 172.21.15.22:0/2676375344 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654014620 con 0x7f165c0b41f0 2024-03-29T04:20:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f16537fe640 1 -- 172.21.15.22:0/2676375344 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1654019070 con 0x7f165c0b41f0 2024-03-29T04:20:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f166169e640 1 -- 172.21.15.22:0/2676375344 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f165c140cd0 con 0x7f165c0b41f0 2024-03-29T04:20:24.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f16537fe640 1 -- 172.21.15.22:0/2676375344 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654014a20 con 0x7f165c0b41f0 2024-03-29T04:20:24.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.339+0000 7f166169e640 1 -- 172.21.15.22:0/2676375344 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f165c141290 con 0x7f165c0b41f0 2024-03-29T04:20:24.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.343+0000 7f166169e640 1 -- 172.21.15.22:0/2676375344 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f165c068890 con 0x7f165c0b41f0 2024-03-29T04:20:24.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.343+0000 7f16537fe640 1 -- 172.21.15.22:0/2676375344 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1654014bc0 con 0x7f165c0b41f0 2024-03-29T04:20:24.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.343+0000 7f16537fe640 1 --2- 172.21.15.22:0/2676375344 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f163c067fc0 0x7f163c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:24.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.343+0000 7f16537fe640 1 -- 172.21.15.22:0/2676375344 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f1654086bf0 con 0x7f165c0b41f0 2024-03-29T04:20:24.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.343+0000 7f165affd640 1 --2- 172.21.15.22:0/2676375344 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f163c067fc0 0x7f163c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:24.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.343+0000 7f165affd640 1 --2- 172.21.15.22:0/2676375344 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f163c067fc0 0x7f163c06a480 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f164c0097c0 tx=0x7f164c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.347+0000 7f16537fe640 1 -- 172.21.15.22:0/2676375344 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1654011260 con 0x7f165c0b41f0 2024-03-29T04:20:24.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.499+0000 7f166169e640 1 -- 172.21.15.22:0/2676375344 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f165c0d0510 con 0x7f163c067fc0 2024-03-29T04:20:24.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.499+0000 7f16537fe640 1 -- 172.21.15.22:0/2676375344 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13381 (secure 0 0 0) 0x7f165c0d0510 con 0x7f163c067fc0 2024-03-29T04:20:24.483 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:20:24.483 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-29T04:20:24.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.503+0000 7f166169e640 1 -- 172.21.15.22:0/2676375344 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f163c067fc0 msgr2=0x7f163c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:24.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.503+0000 7f166169e640 1 --2- 172.21.15.22:0/2676375344 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f163c067fc0 0x7f163c06a480 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f164c0097c0 tx=0x7f164c008040 comp rx=0 tx=0).stop 2024-03-29T04:20:24.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.503+0000 7f166169e640 1 -- 172.21.15.22:0/2676375344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f165c0b41f0 msgr2=0x7f165c13ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:24.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.503+0000 7f166169e640 1 --2- 172.21.15.22:0/2676375344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f165c0b41f0 0x7f165c13ed10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f1654007e70 tx=0x7f165400e4b0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.503+0000 7f166169e640 1 -- 172.21.15.22:0/2676375344 shutdown_connections 2024-03-29T04:20:24.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.503+0000 7f166169e640 1 --2- 172.21.15.22:0/2676375344 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f163c067fc0 0x7f163c06a480 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.503+0000 7f166169e640 1 --2- 172.21.15.22:0/2676375344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f165c0b41f0 0x7f165c13ed10 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.503+0000 7f166169e640 1 --2- 172.21.15.22:0/2676375344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f165c06ca50 0x7f165c13e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.503+0000 7f166169e640 1 --2- 172.21.15.22:0/2676375344 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f165c06c670 0x7f165c159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:24.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.503+0000 7f166169e640 1 -- 172.21.15.22:0/2676375344 >> 172.21.15.22:0/2676375344 conn(0x7f165c0cfee0 msgr2=0x7f165c1437d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:24.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.503+0000 7f166169e640 1 -- 172.21.15.22:0/2676375344 shutdown_connections 2024-03-29T04:20:24.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:24.503+0000 7f166169e640 1 -- 172.21.15.22:0/2676375344 wait complete. 2024-03-29T04:20:24.500 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-03-29T04:20:23.751089+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":250624,"kb_used_data":5320,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754724096,"statfs":{"total":773094113280,"available":772837474304,"internally_reserved":0,"allocated":5447680,"data_stored":3048200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-173890432,"num_objects":-116,"num_object_clones":0,"num_object_copies":-232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-116,"num_whiteouts":0,"num_read":-41,"num_read_kb":0,"num_write":-175,"num_write_kb":-169853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.742748"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":217,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-29T04:20:16.100438+0000","last_change":"2024-03-29T04:17:25.361826+0000","last_active":"2024-03-29T04:20:16.100438+0000","last_peered":"2024-03-29T04:20:16.100438+0000","last_clean":"2024-03-29T04:20:16.100438+0000","last_became_active":"2024-03-29T04:17:25.361259+0000","last_became_peered":"2024-03-29T04:17:25.361259+0000","last_unstale":"2024-03-29T04:20:16.100438+0000","last_undegraded":"2024-03-29T04:20:16.100438+0000","last_fullsized":"2024-03-29T04:20:16.100438+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:17:24.334639+0000","last_clean_scrub_stamp":"2024-03-29T04:17:24.334639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:40.027873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":823801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35860,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335980,"statfs":{"total":96636764160,"available":96600043520,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35864,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335976,"statfs":{"total":96636764160,"available":96600039424,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31704,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":823801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:20:24.503 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-29T04:17:24.334639+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=29, tm_hour=4, tm_min=17, tm_sec=24, tm_wday=4, tm_yday=89, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=29, tm_hour=4, tm_min=20, tm_sec=19, tm_wday=4, tm_yday=89, tm_isdst=0) 2024-03-29T04:20:24.503 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-29T04:20:31.706 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:20:31.709 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:20:44.504 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:20:44.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.655+0000 7fb6598be640 1 Processor -- start 2024-03-29T04:20:44.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.655+0000 7fb6598be640 1 -- start start 2024-03-29T04:20:44.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.659+0000 7fb6598be640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7fb6540b41f0 0x7fb6540cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:44.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.659+0000 7fb6598be640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb65406ca50 0x7fb6540ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:44.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.659+0000 7fb6598be640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6540ccef0 con 0x7fb65406c670 2024-03-29T04:20:44.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.659+0000 7fb6598be640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fb6540b34c0 con 0x7fb65406ca50 2024-03-29T04:20:44.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.659+0000 7fb6598be640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7fb6540b3710 con 0x7fb6540b41f0 2024-03-29T04:20:44.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.659+0000 7fb6537fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb65406ca50 0x7fb6540ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:44.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.659+0000 7fb652ffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb65406c670 0x7fb65406e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:41392/0 (socket says 172.21.15.22:41392) 2024-03-29T04:20:44.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.659+0000 7fb652ffd640 1 -- 172.21.15.22:0/1864888378 learned_addr learned my addr 172.21.15.22:0/1864888378 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:44.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.659+0000 7fb6527fc640 1 --2- 172.21.15.22:0/1864888378 >> v2:172.21.15.151:3300/0 conn(0x7fb6540b41f0 0x7fb6540cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:44.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.659+0000 7fb6537fe640 1 -- 172.21.15.22:0/1864888378 >> v2:172.21.15.151:3300/0 conn(0x7fb6540b41f0 msgr2=0x7fb6540cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:44.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.659+0000 7fb6537fe640 1 --2- 172.21.15.22:0/1864888378 >> v2:172.21.15.151:3300/0 conn(0x7fb6540b41f0 0x7fb6540cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:44.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.659+0000 7fb6537fe640 1 -- 172.21.15.22:0/1864888378 >> v1:172.21.15.22:6789/0 conn(0x7fb65406c670 legacy=0x7fb65406e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:44.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.659+0000 7fb651ffb640 1 -- 172.21.15.22:0/1864888378 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1042099012 0 0) 0x7fb6540ccef0 con 0x7fb65406c670 2024-03-29T04:20:44.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.659+0000 7fb6537fe640 1 -- 172.21.15.22:0/1864888378 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6540b39f0 con 0x7fb65406ca50 2024-03-29T04:20:44.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb6537fe640 1 --2- 172.21.15.22:0/1864888378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb65406ca50 0x7fb6540ce5b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb65406ebe0 tx=0x7fb64c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d623fa4d4f7f6f0c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:44.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb651ffb640 1 -- 172.21.15.22:0/1864888378 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c0b1070 con 0x7fb65406ca50 2024-03-29T04:20:44.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb651ffb640 1 -- 172.21.15.22:0/1864888378 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb64c0ac5f0 con 0x7fb65406ca50 2024-03-29T04:20:44.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb651ffb640 1 -- 172.21.15.22:0/1864888378 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c0ac9c0 con 0x7fb65406ca50 2024-03-29T04:20:44.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb6598be640 1 -- 172.21.15.22:0/1864888378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb65406ca50 msgr2=0x7fb6540ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:44.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb6598be640 1 --2- 172.21.15.22:0/1864888378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb65406ca50 0x7fb6540ce5b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb65406ebe0 tx=0x7fb64c01c480 comp rx=0 tx=0).stop 2024-03-29T04:20:44.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb6598be640 1 -- 172.21.15.22:0/1864888378 shutdown_connections 2024-03-29T04:20:44.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb6598be640 1 --2- 172.21.15.22:0/1864888378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb65406ca50 0x7fb6540ce5b0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:44.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb6598be640 1 --2- 172.21.15.22:0/1864888378 >> v2:172.21.15.151:3300/0 conn(0x7fb6540b41f0 0x7fb6540cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:44.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb6598be640 1 -- 172.21.15.22:0/1864888378 >> 172.21.15.22:0/1864888378 conn(0x7fb6540cfee0 msgr2=0x7fb6540d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:44.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb6598be640 1 -- 172.21.15.22:0/1864888378 shutdown_connections 2024-03-29T04:20:44.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb6598be640 1 -- 172.21.15.22:0/1864888378 wait complete. 2024-03-29T04:20:44.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb6598be640 1 Processor -- start 2024-03-29T04:20:44.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb6598be640 1 -- start start 2024-03-29T04:20:44.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb6598be640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65406c670 0x7fb654072490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:44.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb6598be640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb65406ca50 0x7fb65415e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:44.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb6598be640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6540b41f0 0x7fb65415e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:44.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb6598be640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb654169070 con 0x7fb65406ca50 2024-03-29T04:20:44.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb6598be640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb654168ef0 con 0x7fb65406c670 2024-03-29T04:20:44.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb6598be640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7fb6541691f0 con 0x7fb6540b41f0 2024-03-29T04:20:44.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb6527fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb65406ca50 0x7fb65415e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:44.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb6527fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb65406ca50 0x7fb65415e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52110/0 (socket says 172.21.15.22:52110) 2024-03-29T04:20:44.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.663+0000 7fb6527fc640 1 -- 172.21.15.22:0/344477193 learned_addr learned my addr 172.21.15.22:0/344477193 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:44.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.667+0000 7fb652ffd640 1 --2- 172.21.15.22:0/344477193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65406c670 0x7fb654072490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:44.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.667+0000 7fb6537fe640 1 --2- 172.21.15.22:0/344477193 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6540b41f0 0x7fb65415e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:44.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.667+0000 7fb6527fc640 1 -- 172.21.15.22:0/344477193 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6540b41f0 msgr2=0x7fb65415e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:44.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.667+0000 7fb6527fc640 1 --2- 172.21.15.22:0/344477193 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6540b41f0 0x7fb65415e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:44.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.667+0000 7fb6527fc640 1 -- 172.21.15.22:0/344477193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65406c670 msgr2=0x7fb654072490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:44.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.667+0000 7fb6527fc640 1 --2- 172.21.15.22:0/344477193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65406c670 0x7fb654072490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:44.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.667+0000 7fb6527fc640 1 -- 172.21.15.22:0/344477193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb654072ab0 con 0x7fb65406ca50 2024-03-29T04:20:44.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.667+0000 7fb6527fc640 1 --2- 172.21.15.22:0/344477193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb65406ca50 0x7fb65415e1f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb648007ef0 tx=0x7fb64800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:44.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.667+0000 7fb63bfff640 1 -- 172.21.15.22:0/344477193 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648014620 con 0x7fb65406ca50 2024-03-29T04:20:44.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.667+0000 7fb63bfff640 1 -- 172.21.15.22:0/344477193 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb648019070 con 0x7fb65406ca50 2024-03-29T04:20:44.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.667+0000 7fb6598be640 1 -- 172.21.15.22:0/344477193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb654165ea0 con 0x7fb65406ca50 2024-03-29T04:20:44.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.667+0000 7fb63bfff640 1 -- 172.21.15.22:0/344477193 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648014a00 con 0x7fb65406ca50 2024-03-29T04:20:44.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.667+0000 7fb6598be640 1 -- 172.21.15.22:0/344477193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb654166460 con 0x7fb65406ca50 2024-03-29T04:20:44.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.667+0000 7fb6598be640 1 -- 172.21.15.22:0/344477193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb654068890 con 0x7fb65406ca50 2024-03-29T04:20:44.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.667+0000 7fb63bfff640 1 -- 172.21.15.22:0/344477193 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb648014ba0 con 0x7fb65406ca50 2024-03-29T04:20:44.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.667+0000 7fb63bfff640 1 --2- 172.21.15.22:0/344477193 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb640067fc0 0x7fb64006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:44.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.667+0000 7fb63bfff640 1 -- 172.21.15.22:0/344477193 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fb648086d10 con 0x7fb65406ca50 2024-03-29T04:20:44.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.667+0000 7fb652ffd640 1 --2- 172.21.15.22:0/344477193 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb640067fc0 0x7fb64006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:44.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.671+0000 7fb652ffd640 1 --2- 172.21.15.22:0/344477193 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb640067fc0 0x7fb64006a480 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb644000f70 tx=0x7fb644007c10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:44.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.671+0000 7fb63bfff640 1 -- 172.21.15.22:0/344477193 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fb648057630 con 0x7fb65406ca50 2024-03-29T04:20:44.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.827+0000 7fb6598be640 1 -- 172.21.15.22:0/344477193 --> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb6540b32a0 con 0x7fb640067fc0 2024-03-29T04:20:44.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.831+0000 7fb63bfff640 1 -- 172.21.15.22:0/344477193 <== mgr.4108 v2:172.21.15.151:6832/1191011275 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13360 (secure 0 0 0) 0x7fb6540b32a0 con 0x7fb640067fc0 2024-03-29T04:20:44.812 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:20:44.813 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-29T04:20:44.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.831+0000 7fb6598be640 1 -- 172.21.15.22:0/344477193 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb640067fc0 msgr2=0x7fb64006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:44.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.831+0000 7fb6598be640 1 --2- 172.21.15.22:0/344477193 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb640067fc0 0x7fb64006a480 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb644000f70 tx=0x7fb644007c10 comp rx=0 tx=0).stop 2024-03-29T04:20:44.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.831+0000 7fb6598be640 1 -- 172.21.15.22:0/344477193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb65406ca50 msgr2=0x7fb65415e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:44.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.831+0000 7fb6598be640 1 --2- 172.21.15.22:0/344477193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb65406ca50 0x7fb65415e1f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb648007ef0 tx=0x7fb64800e4b0 comp rx=0 tx=0).stop 2024-03-29T04:20:44.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.831+0000 7fb6598be640 1 -- 172.21.15.22:0/344477193 shutdown_connections 2024-03-29T04:20:44.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.831+0000 7fb6598be640 1 --2- 172.21.15.22:0/344477193 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7fb6540b41f0 0x7fb65415e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:44.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.831+0000 7fb6598be640 1 --2- 172.21.15.22:0/344477193 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7fb640067fc0 0x7fb64006a480 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:44.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.831+0000 7fb6598be640 1 --2- 172.21.15.22:0/344477193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb65406ca50 0x7fb65415e1f0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:44.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.831+0000 7fb6598be640 1 --2- 172.21.15.22:0/344477193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65406c670 0x7fb654072490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:44.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.831+0000 7fb6598be640 1 -- 172.21.15.22:0/344477193 >> 172.21.15.22:0/344477193 conn(0x7fb6540cfee0 msgr2=0x7fb654160060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:44.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.831+0000 7fb6598be640 1 -- 172.21.15.22:0/344477193 shutdown_connections 2024-03-29T04:20:44.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.835+0000 7fb6598be640 1 -- 172.21.15.22:0/344477193 wait complete. 2024-03-29T04:20:44.830 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-03-29T04:20:43.756017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":250612,"kb_used_data":5320,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754724108,"statfs":{"total":773094113280,"available":772837486592,"internally_reserved":0,"allocated":5447680,"data_stored":3048200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002965"},"pg_stats":[{"pgid":"1.0","version":"45'119","reported_seq":228,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-29T04:20:24.853518+0000","last_change":"2024-03-29T04:20:24.853590+0000","last_active":"2024-03-29T04:20:24.853518+0000","last_peered":"2024-03-29T04:20:24.853518+0000","last_clean":"2024-03-29T04:20:24.853518+0000","last_became_active":"2024-03-29T04:17:25.361259+0000","last_became_peered":"2024-03-29T04:17:25.361259+0000","last_unstale":"2024-03-29T04:20:24.853518+0000","last_undegraded":"2024-03-29T04:20:24.853518+0000","last_fullsized":"2024-03-29T04:20:24.853518+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'119","last_scrub_stamp":"2024-03-29T04:20:24.853346+0000","last_deep_scrub":"45'119","last_deep_scrub_stamp":"2024-03-29T04:20:24.853346+0000","last_clean_scrub_stamp":"2024-03-29T04:20:24.853346+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:42:08.919159+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1126400,"data_stored":823801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35860,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335980,"statfs":{"total":96636764160,"available":96600043520,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35864,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335976,"statfs":{"total":96636764160,"available":96600039424,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31704,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":532480,"data_stored":233433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1126400,"data_stored":823801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:20:44.831 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-29T04:20:44.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75e79f0640 1 Processor -- start 2024-03-29T04:20:44.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75e79f0640 1 -- start start 2024-03-29T04:20:44.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75e79f0640 1 --2- >> v2:172.21.15.151:3300/0 conn(0x7f75e006c4f0 0x7f75e00b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:44.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75e79f0640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75e00b4110 con 0x7f75e006c070 2024-03-29T04:20:44.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75e79f0640 1 -- --> v1:172.21.15.151:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75e00b43f0 con 0x7f75e0147d50 2024-03-29T04:20:44.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75e79f0640 1 -- --> v2:172.21.15.151:3300/0 -- mon_getmap magic: 0 -- 0x7f75e0148130 con 0x7f75e006c4f0 2024-03-29T04:20:44.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75e4f8b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f75e006c070 0x7f75e00b51b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:60596/0 (socket says 172.21.15.22:60596) 2024-03-29T04:20:44.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75e4f8b640 1 -- 172.21.15.22:0/1237671425 learned_addr learned my addr 172.21.15.22:0/1237671425 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-03-29T04:20:44.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75d7fff640 1 -- 172.21.15.22:0/1237671425 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1907318488 0 0) 0x7f75e00b4110 con 0x7f75e006c070 2024-03-29T04:20:44.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75d7fff640 1 -- 172.21.15.22:0/1237671425 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75cc003530 con 0x7f75e006c070 2024-03-29T04:20:44.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75e5f8d640 1 --2- 172.21.15.22:0/1237671425 >> v2:172.21.15.151:3300/0 conn(0x7f75e006c4f0 0x7f75e00b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:44.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75d7fff640 1 -- 172.21.15.22:0/1237671425 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2795787134 0 0) 0x7f75cc003530 con 0x7f75e006c070 2024-03-29T04:20:44.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75d7fff640 1 -- 172.21.15.22:0/1237671425 >> v2:172.21.15.151:3300/0 conn(0x7f75e006c4f0 msgr2=0x7f75e00b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:44.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75d7fff640 1 --2- 172.21.15.22:0/1237671425 >> v2:172.21.15.151:3300/0 conn(0x7f75e006c4f0 0x7f75e00b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:44.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75d7fff640 1 -- 172.21.15.22:0/1237671425 >> v1:172.21.15.151:6789/0 conn(0x7f75e0147d50 legacy=0x7f75e006b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:44.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75d7fff640 1 -- 172.21.15.22:0/1237671425 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e00b46d0 con 0x7f75e006c070 2024-03-29T04:20:44.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75d7fff640 1 -- 172.21.15.22:0/1237671425 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2897710572 0 0) 0x7f75d0001680 con 0x7f75e006c070 2024-03-29T04:20:44.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75d7fff640 1 -- 172.21.15.22:0/1237671425 >> v1:172.21.15.22:6789/0 conn(0x7f75e006c070 legacy=0x7f75e00b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:44.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75d7fff640 1 --2- 172.21.15.22:0/1237671425 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75cc003a10 0x7f75cc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:44.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75d7fff640 1 --2- 172.21.15.22:0/1237671425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75cc004810 0x7f75cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:44.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75d7fff640 1 --2- 172.21.15.22:0/1237671425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75cc005220 0x7f75cc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:44.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75d7fff640 1 -- 172.21.15.22:0/1237671425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc00c140 con 0x7f75cc005220 2024-03-29T04:20:44.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75d7fff640 1 -- 172.21.15.22:0/1237671425 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc00c2f0 con 0x7f75cc004810 2024-03-29T04:20:44.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75d7fff640 1 -- 172.21.15.22:0/1237671425 --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f75cc00c4f0 con 0x7f75cc003a10 2024-03-29T04:20:44.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75e5f8d640 1 --2- 172.21.15.22:0/1237671425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75cc005220 0x7f75cc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:44.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75e578c640 1 --2- 172.21.15.22:0/1237671425 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75cc003a10 0x7f75cc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:44.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75e5f8d640 1 -- 172.21.15.22:0/1237671425 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75cc003a10 msgr2=0x7f75cc003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:44.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75e5f8d640 1 --2- 172.21.15.22:0/1237671425 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75cc003a10 0x7f75cc003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:44.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75e5f8d640 1 -- 172.21.15.22:0/1237671425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75cc004810 msgr2=0x7f75cc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:44.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75e4f8b640 1 --2- 172.21.15.22:0/1237671425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75cc004810 0x7f75cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:44.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75e5f8d640 1 --2- 172.21.15.22:0/1237671425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75cc004810 0x7f75cc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:44.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75e5f8d640 1 -- 172.21.15.22:0/1237671425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75cc003530 con 0x7f75cc005220 2024-03-29T04:20:44.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.975+0000 7f75e4f8b640 1 --2- 172.21.15.22:0/1237671425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75cc004810 0x7f75cc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:44.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.979+0000 7f75e5f8d640 1 -- 172.21.15.22:0/1237671425 reap_dead start 2024-03-29T04:20:44.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.979+0000 7f75e5f8d640 1 --2- 172.21.15.22:0/1237671425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75cc005220 0x7f75cc00b9d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f75dc009ab0 tx=0x7f75dc01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a1e3870c3f02e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:44.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.979+0000 7f75d7fff640 1 -- 172.21.15.22:0/1237671425 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0096b0 con 0x7f75cc005220 2024-03-29T04:20:44.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.979+0000 7f75d7fff640 1 -- 172.21.15.22:0/1237671425 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75dc004020 con 0x7f75cc005220 2024-03-29T04:20:44.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.979+0000 7f75d7fff640 1 -- 172.21.15.22:0/1237671425 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc004410 con 0x7f75cc005220 2024-03-29T04:20:44.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.979+0000 7f75e79f0640 1 -- 172.21.15.22:0/1237671425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75cc005220 msgr2=0x7f75cc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:44.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.979+0000 7f75e79f0640 1 --2- 172.21.15.22:0/1237671425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75cc005220 0x7f75cc00b9d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f75dc009ab0 tx=0x7f75dc01ca90 comp rx=0 tx=0).stop 2024-03-29T04:20:44.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.979+0000 7f75e79f0640 1 -- 172.21.15.22:0/1237671425 shutdown_connections 2024-03-29T04:20:44.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.979+0000 7f75e79f0640 1 --2- 172.21.15.22:0/1237671425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75cc005220 0x7f75cc00b9d0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:44.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.979+0000 7f75e79f0640 1 -- 172.21.15.22:0/1237671425 >> 172.21.15.22:0/1237671425 conn(0x7f75e00d1070 msgr2=0x7f75e00d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:44.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.979+0000 7f75e79f0640 1 -- 172.21.15.22:0/1237671425 shutdown_connections 2024-03-29T04:20:44.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.979+0000 7f75e79f0640 1 -- 172.21.15.22:0/1237671425 wait complete. 2024-03-29T04:20:44.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.979+0000 7f75e79f0640 1 Processor -- start 2024-03-29T04:20:44.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.979+0000 7f75e79f0640 1 -- start start 2024-03-29T04:20:44.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e79f0640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75cc005220 0x7f75e01f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:44.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e79f0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75e0156cf0 0x7f75e01591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:44.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e79f0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75e013d7d0 0x7f75e013dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:44.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e79f0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f75e00b4c90 con 0x7f75e0156cf0 2024-03-29T04:20:44.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e79f0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f75e006c940 con 0x7f75e013d7d0 2024-03-29T04:20:44.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e79f0640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f75e00705a0 con 0x7f75cc005220 2024-03-29T04:20:44.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e4f8b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75e0156cf0 0x7f75e01591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:44.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e4f8b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75e0156cf0 0x7f75e01591a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44452/0 (socket says 172.21.15.22:44452) 2024-03-29T04:20:44.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e4f8b640 1 -- 172.21.15.22:0/3370418995 learned_addr learned my addr 172.21.15.22:0/3370418995 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:44.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e578c640 1 --2- 172.21.15.22:0/3370418995 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75cc005220 0x7f75e01f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:44.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e5f8d640 1 --2- 172.21.15.22:0/3370418995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75e013d7d0 0x7f75e013dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:44.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e4f8b640 1 -- 172.21.15.22:0/3370418995 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75cc005220 msgr2=0x7f75e01f4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:44.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e4f8b640 1 --2- 172.21.15.22:0/3370418995 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75cc005220 0x7f75e01f4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:44.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e4f8b640 1 -- 172.21.15.22:0/3370418995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75e013d7d0 msgr2=0x7f75e013dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:44.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e4f8b640 1 --2- 172.21.15.22:0/3370418995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75e013d7d0 0x7f75e013dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:44.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e4f8b640 1 -- 172.21.15.22:0/3370418995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e01585a0 con 0x7f75e0156cf0 2024-03-29T04:20:44.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e5f8d640 1 --2- 172.21.15.22:0/3370418995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75e013d7d0 0x7f75e013dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:20:44.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e4f8b640 1 --2- 172.21.15.22:0/3370418995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75e0156cf0 0x7f75e01591a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f75d0010b10 tx=0x7f75d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:44.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75d5ffb640 1 -- 172.21.15.22:0/3370418995 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d001b070 con 0x7f75e0156cf0 2024-03-29T04:20:44.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75d5ffb640 1 -- 172.21.15.22:0/3370418995 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75d0011040 con 0x7f75e0156cf0 2024-03-29T04:20:44.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75d5ffb640 1 -- 172.21.15.22:0/3370418995 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d000ee60 con 0x7f75e0156cf0 2024-03-29T04:20:44.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e79f0640 1 -- 172.21.15.22:0/3370418995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e0157500 con 0x7f75e0156cf0 2024-03-29T04:20:44.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e79f0640 1 -- 172.21.15.22:0/3370418995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e0157ac0 con 0x7f75e0156cf0 2024-03-29T04:20:44.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e79f0640 1 -- 172.21.15.22:0/3370418995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75e0068900 con 0x7f75e0156cf0 2024-03-29T04:20:44.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75d5ffb640 1 -- 172.21.15.22:0/3370418995 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f75d001e070 con 0x7f75e0156cf0 2024-03-29T04:20:44.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75d5ffb640 1 --2- 172.21.15.22:0/3370418995 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f75c4067ff0 0x7f75c406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:44.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75d5ffb640 1 -- 172.21.15.22:0/3370418995 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f75d0088ea0 con 0x7f75e0156cf0 2024-03-29T04:20:44.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.983+0000 7f75e578c640 1 --2- 172.21.15.22:0/3370418995 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f75c4067ff0 0x7f75c406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:44.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.987+0000 7f75e578c640 1 --2- 172.21.15.22:0/3370418995 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f75c4067ff0 0x7f75c406a4b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f75e0145b50 tx=0x7f75d8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:44.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:44.987+0000 7f75d5ffb640 1 -- 172.21.15.22:0/3370418995 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f75d0059860 con 0x7f75e0156cf0 2024-03-29T04:20:45.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.150+0000 7f75e79f0640 1 -- 172.21.15.22:0/3370418995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f75e0145ea0 con 0x7f75e0156cf0 2024-03-29T04:20:45.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.158+0000 7f75d5ffb640 1 -- 172.21.15.22:0/3370418995 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f75d0059160 con 0x7f75e0156cf0 2024-03-29T04:20:45.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.158+0000 7f75d5ffb640 1 -- 172.21.15.22:0/3370418995 <== mon.0 v2:172.21.15.22:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f75d005d510 con 0x7f75e0156cf0 2024-03-29T04:20:45.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.162+0000 7f75e79f0640 1 -- 172.21.15.22:0/3370418995 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f75c4067ff0 msgr2=0x7f75c406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:45.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.162+0000 7f75e79f0640 1 --2- 172.21.15.22:0/3370418995 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f75c4067ff0 0x7f75c406a4b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f75e0145b50 tx=0x7f75d8009040 comp rx=0 tx=0).stop 2024-03-29T04:20:45.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.162+0000 7f75e79f0640 1 -- 172.21.15.22:0/3370418995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75e0156cf0 msgr2=0x7f75e01591a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:45.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.162+0000 7f75e79f0640 1 --2- 172.21.15.22:0/3370418995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75e0156cf0 0x7f75e01591a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f75d0010b10 tx=0x7f75d00057f0 comp rx=0 tx=0).stop 2024-03-29T04:20:45.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.162+0000 7f75e79f0640 1 -- 172.21.15.22:0/3370418995 shutdown_connections 2024-03-29T04:20:45.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.162+0000 7f75e79f0640 1 --2- 172.21.15.22:0/3370418995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75e013d7d0 0x7f75e013dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:45.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.162+0000 7f75e79f0640 1 --2- 172.21.15.22:0/3370418995 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f75c4067ff0 0x7f75c406a4b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:45.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.162+0000 7f75e79f0640 1 --2- 172.21.15.22:0/3370418995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75e0156cf0 0x7f75e01591a0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:45.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.162+0000 7f75e79f0640 1 --2- 172.21.15.22:0/3370418995 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f75cc005220 0x7f75e01f4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:45.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.162+0000 7f75e79f0640 1 -- 172.21.15.22:0/3370418995 >> 172.21.15.22:0/3370418995 conn(0x7f75e00d1070 msgr2=0x7f75e015dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:45.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.162+0000 7f75e79f0640 1 -- 172.21.15.22:0/3370418995 shutdown_connections 2024-03-29T04:20:45.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.162+0000 7f75e79f0640 1 -- 172.21.15.22:0/3370418995 wait complete. 2024-03-29T04:20:45.160 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T04:20:45.160 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:20:45.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.302+0000 7f9cb3d0b640 1 Processor -- start 2024-03-29T04:20:45.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.302+0000 7f9cb3d0b640 1 -- start start 2024-03-29T04:20:45.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.306+0000 7f9cb3d0b640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9cac163370 0x7f9cac166e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:45.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.306+0000 7f9cb3d0b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9cac163750 0x7f9cac0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:45.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.306+0000 7f9cb3d0b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cac0b3890 con 0x7f9cac06c670 2024-03-29T04:20:45.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.306+0000 7f9cb3d0b640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f9cac0b3b70 con 0x7f9cac163750 2024-03-29T04:20:45.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.306+0000 7f9cb3d0b640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9cac166390 con 0x7f9cac163370 2024-03-29T04:20:45.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.306+0000 7f9cb12a6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9cac163750 0x7f9cac0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:45.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.306+0000 7f9cb12a6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9cac163750 0x7f9cac0b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44464/0 (socket says 172.21.15.22:44464) 2024-03-29T04:20:45.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.306+0000 7f9cb22a8640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f9cac06c670 0x7f9cac0714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:60602/0 (socket says 172.21.15.22:60602) 2024-03-29T04:20:45.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.306+0000 7f9cb12a6640 1 -- 172.21.15.22:0/2158084105 learned_addr learned my addr 172.21.15.22:0/2158084105 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:45.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.306+0000 7f9cb1aa7640 1 --2- 172.21.15.22:0/2158084105 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9cac163370 0x7f9cac166e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:45.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.306+0000 7f9cb12a6640 1 -- 172.21.15.22:0/2158084105 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9cac163370 msgr2=0x7f9cac166e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:20:45.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.306+0000 7f9cb12a6640 1 --2- 172.21.15.22:0/2158084105 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9cac163370 0x7f9cac166e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:45.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.306+0000 7f9cb12a6640 1 -- 172.21.15.22:0/2158084105 >> v1:172.21.15.22:6789/0 conn(0x7f9cac06c670 legacy=0x7f9cac0714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:45.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.306+0000 7f9cb0aa5640 1 -- 172.21.15.22:0/2158084105 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3849920416 0 0) 0x7f9cac0b3890 con 0x7f9cac06c670 2024-03-29T04:20:45.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.306+0000 7f9cb12a6640 1 -- 172.21.15.22:0/2158084105 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cac166670 con 0x7f9cac163750 2024-03-29T04:20:45.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.306+0000 7f9cb12a6640 1 --2- 172.21.15.22:0/2158084105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cac163750 0x7f9cac0b2ef0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0098a0 tx=0x7f9c9c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c487cc7d4ce86d36 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:45.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.306+0000 7f9cb0aa5640 1 -- 172.21.15.22:0/2158084105 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c0b0070 con 0x7f9cac163750 2024-03-29T04:20:45.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.306+0000 7f9cb0aa5640 1 -- 172.21.15.22:0/2158084105 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9c9c0ab600 con 0x7f9cac163750 2024-03-29T04:20:45.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.306+0000 7f9cb0aa5640 1 -- 172.21.15.22:0/2158084105 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c0ab9f0 con 0x7f9cac163750 2024-03-29T04:20:45.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb3d0b640 1 -- 172.21.15.22:0/2158084105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cac163750 msgr2=0x7f9cac0b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:45.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb3d0b640 1 --2- 172.21.15.22:0/2158084105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cac163750 0x7f9cac0b2ef0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0098a0 tx=0x7f9c9c01c4b0 comp rx=0 tx=0).stop 2024-03-29T04:20:45.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb3d0b640 1 -- 172.21.15.22:0/2158084105 shutdown_connections 2024-03-29T04:20:45.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb3d0b640 1 --2- 172.21.15.22:0/2158084105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cac163750 0x7f9cac0b2ef0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:45.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb3d0b640 1 --2- 172.21.15.22:0/2158084105 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9cac163370 0x7f9cac166e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:45.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb3d0b640 1 -- 172.21.15.22:0/2158084105 >> 172.21.15.22:0/2158084105 conn(0x7f9cac0cfee0 msgr2=0x7f9cac0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:45.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb3d0b640 1 -- 172.21.15.22:0/2158084105 shutdown_connections 2024-03-29T04:20:45.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb3d0b640 1 -- 172.21.15.22:0/2158084105 wait complete. 2024-03-29T04:20:45.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb3d0b640 1 Processor -- start 2024-03-29T04:20:45.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb3d0b640 1 -- start start 2024-03-29T04:20:45.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb3d0b640 1 --2- >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9cac06c670 0x7f9cac15c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:45.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb3d0b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cac163370 0x7f9cac158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:45.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb3d0b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cac163750 0x7f9cac1560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:45.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb3d0b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac0b35b0 con 0x7f9cac163370 2024-03-29T04:20:45.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb3d0b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac0b3430 con 0x7f9cac163750 2024-03-29T04:20:45.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb3d0b640 1 -- --> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] -- mon_getmap magic: 0 -- 0x7f9cac071fc0 con 0x7f9cac06c670 2024-03-29T04:20:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb12a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cac163370 0x7f9cac158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb12a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cac163370 0x7f9cac158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44476/0 (socket says 172.21.15.22:44476) 2024-03-29T04:20:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb12a6640 1 -- 172.21.15.22:0/2085838583 learned_addr learned my addr 172.21.15.22:0/2085838583 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-29T04:20:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb12a6640 1 -- 172.21.15.22:0/2085838583 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9cac06c670 msgr2=0x7f9cac15c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb12a6640 1 --2- 172.21.15.22:0/2085838583 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9cac06c670 0x7f9cac15c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb12a6640 1 -- 172.21.15.22:0/2085838583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cac163750 msgr2=0x7f9cac1560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb12a6640 1 --2- 172.21.15.22:0/2085838583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cac163750 0x7f9cac1560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb12a6640 1 -- 172.21.15.22:0/2085838583 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cac151170 con 0x7f9cac163370 2024-03-29T04:20:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb12a6640 1 --2- 172.21.15.22:0/2085838583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cac163370 0x7f9cac158e40 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0ae040 tx=0x7f9c9c019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9ca27fc640 1 -- 172.21.15.22:0/2085838583 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c0b0040 con 0x7f9cac163370 2024-03-29T04:20:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9ca27fc640 1 -- 172.21.15.22:0/2085838583 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9c9c0b9070 con 0x7f9cac163370 2024-03-29T04:20:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb3d0b640 1 -- 172.21.15.22:0/2085838583 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cac151450 con 0x7f9cac163370 2024-03-29T04:20:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9ca27fc640 1 -- 172.21.15.22:0/2085838583 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c0b48c0 con 0x7f9cac163370 2024-03-29T04:20:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.310+0000 7f9cb3d0b640 1 -- 172.21.15.22:0/2085838583 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cac151a10 con 0x7f9cac163370 2024-03-29T04:20:45.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.314+0000 7f9cb3d0b640 1 -- 172.21.15.22:0/2085838583 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cac1524d0 con 0x7f9cac163370 2024-03-29T04:20:45.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.314+0000 7f9ca27fc640 1 -- 172.21.15.22:0/2085838583 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9c9c0b4a60 con 0x7f9cac163370 2024-03-29T04:20:45.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.314+0000 7f9ca27fc640 1 --2- 172.21.15.22:0/2085838583 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9c90067fc0 0x7f9c9006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:20:45.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.314+0000 7f9ca27fc640 1 -- 172.21.15.22:0/2085838583 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f9c9c120280 con 0x7f9cac163370 2024-03-29T04:20:45.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.314+0000 7f9cb1aa7640 1 --2- 172.21.15.22:0/2085838583 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9c90067fc0 0x7f9c9006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:20:45.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.314+0000 7f9cb1aa7640 1 --2- 172.21.15.22:0/2085838583 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9c90067fc0 0x7f9c9006a480 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400a810 tx=0x7f9ca4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:20:45.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.318+0000 7f9ca27fc640 1 -- 172.21.15.22:0/2085838583 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9cac1524d0 con 0x7f9cac163370 2024-03-29T04:20:45.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.510+0000 7f9cb3d0b640 1 -- 172.21.15.22:0/2085838583 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cac068890 con 0x7f9cac163370 2024-03-29T04:20:45.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.510+0000 7f9ca27fc640 1 -- 172.21.15.22:0/2085838583 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+3866 (secure 0 0 0) 0x7f9c9c016130 con 0x7f9cac163370 2024-03-29T04:20:45.492 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-29T04:20:45.492 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":22,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/3112156479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":3112156479},{"type":"v1","addr":"172.21.15.151:6835","nonce":3112156479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5308,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/491273202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":491273202},{"type":"v1","addr":"172.21.15.151:6837","nonce":491273202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/434199693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":434199693},{"type":"v1","addr":"172.21.15.22:6833","nonce":434199693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5976,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/409109813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":409109813},{"type":"v1","addr":"172.21.15.22:6835","nonce":409109813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[]} 2024-03-29T04:20:45.492 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 22 2024-03-29T04:20:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.510+0000 7f9cb3d0b640 1 -- 172.21.15.22:0/2085838583 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9c90067fc0 msgr2=0x7f9c9006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.510+0000 7f9cb3d0b640 1 --2- 172.21.15.22:0/2085838583 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9c90067fc0 0x7f9c9006a480 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400a810 tx=0x7f9ca4008040 comp rx=0 tx=0).stop 2024-03-29T04:20:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.510+0000 7f9cb3d0b640 1 -- 172.21.15.22:0/2085838583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cac163370 msgr2=0x7f9cac158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:20:45.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.510+0000 7f9cb3d0b640 1 --2- 172.21.15.22:0/2085838583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cac163370 0x7f9cac158e40 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0ae040 tx=0x7f9c9c019250 comp rx=0 tx=0).stop 2024-03-29T04:20:45.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.510+0000 7f9cb3d0b640 1 -- 172.21.15.22:0/2085838583 shutdown_connections 2024-03-29T04:20:45.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.510+0000 7f9cb3d0b640 1 --2- 172.21.15.22:0/2085838583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cac163750 0x7f9cac1560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:45.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.510+0000 7f9cb3d0b640 1 --2- 172.21.15.22:0/2085838583 >> [v2:172.21.15.151:6832/1191011275,v1:172.21.15.151:6833/1191011275] conn(0x7f9c90067fc0 0x7f9c9006a480 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:45.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.510+0000 7f9cb3d0b640 1 --2- 172.21.15.22:0/2085838583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cac163370 0x7f9cac158e40 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:45.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.510+0000 7f9cb3d0b640 1 --2- 172.21.15.22:0/2085838583 >> [v2:172.21.15.151:3301/0,v1:172.21.15.151:6790/0] conn(0x7f9cac06c670 0x7f9cac15c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:20:45.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.510+0000 7f9cb3d0b640 1 -- 172.21.15.22:0/2085838583 >> 172.21.15.22:0/2085838583 conn(0x7f9cac0cfee0 msgr2=0x7f9cac0d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:20:45.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.510+0000 7f9cb3d0b640 1 -- 172.21.15.22:0/2085838583 shutdown_connections 2024-03-29T04:20:45.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-29T04:20:45.510+0000 7f9cb3d0b640 1 -- 172.21.15.22:0/2085838583 wait complete. 2024-03-29T04:20:45.509 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-29T04:20:45.509 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-29T04:20:45.509 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-29T04:20:45.510 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:20:45.521 INFO:tasks.ceph.mds.a:Stopped 2024-03-29T04:20:45.522 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-29T04:20:45.522 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:20:45.573 INFO:tasks.ceph.mds.c:Stopped 2024-03-29T04:20:45.574 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-29T04:20:45.574 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:20:45.592 INFO:tasks.ceph.mds.b:Stopped 2024-03-29T04:20:45.593 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-29T04:20:45.593 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:20:45.655 INFO:tasks.ceph.mds.d:Stopped 2024-03-29T04:20:45.655 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-29T04:20:45.656 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-29T04:20:45.656 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:20:45.727 INFO:tasks.ceph.osd.0:Stopped 2024-03-29T04:20:45.728 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-29T04:20:45.728 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:20:51.082 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-03-29T04:20:51.834 INFO:tasks.ceph.osd.1:Stopped 2024-03-29T04:20:51.834 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-29T04:20:51.834 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:20:51.893 INFO:tasks.ceph.osd.2:Stopped 2024-03-29T04:20:51.893 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-29T04:20:51.893 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:20:51.990 INFO:tasks.ceph.osd.3:Stopped 2024-03-29T04:20:51.990 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-29T04:20:51.990 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:20:52.059 INFO:tasks.ceph.osd.4:Stopped 2024-03-29T04:20:52.059 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-29T04:20:52.059 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:20:56.789 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-03-29T04:20:56.789 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-03-29T04:20:58.161 INFO:tasks.ceph.osd.5:Stopped 2024-03-29T04:20:58.163 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-29T04:20:58.163 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:20:58.226 INFO:tasks.ceph.osd.6:Stopped 2024-03-29T04:20:58.227 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-29T04:20:58.227 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:21:01.812 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:21:01.815 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:21:02.294 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-03-29T04:21:02.294 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-03-29T04:21:04.329 INFO:tasks.ceph.osd.7:Stopped 2024-03-29T04:21:04.329 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-29T04:21:04.330 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-29T04:21:04.330 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:21:04.358 INFO:tasks.ceph.mgr.y:Stopped 2024-03-29T04:21:04.359 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-29T04:21:04.359 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:21:04.388 INFO:tasks.ceph.mgr.x:Stopped 2024-03-29T04:21:04.389 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-29T04:21:04.389 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-29T04:21:04.389 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:21:04.421 INFO:tasks.ceph.mon.a:Stopped 2024-03-29T04:21:04.421 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-29T04:21:04.422 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:21:04.445 INFO:tasks.ceph.mon.b:Stopped 2024-03-29T04:21:04.445 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-29T04:21:04.445 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:21:04.501 INFO:tasks.ceph.mon.c:Stopped 2024-03-29T04:21:04.502 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-29T04:21:04.502 DEBUG:teuthology.orchestra.run.smithi022:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-03-29T04:21:04.520 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi022.front.sepia.ceph.com 2024-03-29T04:21:04.520 DEBUG:teuthology.orchestra.run.smithi022:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-29T04:21:04.804 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi022.front.sepia.ceph.com 2024-03-29T04:21:04.804 DEBUG:teuthology.orchestra.run.smithi022:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-29T04:21:05.072 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi022.front.sepia.ceph.com 2024-03-29T04:21:05.073 DEBUG:teuthology.orchestra.run.smithi022:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-29T04:21:05.289 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi022.front.sepia.ceph.com 2024-03-29T04:21:05.290 DEBUG:teuthology.orchestra.run.smithi022:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-29T04:21:05.481 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi151.front.sepia.ceph.com 2024-03-29T04:21:05.481 DEBUG:teuthology.orchestra.run.smithi151:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-29T04:21:05.662 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi151.front.sepia.ceph.com 2024-03-29T04:21:05.662 DEBUG:teuthology.orchestra.run.smithi151:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-29T04:21:05.855 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi151.front.sepia.ceph.com 2024-03-29T04:21:05.855 DEBUG:teuthology.orchestra.run.smithi151:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-29T04:21:06.012 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi151.front.sepia.ceph.com 2024-03-29T04:21:06.013 DEBUG:teuthology.orchestra.run.smithi151:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-29T04:21:06.180 INFO:tasks.ceph:Archiving mon data... 2024-03-29T04:21:06.182 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629086/data/mon.a.tgz 2024-03-29T04:21:06.182 DEBUG:teuthology.orchestra.run.smithi022:> mktemp 2024-03-29T04:21:06.188 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.Lq3l1TAWFT 2024-03-29T04:21:06.188 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Lq3l1TAWFT 2024-03-29T04:21:06.310 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.Lq3l1TAWFT 2024-03-29T04:21:06.375 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.Lq3l1TAWFT is 330KB 2024-03-29T04:21:06.436 DEBUG:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.Lq3l1TAWFT 2024-03-29T04:21:06.442 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629086/data/mon.b.tgz 2024-03-29T04:21:06.442 DEBUG:teuthology.orchestra.run.smithi151:> mktemp 2024-03-29T04:21:06.448 INFO:teuthology.orchestra.run.smithi151.stdout:/tmp/tmp.PjtXOU6RQQ 2024-03-29T04:21:06.449 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.PjtXOU6RQQ 2024-03-29T04:21:06.597 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0666 /tmp/tmp.PjtXOU6RQQ 2024-03-29T04:21:06.661 DEBUG:teuthology.orchestra.remote:smithi151:/tmp/tmp.PjtXOU6RQQ is 437KB 2024-03-29T04:21:06.727 DEBUG:teuthology.orchestra.run.smithi151:> rm -fr /tmp/tmp.PjtXOU6RQQ 2024-03-29T04:21:06.732 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629086/data/mon.c.tgz 2024-03-29T04:21:06.733 DEBUG:teuthology.orchestra.run.smithi151:> mktemp 2024-03-29T04:21:06.781 INFO:teuthology.orchestra.run.smithi151.stdout:/tmp/tmp.U4CPfDvtPz 2024-03-29T04:21:06.781 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.U4CPfDvtPz 2024-03-29T04:21:06.930 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0666 /tmp/tmp.U4CPfDvtPz 2024-03-29T04:21:06.992 DEBUG:teuthology.orchestra.remote:smithi151:/tmp/tmp.U4CPfDvtPz is 437KB 2024-03-29T04:21:07.051 DEBUG:teuthology.orchestra.run.smithi151:> rm -fr /tmp/tmp.U4CPfDvtPz 2024-03-29T04:21:07.056 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-29T04:21:07.056 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-29T04:21:07.058 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-29T04:21:07.119 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-29T04:21:07.161 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-29T04:21:07.162 INFO:tasks.ceph:Archiving crash dumps... 2024-03-29T04:21:07.163 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629086/remote/smithi022/crash 2024-03-29T04:21:07.164 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-29T04:21:07.179 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629086/remote/smithi151/crash 2024-03-29T04:21:07.180 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-29T04:21:07.216 INFO:tasks.ceph:Shutting down logrotate 2024-03-29T04:21:07.217 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:21:07.223 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:21:07.293 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-29T04:21:07.295 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-03-29T04:21:07.323 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-29T04:21:07.349 INFO:tasks.ceph:Compressing logs... 2024-03-29T04:21:07.350 DEBUG:teuthology.orchestra.run.smithi022:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-29T04:21:07.368 DEBUG:teuthology.orchestra.run.smithi151:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-29T04:21:07.389 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27646.log 2024-03-29T04:21:07.390 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34128.log 2024-03-29T04:21:07.390 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.27646.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.27646.log.gz 2024-03-29T04:21:07.390 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32579.log 2024-03-29T04:21:07.391 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34323.log 2024-03-29T04:21:07.391 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34128.log: /var/log/ceph/ceph-client.admin.32579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38269.log 2024-03-29T04:21:07.391 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34128.log.gz 2024-03-29T04:21:07.392 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32579.log.gz 2024-03-29T04:21:07.392 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36826.log 2024-03-29T04:21:07.393 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34323.log.gz 2024-03-29T04:21:07.394 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34133.log 2024-03-29T04:21:07.394 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38269.log: /var/log/ceph/ceph-client.admin.36826.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-29T04:21:07.394 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39184.log 2024-03-29T04:21:07.394 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36826.log.gz 2024-03-29T04:21:07.394 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-mgr.y.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38269.log.gz 2024-03-29T04:21:07.394 INFO:teuthology.orchestra.run.smithi022.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34133.log.gz 2024-03-29T04:21:07.394 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39526.log 2024-03-29T04:21:07.394 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33057.log 2024-03-29T04:21:07.407 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39526.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39184.log.gz 2024-03-29T04:21:07.408 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36983.log 2024-03-29T04:21:07.408 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33057.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39526.log.gz 2024-03-29T04:21:07.408 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38431.log 2024-03-29T04:21:07.408 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35743.log 2024-03-29T04:21:07.408 INFO:teuthology.orchestra.run.smithi022.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.33057.log.gz 2024-03-29T04:21:07.408 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38431.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36983.log.gz 2024-03-29T04:21:07.408 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37300.log 2024-03-29T04:21:07.409 INFO:teuthology.orchestra.run.smithi022.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38431.log.gz 2024-03-29T04:21:07.409 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36081.log 2024-03-29T04:21:07.409 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35743.log: /var/log/ceph/ceph-client.admin.37300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35026.log 2024-03-29T04:21:07.409 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35743.log.gz 2024-03-29T04:21:07.409 INFO:teuthology.orchestra.run.smithi022.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37300.log.gz 2024-03-29T04:21:07.409 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39953.log 2024-03-29T04:21:07.410 INFO:teuthology.orchestra.run.smithi022.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36081.log.gz 2024-03-29T04:21:07.410 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37586.log 2024-03-29T04:21:07.410 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39953.log: 93.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-29T04:21:07.410 INFO:teuthology.orchestra.run.smithi022.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35026.log.gz 2024-03-29T04:21:07.410 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35612.log 2024-03-29T04:21:07.410 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39953.log.gz 2024-03-29T04:21:07.410 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35795.log 2024-03-29T04:21:07.411 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37586.log.gz 2024-03-29T04:21:07.411 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35612.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-29T04:21:07.411 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35795.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35612.log.gz 2024-03-29T04:21:07.411 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36401.log 2024-03-29T04:21:07.411 INFO:teuthology.orchestra.run.smithi022.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35795.log.gz 2024-03-29T04:21:07.411 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33923.log 2024-03-29T04:21:07.411 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39849.log 2024-03-29T04:21:07.411 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37170.log 2024-03-29T04:21:07.412 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36401.log.gz 2024-03-29T04:21:07.412 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39849.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33923.log.gz 2024-03-29T04:21:07.412 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39343.log 2024-03-29T04:21:07.412 INFO:teuthology.orchestra.run.smithi022.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39849.log.gz 2024-03-29T04:21:07.412 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38747.log 2024-03-29T04:21:07.412 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39343.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37170.log.gz 2024-03-29T04:21:07.412 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32821.log 2024-03-29T04:21:07.412 INFO:teuthology.orchestra.run.smithi022.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39343.log.gz 2024-03-29T04:21:07.412 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36774.log 2024-03-29T04:21:07.413 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38747.log.gz 2024-03-29T04:21:07.413 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34913.log 2024-03-29T04:21:07.413 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33917.log 2024-03-29T04:21:07.413 INFO:teuthology.orchestra.run.smithi022.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32821.log.gz 2024-03-29T04:21:07.413 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36774.log: /var/log/ceph/ceph-client.admin.34913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33005.log 2024-03-29T04:21:07.413 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36774.log.gz 2024-03-29T04:21:07.413 INFO:teuthology.orchestra.run.smithi022.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34913.log.gz 2024-03-29T04:21:07.413 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34625.log 2024-03-29T04:21:07.414 INFO:teuthology.orchestra.run.smithi022.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33917.log.gz 2024-03-29T04:21:07.414 INFO:teuthology.orchestra.run.smithi022.stderr:gzip/var/log/ceph/ceph-client.admin.33005.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39552.log 2024-03-29T04:21:07.414 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34625.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33005.log.gz 2024-03-29T04:21:07.414 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36271.log 2024-03-29T04:21:07.414 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34625.log.gz 2024-03-29T04:21:07.414 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39875.log 2024-03-29T04:21:07.414 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39552.log.gz 2024-03-29T04:21:07.414 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35847.log 2024-03-29T04:21:07.415 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39875.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36271.log.gz 2024-03-29T04:21:07.415 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35664.log 2024-03-29T04:21:07.415 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34939.log 2024-03-29T04:21:07.415 INFO:teuthology.orchestra.run.smithi022.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39875.log.gz 2024-03-29T04:21:07.415 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34341.log 2024-03-29T04:21:07.415 INFO:teuthology.orchestra.run.smithi022.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35847.log.gz 2024-03-29T04:21:07.415 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35664.log.gz 2024-03-29T04:21:07.415 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34547.log 2024-03-29T04:21:07.415 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34939.log.gz 2024-03-29T04:21:07.416 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33611.log 2024-03-29T04:21:07.416 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38022.log 2024-03-29T04:21:07.416 INFO:teuthology.orchestra.run.smithi022.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34341.log.gz 2024-03-29T04:21:07.416 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33611.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34547.log.gz 2024-03-29T04:21:07.416 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33922.log 2024-03-29T04:21:07.416 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33611.log.gz 2024-03-29T04:21:07.416 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33715.log 2024-03-29T04:21:07.418 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27666.log 2024-03-29T04:21:07.419 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33833.log 2024-03-29T04:21:07.419 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.27666.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-29T04:21:07.419 INFO:teuthology.orchestra.run.smithi151.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27666.log.gz 2024-03-29T04:21:07.419 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.1.33833.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-29T04:21:07.433 INFO:teuthology.orchestra.run.smithi151.stderr:gzip 90.5% -- replaced with /var/log/ceph/ceph-client.1.33833.log.gz -5 2024-03-29T04:21:07.433 INFO:teuthology.orchestra.run.smithi151.stderr: --verbose -- /var/log/ceph/ceph-client.admin.27793.log 2024-03-29T04:21:07.434 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27714.log 2024-03-29T04:21:07.434 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.27793.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-29T04:21:07.434 INFO:teuthology.orchestra.run.smithi151.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.27793.log.gz 2024-03-29T04:21:07.434 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.27714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27642.log 2024-03-29T04:21:07.434 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-osd.5.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.27714.log.gz 2024-03-29T04:21:07.434 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34129.log 2024-03-29T04:21:07.434 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.27642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33952.log 2024-03-29T04:21:07.434 INFO:teuthology.orchestra.run.smithi151.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.27642.log.gz 2024-03-29T04:21:07.435 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.1.34129.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-29T04:21:07.435 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.3.33952.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-29T04:21:07.435 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-29T04:21:07.435 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph.log: 91.5% -- replaced with /var/log/ceph/ceph-client.3.33952.log.gz 2024-03-29T04:21:07.435 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27762.log 2024-03-29T04:21:07.435 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-mon.c.log: 93.4% -- replaced with /var/log/ceph/ceph-client.1.34129.log.gz 2024-03-29T04:21:07.435 INFO:teuthology.orchestra.run.smithi151.stderr: 92.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-29T04:21:07.436 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-29T04:21:07.436 INFO:teuthology.orchestra.run.smithi151.stderr:gzip/var/log/ceph/ceph-client.admin.27762.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.27738.log 2024-03-29T04:21:07.436 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph.audit.log: gzip 88.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.27762.log.gz --verbose 2024-03-29T04:21:07.436 INFO:teuthology.orchestra.run.smithi151.stderr: -- /var/log/ceph/ceph-client.admin.27817.log 2024-03-29T04:21:07.436 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.27738.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-03-29T04:21:07.436 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.27817.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.27738.log.gz 2024-03-29T04:21:07.436 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27690.log 2024-03-29T04:21:07.437 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27817.log.gz 2024-03-29T04:21:07.437 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-29T04:21:07.437 INFO:teuthology.orchestra.run.smithi151.stderr: 93.7%/var/log/ceph/ceph-client.admin.27690.log: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-29T04:21:07.437 INFO:teuthology.orchestra.run.smithi151.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.27690.log.gz 2024-03-29T04:21:07.437 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-29T04:21:07.437 INFO:teuthology.orchestra.run.smithi151.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-03-29T04:21:07.437 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-29T04:21:07.458 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33319.log 2024-03-29T04:21:07.458 INFO:teuthology.orchestra.run.smithi022.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38022.log.gz 2024-03-29T04:21:07.458 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33715.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33922.log.gz 2024-03-29T04:21:07.459 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35560.log 2024-03-29T04:21:07.459 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33715.log.gz 2024-03-29T04:21:07.459 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33916.log 2024-03-29T04:21:07.459 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39158.log 2024-03-29T04:21:07.459 INFO:teuthology.orchestra.run.smithi022.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33319.log.gz 2024-03-29T04:21:07.459 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33916.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35560.log.gz 2024-03-29T04:21:07.459 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32847.log 2024-03-29T04:21:07.459 INFO:teuthology.orchestra.run.smithi022.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33916.log.gz 2024-03-29T04:21:07.460 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36931.log 2024-03-29T04:21:07.460 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32605.log 2024-03-29T04:21:07.460 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39158.log.gz 2024-03-29T04:21:07.460 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36931.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32847.log.gz 2024-03-29T04:21:07.460 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-29T04:21:07.460 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36931.log.gz 2024-03-29T04:21:07.460 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39023.log 2024-03-29T04:21:07.460 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40266.log 2024-03-29T04:21:07.461 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32605.log.gz 2024-03-29T04:21:07.461 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36166.log 2024-03-29T04:21:07.461 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39023.log.gz 2024-03-29T04:21:07.461 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.40266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39675.log 2024-03-29T04:21:07.461 INFO:teuthology.orchestra.run.smithi022.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40266.log.gz 2024-03-29T04:21:07.461 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35430.log 2024-03-29T04:21:07.461 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36166.log: /var/log/ceph/ceph-client.admin.39675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36668.log 2024-03-29T04:21:07.461 INFO:teuthology.orchestra.run.smithi022.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36166.log.gz 2024-03-29T04:21:07.462 INFO:teuthology.orchestra.run.smithi022.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39675.log.gz 2024-03-29T04:21:07.462 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39395.log 2024-03-29T04:21:07.462 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39473.log 2024-03-29T04:21:07.462 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36668.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35430.log.gz 2024-03-29T04:21:07.462 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39395.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36668.log.gz 2024-03-29T04:21:07.462 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36905.log 2024-03-29T04:21:07.462 INFO:teuthology.orchestra.run.smithi022.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39395.log.gz 2024-03-29T04:21:07.462 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38074.log 2024-03-29T04:21:07.463 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38561.log 2024-03-29T04:21:07.463 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39473.log.gz 2024-03-29T04:21:07.463 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36905.log.gz 2024-03-29T04:21:07.463 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37716.log 2024-03-29T04:21:07.463 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38074.log: /var/log/ceph/ceph-client.admin.38561.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38074.log.gz 2024-03-29T04:21:07.463 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38693.log 2024-03-29T04:21:07.463 INFO:teuthology.orchestra.run.smithi022.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38561.log.gz 2024-03-29T04:21:07.463 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37768.log 2024-03-29T04:21:07.464 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33637.log 2024-03-29T04:21:07.464 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.38693.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37716.log.gz 2024-03-29T04:21:07.464 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37768.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-29T04:21:07.464 INFO:teuthology.orchestra.run.smithi022.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37768.log.gz 2024-03-29T04:21:07.464 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38405.log 2024-03-29T04:21:07.464 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-mon.a.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33637.log.gz 2024-03-29T04:21:07.464 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40188.log 2024-03-29T04:21:07.464 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32661.log 2024-03-29T04:21:07.465 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38405.log: /var/log/ceph/ceph-client.admin.40188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32979.log 2024-03-29T04:21:07.465 INFO:teuthology.orchestra.run.smithi022.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38405.log.gz 2024-03-29T04:21:07.465 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.40188.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.37690.log 2024-03-29T04:21:07.465 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-03-29T04:21:07.465 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32661.log: /var/log/ceph/ceph-client.admin.32979.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32661.log.gz 2024-03-29T04:21:07.465 INFO:teuthology.orchestra.run.smithi022.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32979.log.gz 2024-03-29T04:21:07.465 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37274.log 2024-03-29T04:21:07.465 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36590.log 2024-03-29T04:21:07.465 INFO:teuthology.orchestra.run.smithi022.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37690.log.gz 2024-03-29T04:21:07.466 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33235.log 2024-03-29T04:21:07.466 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37274.log: /var/log/ceph/ceph-client.admin.36590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39771.log 2024-03-29T04:21:07.466 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37274.log.gz 2024-03-29T04:21:07.466 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36590.log.gz 2024-03-29T04:21:07.466 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37144.log 2024-03-29T04:21:07.466 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.39771.log: /var/log/ceph/ceph-client.admin.38126.log 2024-03-29T04:21:07.466 INFO:teuthology.orchestra.run.smithi022.stderr: 88.5%/var/log/ceph/ceph-client.admin.37144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37560.log 2024-03-29T04:21:07.467 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39771.log.gz 2024-03-29T04:21:07.467 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37144.log.gz 2024-03-29T04:21:07.467 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38126.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38126.log.gz 2024-03-29T04:21:07.467 INFO:teuthology.orchestra.run.smithi022.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.33235.log.gz 2024-03-29T04:21:07.467 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34703.log 2024-03-29T04:21:07.467 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32901.log 2024-03-29T04:21:07.467 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8%/var/log/ceph/ceph-client.admin.34703.log: -- replaced with /var/log/ceph/ceph-client.admin.37560.log.gz 2024-03-29T04:21:07.467 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40345.log 2024-03-29T04:21:07.467 INFO:teuthology.orchestra.run.smithi022.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39979.log 2024-03-29T04:21:07.468 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34703.log.gz/var/log/ceph/ceph-client.admin.32901.log: 2024-03-29T04:21:07.468 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.40345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34887.log 2024-03-29T04:21:07.468 INFO:teuthology.orchestra.run.smithi022.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32901.log.gz 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40345.log.gz 2024-03-29T04:21:07.468 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-03-29T04:21:07.468 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38353.log 2024-03-29T04:21:07.468 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39979.log: /var/log/ceph/ceph-client.admin.34887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34521.log 2024-03-29T04:21:07.468 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38353.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34887.log.gz 2024-03-29T04:21:07.468 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39979.log.gz 2024-03-29T04:21:07.469 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37404.log 2024-03-29T04:21:07.469 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34599.log 2024-03-29T04:21:07.469 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34521.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38353.log.gz 2024-03-29T04:21:07.469 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33423.log 2024-03-29T04:21:07.469 INFO:teuthology.orchestra.run.smithi022.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34521.log.gz 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37404.log.gz 2024-03-29T04:21:07.469 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34599.log: 2024-03-29T04:21:07.469 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38379.log 2024-03-29T04:21:07.469 INFO:teuthology.orchestra.run.smithi022.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34599.log.gz 2024-03-29T04:21:07.469 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38509.log 2024-03-29T04:21:07.470 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33423.log: /var/log/ceph/ceph-client.admin.38379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40162.log 2024-03-29T04:21:07.470 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33423.log.gz 2024-03-29T04:21:07.470 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38379.log.gz 2024-03-29T04:21:07.470 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32875.log 2024-03-29T04:21:07.470 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38509.log: /var/log/ceph/ceph-client.admin.40162.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38509.log.gz 2024-03-29T04:21:07.470 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40162.log.gz 2024-03-29T04:21:07.470 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35534.log 2024-03-29T04:21:07.471 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34965.log 2024-03-29T04:21:07.471 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32875.log.gz 2024-03-29T04:21:07.471 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34112.log 2024-03-29T04:21:07.471 INFO:teuthology.orchestra.run.smithi022.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35534.log.gz 2024-03-29T04:21:07.471 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36486.log 2024-03-29T04:21:07.471 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34965.log: /var/log/ceph/ceph-client.admin.34112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38327.log 2024-03-29T04:21:07.471 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35769.log 2024-03-29T04:21:07.471 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34112.log.gz 2024-03-29T04:21:07.472 INFO:teuthology.orchestra.run.smithi022.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.34965.log.gz 2024-03-29T04:21:07.472 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38327.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36486.log.gz 2024-03-29T04:21:07.472 INFO:teuthology.orchestra.run.smithi022.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38327.log.gz 2024-03-29T04:21:07.472 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-29T04:21:07.472 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37996.log 2024-03-29T04:21:07.472 INFO:teuthology.orchestra.run.smithi022.stderr: 87.8%/var/log/ceph/ceph-osd.3.log: -- replaced with /var/log/ceph/ceph-client.admin.35769.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35977.log 2024-03-29T04:21:07.472 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-03-29T04:21:07.472 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39236.log 2024-03-29T04:21:07.472 INFO:teuthology.orchestra.run.smithi022.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37996.log.gz 2024-03-29T04:21:07.473 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34861.log 2024-03-29T04:21:07.473 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33031.log 2024-03-29T04:21:07.473 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39236.log: 89.1%/var/log/ceph/ceph-client.admin.34861.log: -- replaced with /var/log/ceph/ceph-client.admin.35977.log.gz 2024-03-29T04:21:07.473 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39236.log.gz 2024-03-29T04:21:07.473 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34861.log.gz 2024-03-29T04:21:07.473 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38100.log 2024-03-29T04:21:07.473 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33293.log 2024-03-29T04:21:07.473 INFO:teuthology.orchestra.run.smithi022.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33031.log.gz 2024-03-29T04:21:07.474 INFO:teuthology.orchestra.run.smithi022.stderr:gzip/var/log/ceph/ceph-client.admin.38100.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38457.log 2024-03-29T04:21:07.474 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33293.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38100.log.gz 2024-03-29T04:21:07.474 INFO:teuthology.orchestra.run.smithi022.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33293.log.gz 2024-03-29T04:21:07.474 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37865.log 2024-03-29T04:21:07.474 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40136.log 2024-03-29T04:21:07.474 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7%/var/log/ceph/ceph-client.admin.37865.log: -- replaced with /var/log/ceph/ceph-client.admin.38457.log.gz 2024-03-29T04:21:07.474 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33904.log 2024-03-29T04:21:07.474 INFO:teuthology.orchestra.run.smithi022.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37865.log.gz 2024-03-29T04:21:07.475 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.40136.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40136.log.gz 2024-03-29T04:21:07.475 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37092.log 2024-03-29T04:21:07.475 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39369.log 2024-03-29T04:21:07.475 INFO:teuthology.orchestra.run.smithi022.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33904.log.gz 2024-03-29T04:21:07.475 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40110.log 2024-03-29T04:21:07.475 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37092.log.gz 2024-03-29T04:21:07.475 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39369.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39369.log.gz 2024-03-29T04:21:07.475 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36642.log 2024-03-29T04:21:07.475 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.40110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34677.log 2024-03-29T04:21:07.476 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6%/var/log/ceph/ceph-client.admin.36642.log: -- replaced with /var/log/ceph/ceph-client.admin.40110.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38587.log 2024-03-29T04:21:07.476 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-03-29T04:21:07.476 INFO:teuthology.orchestra.run.smithi022.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36642.log.gz 2024-03-29T04:21:07.476 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34677.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34677.log.gz 2024-03-29T04:21:07.476 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39317.log 2024-03-29T04:21:07.476 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38587.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-29T04:21:07.476 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36140.log 2024-03-29T04:21:07.476 INFO:teuthology.orchestra.run.smithi022.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.38587.log.gz 2024-03-29T04:21:07.477 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-osd.1.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39317.log.gz 2024-03-29T04:21:07.477 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32767.log 2024-03-29T04:21:07.477 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33793.log 2024-03-29T04:21:07.477 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36140.log.gz 2024-03-29T04:21:07.477 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35690.log 2024-03-29T04:21:07.477 INFO:teuthology.orchestra.run.smithi022.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32767.log.gz 2024-03-29T04:21:07.477 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37638.log 2024-03-29T04:21:07.477 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33793.log.gz 2024-03-29T04:21:07.478 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34835.log 2024-03-29T04:21:07.478 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37638.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35690.log.gz 2024-03-29T04:21:07.478 INFO:teuthology.orchestra.run.smithi022.stderr: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39649.log 2024-03-29T04:21:07.478 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37638.log.gz 2024-03-29T04:21:07.478 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36029.log 2024-03-29T04:21:07.478 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39649.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34835.log.gz 2024-03-29T04:21:07.478 INFO:teuthology.orchestra.run.smithi022.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32687.log 2024-03-29T04:21:07.478 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39649.log.gz 2024-03-29T04:21:07.479 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40084.log 2024-03-29T04:21:07.479 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32687.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36029.log.gz 2024-03-29T04:21:07.479 INFO:teuthology.orchestra.run.smithi022.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32687.log.gzgzip 2024-03-29T04:21:07.479 INFO:teuthology.orchestra.run.smithi022.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34809.log 2024-03-29T04:21:07.479 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36375.log 2024-03-29T04:21:07.479 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.40084.log: /var/log/ceph/ceph-client.admin.34809.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40084.log.gz 2024-03-29T04:21:07.479 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34809.log.gz 2024-03-29T04:21:07.479 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39447.log 2024-03-29T04:21:07.479 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37742.log 2024-03-29T04:21:07.481 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-osd.4.log: 92.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-29T04:21:07.493 INFO:teuthology.orchestra.run.smithi151.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-29T04:21:07.507 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36375.log.gz 2024-03-29T04:21:07.507 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37612.log 2024-03-29T04:21:07.507 INFO:teuthology.orchestra.run.smithi022.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39447.log.gz 2024-03-29T04:21:07.507 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38842.log 2024-03-29T04:21:07.507 INFO:teuthology.orchestra.run.smithi022.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37742.log.gz 2024-03-29T04:21:07.508 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35482.log 2024-03-29T04:21:07.508 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38842.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37612.log.gz 2024-03-29T04:21:07.508 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33741.log 2024-03-29T04:21:07.508 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35482.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.38842.log.gz 2024-03-29T04:21:07.508 INFO:teuthology.orchestra.run.smithi022.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35482.log.gz 2024-03-29T04:21:07.508 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39499.log 2024-03-29T04:21:07.508 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40032.log 2024-03-29T04:21:07.508 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2%/var/log/ceph/ceph-client.admin.39499.log: -- replaced with /var/log/ceph/ceph-client.admin.33741.log.gz 2024-03-29T04:21:07.509 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39499.log.gz 2024-03-29T04:21:07.509 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38637.log 2024-03-29T04:21:07.509 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.40032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37430.log 2024-03-29T04:21:07.509 INFO:teuthology.orchestra.run.smithi022.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40032.log.gz 2024-03-29T04:21:07.509 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34313.log 2024-03-29T04:21:07.509 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38637.log.gz 2024-03-29T04:21:07.509 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37430.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37352.log 2024-03-29T04:21:07.509 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37430.log.gz 2024-03-29T04:21:07.509 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33502.log 2024-03-29T04:21:07.510 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37352.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34313.log.gz 2024-03-29T04:21:07.510 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37352.log.gz 2024-03-29T04:21:07.510 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39927.log 2024-03-29T04:21:07.510 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33502.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33502.log.gz 2024-03-29T04:21:07.510 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37222.log 2024-03-29T04:21:07.510 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32515.log 2024-03-29T04:21:07.510 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39927.log: /var/log/ceph/ceph-client.admin.37222.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39927.log.gz 2024-03-29T04:21:07.511 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37222.log.gz 2024-03-29T04:21:07.511 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35346.log 2024-03-29T04:21:07.511 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35456.log 2024-03-29T04:21:07.511 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32515.log: /var/log/ceph/ceph-client.admin.35346.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32515.log.gz 2024-03-29T04:21:07.511 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35346.log.gz 2024-03-29T04:21:07.511 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39049.log 2024-03-29T04:21:07.511 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33397.log 2024-03-29T04:21:07.511 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35456.log: /var/log/ceph/ceph-client.admin.39049.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35456.log.gz 2024-03-29T04:21:07.511 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39049.log.gz 2024-03-29T04:21:07.512 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33476.log 2024-03-29T04:21:07.512 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-29T04:21:07.512 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33397.log: /var/log/ceph/ceph-client.admin.33476.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33397.log.gz 2024-03-29T04:21:07.512 INFO:teuthology.orchestra.run.smithi022.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33476.log.gz 2024-03-29T04:21:07.512 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33450.log 2024-03-29T04:21:07.512 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39578.log 2024-03-29T04:21:07.512 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.33450.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33450.log.gz 2024-03-29T04:21:07.512 INFO:teuthology.orchestra.run.smithi022.stderr: 85.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-29T04:21:07.512 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37009.log 2024-03-29T04:21:07.513 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22615.log 2024-03-29T04:21:07.513 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39578.log: /var/log/ceph/ceph-client.admin.37009.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39578.log.gz 2024-03-29T04:21:07.513 INFO:teuthology.orchestra.run.smithi022.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37009.log.gz 2024-03-29T04:21:07.513 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34131.log 2024-03-29T04:21:07.513 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37970.log 2024-03-29T04:21:07.513 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph.tmp-client.admin.22615.log: /var/log/ceph/ceph-client.admin.34131.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22615.log.gz 2024-03-29T04:21:07.513 INFO:teuthology.orchestra.run.smithi022.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38048.log 2024-03-29T04:21:07.513 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34131.log.gz 2024-03-29T04:21:07.514 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37970.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37970.log.gz 2024-03-29T04:21:07.514 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34340.log 2024-03-29T04:21:07.514 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32482.log 2024-03-29T04:21:07.514 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38048.log: /var/log/ceph/ceph-client.admin.34340.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38048.log.gz 2024-03-29T04:21:07.514 INFO:teuthology.orchestra.run.smithi022.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34340.log.gz 2024-03-29T04:21:07.514 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35873.log 2024-03-29T04:21:07.514 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27543.log 2024-03-29T04:21:07.514 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32482.log: /var/log/ceph/ceph-client.admin.35873.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32482.log.gz 2024-03-29T04:21:07.515 INFO:teuthology.orchestra.run.smithi022.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35873.log.gz 2024-03-29T04:21:07.515 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37456.log 2024-03-29T04:21:07.515 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36244.log 2024-03-29T04:21:07.515 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.27543.log: /var/log/ceph/ceph-client.admin.37456.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.27543.log.gz 2024-03-29T04:21:07.515 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37456.log.gz 2024-03-29T04:21:07.515 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35638.log 2024-03-29T04:21:07.515 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36244.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36244.log.gz 2024-03-29T04:21:07.515 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38217.log 2024-03-29T04:21:07.515 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39797.log 2024-03-29T04:21:07.516 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35638.log: /var/log/ceph/ceph-client.admin.38217.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35638.log.gz 2024-03-29T04:21:07.516 INFO:teuthology.orchestra.run.smithi022.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38217.log.gz 2024-03-29T04:21:07.516 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33767.log 2024-03-29T04:21:07.516 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39101.log 2024-03-29T04:21:07.516 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39797.log: /var/log/ceph/ceph-client.admin.33767.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39797.log.gz 2024-03-29T04:21:07.516 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33767.log.gz 2024-03-29T04:21:07.516 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38944.log 2024-03-29T04:21:07.516 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38182.log 2024-03-29T04:21:07.517 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39101.log: /var/log/ceph/ceph-client.admin.38944.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39101.log.gz 2024-03-29T04:21:07.517 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39623.log 2024-03-29T04:21:07.517 INFO:teuthology.orchestra.run.smithi022.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.38944.log.gz 2024-03-29T04:21:07.517 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38182.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38182.log.gz 2024-03-29T04:21:07.517 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33689.log 2024-03-29T04:21:07.517 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33345.log 2024-03-29T04:21:07.517 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33689.log: /var/log/ceph/ceph-client.admin.39623.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39623.log.gz 2024-03-29T04:21:07.517 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33689.log.gz 2024-03-29T04:21:07.517 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36218.log 2024-03-29T04:21:07.518 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32426.log 2024-03-29T04:21:07.518 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33345.log: /var/log/ceph/ceph-client.admin.36218.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33345.log.gz 2024-03-29T04:21:07.518 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36218.log.gz 2024-03-29T04:21:07.518 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34573.log 2024-03-29T04:21:07.518 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33915.log 2024-03-29T04:21:07.518 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32426.log: /var/log/ceph/ceph-client.admin.34573.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32426.log.gz 2024-03-29T04:21:07.518 INFO:teuthology.orchestra.run.smithi022.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34573.log.gz 2024-03-29T04:21:07.519 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34130.log 2024-03-29T04:21:07.519 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35716.log 2024-03-29T04:21:07.519 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33915.log: /var/log/ceph/ceph-client.admin.34130.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33915.log.gz 2024-03-29T04:21:07.519 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34130.log.gz 2024-03-29T04:21:07.519 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34124.log 2024-03-29T04:21:07.519 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33528.log 2024-03-29T04:21:07.519 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35716.log: /var/log/ceph/ceph-client.admin.34124.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35716.log.gz 2024-03-29T04:21:07.519 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34124.log.gz 2024-03-29T04:21:07.519 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33925.log 2024-03-29T04:21:07.520 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33528.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33528.log.gz 2024-03-29T04:21:07.520 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38802.log 2024-03-29T04:21:07.520 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32953.log 2024-03-29T04:21:07.520 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33925.log: /var/log/ceph/ceph-client.admin.38802.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33925.log.gz 2024-03-29T04:21:07.520 INFO:teuthology.orchestra.run.smithi022.stderr: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.38802.log.gz 2024-03-29T04:21:07.520 INFO:teuthology.orchestra.run.smithi022.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34320.log 2024-03-29T04:21:07.520 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32953.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32953.log.gz 2024-03-29T04:21:07.520 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34325.log 2024-03-29T04:21:07.521 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34731.log 2024-03-29T04:21:07.521 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34320.log: /var/log/ceph/ceph-client.admin.34325.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34320.log.gz 2024-03-29T04:21:07.521 INFO:teuthology.orchestra.run.smithi022.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34325.log.gz 2024-03-29T04:21:07.521 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36852.log 2024-03-29T04:21:07.521 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39421.log 2024-03-29T04:21:07.521 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34731.log: /var/log/ceph/ceph-client.admin.36852.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34731.log.gz 2024-03-29T04:21:07.521 INFO:teuthology.orchestra.run.smithi022.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36852.log.gz 2024-03-29T04:21:07.521 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32713.log 2024-03-29T04:21:07.522 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39421.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39421.log.gz 2024-03-29T04:21:07.522 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36564.log 2024-03-29T04:21:07.522 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38299.log 2024-03-29T04:21:07.522 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32713.log: /var/log/ceph/ceph-client.admin.36564.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32713.log.gz 2024-03-29T04:21:07.522 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36564.log.gz 2024-03-29T04:21:07.522 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35899.log 2024-03-29T04:21:07.522 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36616.log 2024-03-29T04:21:07.522 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38299.log: /var/log/ceph/ceph-client.admin.35899.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35899.log.gz 2024-03-29T04:21:07.523 INFO:teuthology.orchestra.run.smithi022.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38299.log.gz 2024-03-29T04:21:07.523 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34129.log 2024-03-29T04:21:07.523 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36114.log 2024-03-29T04:21:07.532 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36616.log: /var/log/ceph/ceph-client.admin.34129.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36616.log.gz 2024-03-29T04:21:07.532 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34129.log.gz 2024-03-29T04:21:07.533 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33554.log 2024-03-29T04:21:07.533 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39743.log 2024-03-29T04:21:07.533 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36114.log: /var/log/ceph/ceph-client.admin.33554.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36114.log.gz 2024-03-29T04:21:07.533 INFO:teuthology.orchestra.run.smithi022.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33554.log.gz 2024-03-29T04:21:07.533 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35925.log 2024-03-29T04:21:07.533 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39743.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39743.log.gz 2024-03-29T04:21:07.533 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-29T04:21:07.533 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32927.log 2024-03-29T04:21:07.534 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35925.log: /var/log/ceph/ceph.audit.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.35925.log.gz 2024-03-29T04:21:07.534 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-03-29T04:21:07.534 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32927.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32927.log.gz 2024-03-29T04:21:07.534 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36297.log 2024-03-29T04:21:07.534 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-29T04:21:07.534 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-mds.c.log: 91.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-03-29T04:21:07.534 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-29T04:21:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38535.log 2024-03-29T04:21:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36297.log: /var/log/ceph/ceph-mds.a.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36297.log.gz 2024-03-29T04:21:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36349.log 2024-03-29T04:21:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38535.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38535.log.gz 2024-03-29T04:21:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35586.log 2024-03-29T04:21:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36349.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36349.log.gz 2024-03-29T04:21:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36192.log 2024-03-29T04:21:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35586.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35586.log.gz 2024-03-29T04:21:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33924.log 2024-03-29T04:21:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36192.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36192.log.gz 2024-03-29T04:21:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38773.log 2024-03-29T04:21:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33924.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33924.log.gz 2024-03-29T04:21:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35404.log 2024-03-29T04:21:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38773.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38773.log.gz 2024-03-29T04:21:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36957.log 2024-03-29T04:21:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35404.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35404.log.gz 2024-03-29T04:21:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37891.log 2024-03-29T04:21:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36957.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36957.log.gz 2024-03-29T04:21:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39701.log 2024-03-29T04:21:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37891.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37891.log.gz 2024-03-29T04:21:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34126.log 2024-03-29T04:21:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39701.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39701.log.gz 2024-03-29T04:21:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37917.log 2024-03-29T04:21:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34126.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34126.log.gz 2024-03-29T04:21:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39262.log 2024-03-29T04:21:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37917.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37917.log.gz 2024-03-29T04:21:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32795.log 2024-03-29T04:21:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39262.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39262.log.gz 2024-03-29T04:21:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39291.log 2024-03-29T04:21:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32795.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32795.log.gz 2024-03-29T04:21:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39210.log 2024-03-29T04:21:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39291.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39291.log.gz 2024-03-29T04:21:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35296.log 2024-03-29T04:21:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39210.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39210.log.gz 2024-03-29T04:21:07.543 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39127.log 2024-03-29T04:21:07.543 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.2.35296.log: 91.7% -- replaced with /var/log/ceph/ceph-client.2.35296.log.gz 2024-03-29T04:21:07.543 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36512.log 2024-03-29T04:21:07.543 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39127.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39127.log.gz 2024-03-29T04:21:07.543 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37196.log 2024-03-29T04:21:07.543 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36512.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36512.log.gz 2024-03-29T04:21:07.543 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37800.log 2024-03-29T04:21:07.543 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37196.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37196.log.gz 2024-03-29T04:21:07.544 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36003.log 2024-03-29T04:21:07.544 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37800.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37800.log.gz 2024-03-29T04:21:07.544 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40005.log 2024-03-29T04:21:07.544 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36003.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36003.log.gz 2024-03-29T04:21:07.544 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40214.log 2024-03-29T04:21:07.545 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.40005.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40005.log.gz 2024-03-29T04:21:07.546 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37482.log 2024-03-29T04:21:07.546 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.40214.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40214.log.gz 2024-03-29T04:21:07.546 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37035.log 2024-03-29T04:21:07.546 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37482.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37482.log.gz 2024-03-29T04:21:07.546 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36800.log 2024-03-29T04:21:07.547 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37035.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37035.log.gz 2024-03-29T04:21:07.547 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40319.log 2024-03-29T04:21:07.547 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36800.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36800.log.gz 2024-03-29T04:21:07.548 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38483.log 2024-03-29T04:21:07.548 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.40319.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40319.log.gz 2024-03-29T04:21:07.549 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36460.log 2024-03-29T04:21:07.549 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38483.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38483.log.gz 2024-03-29T04:21:07.550 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27426.log 2024-03-29T04:21:07.550 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36460.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36460.log.gz 2024-03-29T04:21:07.551 INFO:teuthology.orchestra.run.smithi151.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-29T04:21:07.552 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38154.log 2024-03-29T04:21:07.552 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.27426.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.27426.log.gz 2024-03-29T04:21:07.552 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33871.log 2024-03-29T04:21:07.553 INFO:teuthology.orchestra.run.smithi151.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-29T04:21:07.553 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38154.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38154.log.gz 2024-03-29T04:21:07.554 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36878.log 2024-03-29T04:21:07.554 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33871.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33871.log.gz 2024-03-29T04:21:07.555 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37066.log 2024-03-29T04:21:07.555 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36878.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36878.log.gz 2024-03-29T04:21:07.556 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35821.log 2024-03-29T04:21:07.556 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37066.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37066.log.gz 2024-03-29T04:21:07.557 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36748.log 2024-03-29T04:21:07.557 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35821.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35821.log.gz 2024-03-29T04:21:07.558 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37664.log 2024-03-29T04:21:07.559 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36748.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36748.log.gz 2024-03-29T04:21:07.560 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36323.log 2024-03-29T04:21:07.560 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37664.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37664.log.gz 2024-03-29T04:21:07.560 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34338.log 2024-03-29T04:21:07.561 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36323.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36323.log.gz 2024-03-29T04:21:07.562 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35372.log 2024-03-29T04:21:07.562 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34338.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34338.log.gz 2024-03-29T04:21:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37534.log 2024-03-29T04:21:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35372.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35372.log.gz 2024-03-29T04:21:07.564 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39075.log 2024-03-29T04:21:07.565 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37534.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37534.log.gz 2024-03-29T04:21:07.565 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33819.log 2024-03-29T04:21:07.566 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39075.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39075.log.gz 2024-03-29T04:21:07.566 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33580.log 2024-03-29T04:21:07.567 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33819.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33819.log.gz 2024-03-29T04:21:07.567 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38243.log 2024-03-29T04:21:07.568 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33580.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33580.log.gz 2024-03-29T04:21:07.568 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34339.log 2024-03-29T04:21:07.569 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38243.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38243.log.gz 2024-03-29T04:21:07.570 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33845.log 2024-03-29T04:21:07.570 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34339.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34339.log.gz 2024-03-29T04:21:07.570 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39823.log 2024-03-29T04:21:07.571 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33845.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33845.log.gz 2024-03-29T04:21:07.572 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37118.log 2024-03-29T04:21:07.573 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39823.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39823.log.gz 2024-03-29T04:21:07.573 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35151.log 2024-03-29T04:21:07.573 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37118.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37118.log.gz 2024-03-29T04:21:07.574 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33663.log 2024-03-29T04:21:07.575 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.35151.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.35151.log.gz 2024-03-29T04:21:07.576 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37378.log 2024-03-29T04:21:07.576 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33663.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33663.log.gz 2024-03-29T04:21:07.577 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36720.log 2024-03-29T04:21:07.577 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37378.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37378.log.gz 2024-03-29T04:21:07.578 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35508.log 2024-03-29T04:21:07.578 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36720.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36720.log.gz 2024-03-29T04:21:07.579 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34783.log 2024-03-29T04:21:07.580 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35508.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35508.log.gz 2024-03-29T04:21:07.580 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35951.log 2024-03-29T04:21:07.581 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34783.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34783.log.gz 2024-03-29T04:21:07.581 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37508.log 2024-03-29T04:21:07.582 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35951.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35951.log.gz 2024-03-29T04:21:07.583 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34651.log 2024-03-29T04:21:07.583 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37508.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37508.log.gz 2024-03-29T04:21:07.584 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32635.log 2024-03-29T04:21:07.584 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34651.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34651.log.gz 2024-03-29T04:21:07.585 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36055.log 2024-03-29T04:21:07.585 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32635.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32635.log.gz 2024-03-29T04:21:07.586 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39901.log 2024-03-29T04:21:07.586 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36055.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36055.log.gz 2024-03-29T04:21:07.587 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40058.log 2024-03-29T04:21:07.588 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.39901.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39901.log.gz 2024-03-29T04:21:07.588 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33107.log 2024-03-29T04:21:07.589 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.40058.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40058.log.gz 2024-03-29T04:21:07.589 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36434.log 2024-03-29T04:21:07.590 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33107.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33107.log.gz 2024-03-29T04:21:07.590 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37944.log 2024-03-29T04:21:07.591 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36434.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36434.log.gz 2024-03-29T04:21:07.592 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37248.log 2024-03-29T04:21:07.593 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37944.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37944.log.gz 2024-03-29T04:21:07.593 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33371.log 2024-03-29T04:21:07.594 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37248.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37248.log.gz 2024-03-29T04:21:07.594 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38892.log 2024-03-29T04:21:07.595 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.33371.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33371.log.gz 2024-03-29T04:21:07.596 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36694.log 2024-03-29T04:21:07.598 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.38892.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.38892.log.gz 2024-03-29T04:21:07.598 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32739.log 2024-03-29T04:21:07.599 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36694.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36694.log.gz 2024-03-29T04:21:07.599 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37326.log 2024-03-29T04:21:07.600 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.32739.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.32739.log.gz 2024-03-29T04:21:07.600 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36538.log 2024-03-29T04:21:07.601 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37326.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37326.log.gz 2024-03-29T04:21:07.601 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40295.log 2024-03-29T04:21:07.602 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36538.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36538.log.gz 2024-03-29T04:21:07.603 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34757.log 2024-03-29T04:21:07.604 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.40295.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40295.log.gz 2024-03-29T04:21:07.604 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40240.log 2024-03-29T04:21:07.605 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.34757.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34757.log.gz 2024-03-29T04:21:07.605 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35201.log 2024-03-29T04:21:07.606 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.40240.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40240.log.gz 2024-03-29T04:21:07.608 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.35201.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35201.log.gz 2024-03-29T04:21:07.721 INFO:teuthology.orchestra.run.smithi022.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-29T04:21:07.861 INFO:teuthology.orchestra.run.smithi022.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-29T04:21:08.137 INFO:teuthology.orchestra.run.smithi151.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-29T04:21:08.192 INFO:teuthology.orchestra.run.smithi151.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-29T04:21:08.214 INFO:teuthology.orchestra.run.smithi151.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-29T04:21:08.215 INFO:teuthology.orchestra.run.smithi151.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-29T04:21:08.216 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-03-29T04:21:08.217 INFO:teuthology.orchestra.run.smithi151.stderr:real 0m0.818s 2024-03-29T04:21:08.217 INFO:teuthology.orchestra.run.smithi151.stderr:user 0m3.310s 2024-03-29T04:21:08.217 INFO:teuthology.orchestra.run.smithi151.stderr:sys 0m0.167s 2024-03-29T04:21:08.274 INFO:teuthology.orchestra.run.smithi022.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-29T04:21:08.277 INFO:teuthology.orchestra.run.smithi022.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-29T04:21:08.466 INFO:teuthology.orchestra.run.smithi022.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-29T04:21:08.570 INFO:teuthology.orchestra.run.smithi022.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-29T04:21:08.980 INFO:teuthology.orchestra.run.smithi022.stderr: 97.4% -- replaced with /var/log/ceph/ceph-client.0.38693.log.gz 2024-03-29T04:21:08.981 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-03-29T04:21:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:real 0m1.611s 2024-03-29T04:21:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:user 0m6.211s 2024-03-29T04:21:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:sys 0m0.269s 2024-03-29T04:21:08.982 INFO:tasks.ceph:Archiving logs... 2024-03-29T04:21:08.982 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629086/remote/smithi022/log 2024-03-29T04:21:08.984 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-29T04:21:09.852 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629086/remote/smithi151/log 2024-03-29T04:21:09.853 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-29T04:21:10.331 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-29T04:21:10.341 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-29T04:21:10.341 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-29T04:21:10.344 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-29T04:21:10.427 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-29T04:21:10.437 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-29T04:21:10.437 DEBUG:teuthology.orchestra.run.smithi022:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T04:21:10.440 DEBUG:teuthology.orchestra.run.smithi151:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T04:21:10.452 INFO:teuthology.orchestra.run.smithi022.stdout: remote refid st t when poll reach delay offset jitter 2024-03-29T04:21:10.453 INFO:teuthology.orchestra.run.smithi022.stdout:============================================================================== 2024-03-29T04:21:10.453 INFO:teuthology.orchestra.run.smithi022.stdout:*hv01.front.sepi 44.4.53.2 2 u 46 64 177 0.077 -0.586 0.951 2024-03-29T04:21:10.453 INFO:teuthology.orchestra.run.smithi022.stdout:+hv02.front.sepi 63.231.80.2 3 u 51 64 177 0.083 +1.948 0.814 2024-03-29T04:21:10.453 INFO:teuthology.orchestra.run.smithi022.stdout:+hv03.front.sepi 74.6.168.72 3 u 44 64 177 0.095 -1.917 1.086 2024-03-29T04:21:10.453 INFO:teuthology.orchestra.run.smithi022.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-29T04:21:10.455 INFO:teuthology.orchestra.run.smithi151.stdout: remote refid st t when poll reach delay offset jitter 2024-03-29T04:21:10.455 INFO:teuthology.orchestra.run.smithi151.stdout:============================================================================== 2024-03-29T04:21:10.455 INFO:teuthology.orchestra.run.smithi151.stdout:*hv01.front.sepi 44.4.53.2 2 u 46 64 177 0.073 +1.139 1.180 2024-03-29T04:21:10.455 INFO:teuthology.orchestra.run.smithi151.stdout:+hv02.front.sepi 63.231.80.2 3 u 47 64 177 0.077 +1.118 0.922 2024-03-29T04:21:10.455 INFO:teuthology.orchestra.run.smithi151.stdout:+hv03.front.sepi 74.6.168.72 3 u 48 64 177 0.090 -0.486 0.940 2024-03-29T04:21:10.456 INFO:teuthology.orchestra.run.smithi151.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-29T04:21:10.456 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-29T04:21:10.465 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-29T04:21:10.466 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-29T04:21:10.503 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-29T04:21:10.514 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-29T04:21:10.524 INFO:teuthology.task.internal:Duration was 872.949778 seconds 2024-03-29T04:21:10.525 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-29T04:21:10.537 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-29T04:21:10.537 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-29T04:21:10.540 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-29T04:21:10.580 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-29T04:21:10.580 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi022.front.sepia.ceph.com 2024-03-29T04:21:10.580 DEBUG:teuthology.orchestra.run.smithi022:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-29T04:21:10.632 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi151.front.sepia.ceph.com 2024-03-29T04:21:10.632 DEBUG:teuthology.orchestra.run.smithi151:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-29T04:21:10.643 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-29T04:21:10.643 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T04:21:10.675 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T04:21:10.711 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-29T04:21:10.711 DEBUG:teuthology.orchestra.run.smithi022:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-29T04:21:10.747 DEBUG:teuthology.orchestra.run.smithi151:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-29T04:21:10.809 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-29T04:21:10.819 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-29T04:21:10.820 DEBUG:teuthology.orchestra.run.smithi022:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-29T04:21:10.831 DEBUG:teuthology.orchestra.run.smithi151:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-29T04:21:10.842 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-29T04:21:10.853 DEBUG:teuthology.orchestra.run.smithi022:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-29T04:21:10.876 DEBUG:teuthology.orchestra.run.smithi151:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-29T04:21:10.885 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = core 2024-03-29T04:21:10.894 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = core 2024-03-29T04:21:10.904 DEBUG:teuthology.orchestra.run.smithi022:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-29T04:21:10.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:21:10.942 DEBUG:teuthology.orchestra.run.smithi151:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-29T04:21:10.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:21:10.952 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-29T04:21:10.969 INFO:teuthology.task.internal:Transferring archived files... 2024-03-29T04:21:10.970 DEBUG:teuthology.misc:Transferring archived files from smithi022:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629086/remote/smithi022 2024-03-29T04:21:10.970 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-29T04:21:11.008 DEBUG:teuthology.misc:Transferring archived files from smithi151:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629086/remote/smithi151 2024-03-29T04:21:11.008 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-29T04:21:11.028 INFO:teuthology.task.internal:Removing archive directory... 2024-03-29T04:21:11.028 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-29T04:21:11.048 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-29T04:21:11.073 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-29T04:21:11.092 INFO:teuthology.task.internal:Not uploading archives. 2024-03-29T04:21:11.093 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-29T04:21:11.115 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-29T04:21:11.115 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-29T04:21:11.118 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-29T04:21:11.122 INFO:teuthology.orchestra.run.smithi022.stdout: 658626 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 29 04:21 /home/ubuntu/cephtest 2024-03-29T04:21:11.125 INFO:teuthology.orchestra.run.smithi151.stdout: 658638 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 29 04:21 /home/ubuntu/cephtest 2024-03-29T04:21:11.126 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-29T04:21:11.139 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-29T04:21:11.171 INFO:teuthology.nuke:Checking targets against current locks 2024-03-29T04:21:11.201 DEBUG:teuthology.nuke:shortname: smithi022 2024-03-29T04:21:11.202 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T04:21:11.224 DEBUG:teuthology.nuke:shortname: smithi151 2024-03-29T04:21:11.225 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T04:21:11.231 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629086', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 03:57:02.836702', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKCXo6Fu6f+uCk70spbH9SnoOQpq10bOkMvJsyVbHpskB9nuO/VnU7ECSDvwyeyjtUXCiDdkBiSlBW101K0GTAo='} 2024-03-29T04:21:11.247 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629086', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 03:57:02.835208', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHDWNwZHqvmHe//FSX8MiigxaJK++ZfMmopww64yeuTn8TzDtScnW51x2HSUHK62BijHfDGyhbfDRhgiE8FiUoo='} 2024-03-29T04:21:11.281 INFO:teuthology.orchestra.console.smithi022:Power off 2024-03-29T04:21:11.282 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-29T04:21:11.308 DEBUG:teuthology.orchestra.console.smithi022:power off output: Chassis Power Control: Down/Off 2024-03-29T04:21:11.308 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:21:11.331 INFO:teuthology.orchestra.console.smithi151:Power off 2024-03-29T04:21:11.331 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-29T04:21:11.342 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2024-03-29T04:21:11.359 DEBUG:teuthology.orchestra.console.smithi151:power off output: Chassis Power Control: Down/Off 2024-03-29T04:21:11.359 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:21:11.385 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is on 2024-03-29T04:21:15.344 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:21:15.387 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:21:15.456 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2024-03-29T04:21:15.498 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is on 2024-03-29T04:21:19.458 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:21:19.499 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:21:19.569 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off 2024-03-29T04:21:19.609 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is on 2024-03-29T04:21:19.670 INFO:teuthology.orchestra.console.smithi022:Power off completed 2024-03-29T04:21:23.610 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:21:23.721 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is off 2024-03-29T04:21:23.822 INFO:teuthology.orchestra.console.smithi151:Power off completed 2024-03-29T04:21:23.927 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/uninlining} duration: 872.9497776031494 failure_reason: 'Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining)' flavor: default owner: scheduled_mchangir@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=2ae18df486394228bbbfb3b1d334fa2c status: fail success: false 2024-03-29T04:21:23.927 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-29T04:21:24.008 INFO:teuthology.run:FAIL